(--- PHASE 2 of the abstractionless design, PHASE 1 was abstractionless with overrides ---)

BOAR: based loosely on the previous abstractionless programming design (KNOFLOOK), backus FP, and my visual FP design
(Kartoffel, http://strlen.com/proglang/pix/hovpl.gif)

biggest issue with the previous design was sharing overrides (to accommodate for formal parameters, effectively).
There are 2 kinds of languages that can do away with formal parameters: stack languages and FP. Of those, FP
is cognitively the easiest to keep track of values.

FP is purely compositional, so the abstractionless idea can apply using sharing only, and making everything rather
trivial.

FP, in summary:
Every language element is a function that takes a single list as input and a single list as output.
A number takes any list as input and a list with a single element (the number) as output.
Many functions that are fixed number of args in most languages are variable here, e.g. + is really sum.
A sequence is simply many functions concatenated.
A list (as syntax) distributes its input list over all elements (functions), its result being a concat of all result lists.
Lists can be nested of course, so a function like qsort can return its list in a single element list to ensure correct concat.

This, in code, forms a tree structure where every level is either a sequence or a list. Rather than using an actual tree,
it can be layed out in block format similar to http://strlen.com/proglang/pix/hovpl.gif
the syntax would be different in that a list is made to look like a table, almost like a mini excel sheet, and sequences
need no graphical representation.

Orientation of sequences and lists can be either horizontal or vertical, user switchable. There is no ideal auto layout,
because there are two heuristics for this which contradict eachother: maximize space usage, and making tree levels as
"square" as possible (to fit on screen at once).

Selection would be excel-like too, in that dragging in an unselected area selects a set of items, and dragging a selected
item inserts a (shared) copy at the destination. You can share subsets of a sequence, but not of a list.

Sharing is fully automatic, so the default case is no redundancy. You can unshare manually (clone), but that only unshares
the current tree level, and leaves children shared. This minimizes redundancy, and makes unsharing an as explicit action
as possible. Lazyness/beginner style can't create redundancy.

Dragging always insert into a sequence/list, and it shows visually (using a colored bar) where the item(s) will end up.
There is no way to drag on top of an existing item to replace it, since the UI will be quicker and more intuitive with
insert/delete being seperate actions. To be exact, a list or sequence of N elements has N+1 insertion points, any single
element in a sequence has 2 additional insertion points on the opposite direction of the sequence it is in (creates a list),
and a list element has 4 (create a sequence or nested list). Deleting an element removes the sequence or list if it drops to size 1.
Sharing can potentially create a "nested sequence" unlike normal ops, and allows for insertion just inside or just outside the
sharing boundary.

You can label any shared tree. We are stopping people designing abstractions, not using them!
Note that you can't label unshared trees, that be overabstraction. If a shared tree becomes unshared, the system will remember
the name but hide it in the display, and use it once it becomes shared again.
Any labelled tree can be folded manually to simplify screen representation. Zoom/unzoom operations allow you to use the
entire screen for a subtree (shortcuts like ctrl + cursor keys could make parent/child/sibling nav very fast).
Shrink/unshrink operations determine the level at which children automatically get folded
(this remembers/respects any manual fold/unfolds). Using zooming, shrinking & folding together you can manage how big
a chunk of code you look at at once.

new operations can either be dragged/copy pasted from existing code, dragged from a window/menu of operations, or typed on
the keyboard.

Accessor names can be defined for certain indices in a list, allowing lists to be used as structs more comfortably. 
Accessors are dynamically scoped, i.e. they will read/write to whatever list that has that field that is its closest
predecessor in the sequence or its parents. This will allow for environments, easier use of objects, and even globals,
since without this it be purely functional. [FIXME, work out how this works more exactly]
Additional data types like hashtables will be present.
Nested lists can be represented as a 2D grid, allowing it to be used as a fancy excel replacement.
Display boxes show the last results of computations inline with code. This is nice for simple programs
or spreadsheet style usage, but can also be used for debugging etc. 

Would probably use dynamic typing with type inference


Examples in ascii...

// spreadsheet like: average of numbers:

[1,2,3,4,5] [+,len] /


max = < ? second : first

//(recursion requires the label, maybe do it differently)
//(can we avoid uses of "first" where its just a pass thru?)
fac = nil
    ? 1
    : [first, [first, 1] - fac] *

//(definitely want operators + constants as single functions, so we can replace "[first, 1] -" by "first -1"
//(shouldn't really be "first", as it simply wants the whole input

fac = nil
    ? 1
    : [#, # -1 fac] *

//(# is not needed, doing nothing just brings input to output)
//(the empty element will look better in graphics)

fac = nil
    ? 1
    : [ , -1 fac] *


//(hmm this looks much worse in ascii than it does in graphics)
append = first nil
       ? second
       : [first [head, tail], second] [head, tail append] cons

append = first nil
       ? second
       : [first head, [first tail, second] append] cons

//(# means whole input?)
//(F = filter, M = map higher order functions)
//(using concat here rather than append since its built in to the language!)
qsort = nil
      ? null
      : [#, head] [F(<), F(==), F(>)] M(qsort) concat



======

The biggest thing holding FP back is its excessive use of selectors (head, tail, first, second, ...).
Effectively, many steps in code involve combining a bunch of values in a list that do not necessarily have
a relation, only to be taken apart many times over.

Now in the textual case, we could make this more readable by allowing lists to mimic "structs", with
the ability to give elements meaningfull names, e.g.:

	struct person = [ name, age, location ]

and then instead of (to print the location):

	["harry", 28, "tokyo"] third print

the more readable:

	person["harry", 28, "tokyo"] location print

The problem of course for a visual version is that defining data structures is abstraction, which we don't want,
and also it seems "polution" to have to define data structures just for intermediate values.

The more natural solution that fits with the no abstraction / copy paste paradigm, is to allow people to tag
list elements with names wherever they want.

These names will then automatically propagate: i.e. a "third" op will automatically be displayed as "location"
if it refers to a list element with a tag, even though internally it is still "third".

The names can easily be copy pasted just like anything else, i.e. I can do a "paste format" or something, to make
a list conform to the tags used elsewhere.

The system will keep track of the name combinations (there are no official "structs", just tags that happen to
occur in certain sequences) that occur most frequently automatically, which can then be used with a quick popup to
create a list that conforms to some previous format.

This system is then mostly an annotation system that doesn't affect code at all, but can be used to make it more
readable and show structure. The good thing is that you can do the annotating even AFTER you created the code
(as opposed to abstraction, which makes you think of structure BEFORE you create code).


(functions have to be allowed to use tags as selectors even though the input list may have them as different
offsets, so we can't simply represent them as "third" internally).



===================================================================

for the code browsing, use the supreme commander UI... infact, don't even have scrollbars at all,
zoom in & out at a nice log scale, and center around the cursor like SC.

This would make looking at large 2d planes of code a pleasure rather than a burden!
Much like supreme commander, it would have summary text replace the actual code when you zoom out so far
that its not recognizable anymore

Maybe should do the whole interface in OpenGL? Yup, won't be smooth enough with a 2D API. Still can be
in a window with normal menus/icons, and additional panes/windows in classical UI.

does having infinite space and screensize not being such a constraint any more mean we can maybe
always go with the heuristic of using as little space as possible as opposed to being as square as possible?
Problem: smallest space is usually achieved in a single flat line. Maybe some sort of intermediate form, i.e.
combining a 6x1 and a 1x1, smallest wins, but combining a 6x1 and a 4x1, square wins? That sounds really good.
Each node can be set to HORIZ/VERT/AUTO, with AUTO the default. Since you are comparing which of smallest/squarest
has the lowest "error metric", you can provide a slider that allows the use to change the breaking point between
the two.

for lists, for westerners, we want left->right & top->bottom.
For sequences, right->left gives more of a tree view, left->right is more "order of execution" or timeline-like.
bottom->top is slightly better for trees, but again top->bottom is more execution like.
Might have to make this configurable, but seeing other people use different orderings is going to look really confusing
and not good for the language (in screenshots etc, the env can easily adapt of course).
It would also be good to have lists and sequences in the same order


=========================================================================
=========================================================================
(--- the problems that transition PHASE 2 into PHASE 3 ---)

dataflow, stack machines, FP and variable based languages all have different ways of dealing with sharing of values.

values are characterized by how often they are used: 1..N (0 can be ignore for the discussion), and how these
uses are spread out (in sequencing). A more minor factor is ordering of multiple input values.

DF: treats uses and spread indifferently, but pays for it with tangled lines
ST: heavily biased to 1 use, pays for multi use and spread
FP: works best with 1 use, many structures of multi use work easy enough, but not all. pays for spread worse than ST
VL: works best with 1 use, pays for multiuse with var decls. spread is not a big issue. unlike the other 3 however, not compositional!

spread can be defined more formally:
for a function with N inputs, and M operations, sort the operations by dependencies. Rank 1 has all ops that only depend on inputs,
rank 2 on rank 1 + inputs, and so forth. "spread" is then defined as how many ranks back your furthest input is at, so spread 1
means only depends on the rank just before it.

spread 1 usually entails use 1 of its args, though doesn't have to be. If you only allow 1 op per rank (full ordering), then
spread 1 would entail use 1. Ordering can screw things up still but it usually is less important.

Just because a value has a spread >1 doesn't mean it has to have use>1. The spread can be caused by a second arg dependency.

So in essence, the set of values grows with the outputs of every step at ever rank. It shrinks with the removal of all last uses
at a particular rank.

for the set, at any particular rank, theres 3 kinds of values (in order of difficulty):
- those that are consumed here (last use)
- those that are used but passed on (middle use)
- those that are just passed on (in between uses)

FP gives the whole set (at a particular level) to each op in a rank.
Most ops now need to use selectors to pick out their args from the set.
Ops consume, so all non-last uses from the set need additional pure selection ops just to pass things on.
That's a lot of selection ops. The advantage being that if the selection ops get good names, that's a lot of
help in seeing where values are flowing. Naming while still being compositional!
Still, lots of names/selectors is not ideal.

An alternative in some cases to giving the set to all ops, is to distribute the set to the ops 1:1 (which
is non always clear in the case of multi-arg ops but could probably still be done). This would save a lot
of selectors in the case where the entire set is use 1 (consumed or passed thru), basically at a point in
the function where all middle uses have already happened.

in FP some of this optimisation is possible by allowing a rank 2 op to hierarchically be attached to a rank 1
op if thats the only think it needs, avoid a selector on the whole of rank 1 set. But the fuzziness starts
with spread, where more than 1 implementation is possible:

f(a, b) { W(Z(X(a),b),Y(b)) }

[a X, b, b Y] [take2 Z, third] W		// set style

[a X, b Y, b] [[first, third] Z, second] W	// set style, args not ordered right because of other uses

[[a X, b], b Y] [first Z, second] W		// hierarchical

[a X, b] [Z, second Y] W			// lazy ranks! wait with using b until necessary


W-Z-X-a
   -b
 -Y-b

There being so many different ways to encode (work around) the same dataflow is not good.
If you go lazy (work back from the output / last rank back to the first), does that always give
a single representation, and the simplest one?
Appears so on first sight... the bottom up version of append is also easier to read and shorter.

Also with bottom up... if you reverse the order of FP, which completely preserves its meaning,
you make it look less stack machine like, and read more like an exp or tree!

W [Z, Y second] [X a, b]

append = nil first
       ? second
       : cons [head first, append [tail first, second]]

that almost reads like the haskell version!

fac = nil
    ? 1
    : * [ , fac -1]

not much difference there

/ [+,len] [1,2,3,4,5]

here, first thinking of the data and then the operations of it was kinda nice. but this order is palatable too.

 

what's maybe worrying about making it look like an exp, is that people might want to use it like an exp, e.g.:

W [Z [X a, b], Y b]

which actually works!
conclusion: FP makes you write terrible code
 
This always works as long as every leaf is:
- a selector
- a literal
- an op using the entire list (all args) as-is
of whatever the input at the root of the tree is

So you can write FP as a classical expression tree, except that its more
than a tree, in the sense that you can sequence trees with other trees.
Any leaves of any sub tree select elements of the tree before it.
So it is still useful to do it graphically as boxes, otherwise that aspect would not be readable.
Boxes also help with large discrepancies in sizes in 2 boxes to visually "find" stuff

This "sequenced tree" format is exactly what makes the expression tree compositional while
still modelling functions and arguments: essentially, in an expression, arguments are at the leaves,
and in the abstractionless idea, this would mean their actual values. Which required the kludge
of overrides to be able to share only the in-between part. So all this does is seperate function
& args into 2 mappable trees, such that the function becomes an independent block. 
Everything is a function... from 1 arg to 1 result... hurray.

Also, the sequencing effectively merges local var defs and function defs. A function body that
classically would be split up with local var defs now becomes a sequence of smaller functions!

The 1 list idea gives us:
- elegant pass-thru of all args
- elegant ops working on any number of args (sum etc)
- elegant merging of multiple return codes
- making "list of args" and "list construction" syntactically and semantically the same

it IS possible to implement this language idea without the 1 list idea, but that would
lose the above advantage, most notably the syntax and visual editing would be convoluted
with a third type of list (sequence, list construction, list of args). Advantage would be
slightly better argument size error checking, but we can infer formats or arg list in
almost all cases probably anyway.


=====================

issues still:

flattening of lists.
either can make this [headtail, 1] return a list of 3 args (headtail returns a list of 2 values), or 2.
if the former, it means we need a dummy list around returning actual lists as a single value, i.e. [headtail, [ ]]
if the latter, we'd need an easy way to specify when things do get flattened, i.e. [flatten headtail, ]
the problem with the former is that single element lists are supposed to be equivalent to just single elements..
though that is still kinda logical because the single element list gets flattened to a single element, but again
is part of a list. Whereas in the latter, single elements are really always lists just represented as single.
Though the former is mildly more elegant list wise, but also less elegant because you can't visually see the number
of elements from a list construct, it really comes down to, which happens more often, returning multiple values, or
returning a list a single return value. I am leaning towards the latter


some form of state.
The current language is 100% functional, which is neat, but not good enough for a friendly language.

for just globals, can simply have a global operator:

global 1

if this tree is shared, it derives its identity from the code. Its starting value is 1, and of course can be labelled.
what syntax to assign to it?

assign [global 1, 2]

?

function values: all higher order function blocks sofar are not really first class.
We can either stick to using a good set of builtin higher order functions, but since
we have these great building blocks, we might as well allow them to be first class.
All that's needed is some form of quote. So

F(<)

(filter, from qsort above) becomes:

F '<

Hmm... this cannot quote a sequence however... visual syntax may need to be a box?

F {<}

or rather

F [{<}, [1,2,3,4,5], 4]

F = nil second
  ? null
  : (first ? third : cons [second, third])
    [nil call [first, head second, third], head second, F [first, tail second, third]]

This doesn't really do proper free vars, in that the third value is whatever contextual vars the functions needs,
but its not too bad either. You could give any {} its input on creation, this is quite natural:

F [{<}, [1,2,3,4,5]] 4

4 is the input to {} here. "call" would maybe then always supply this as first arg or so?


====================================================================================

(--- here starts PHASE 3 of the design ---)

so FP based VPL has several problems:

- a plumbing problem. as shown above, even for simple examples the number of different plumbings are quite a lot,
  with many suboptimal variants, meaning that beginning programmers can often create suboptimal code with many more
  accessors and constructors than needed. Various plumbing alternatives also affect readability.
  Ideally the language should not depend on expert plumbing to be good.
  Forth also has a plumbing problem (permutations of dup/swap/rot/pop), and C to some extend as well (local var use),
  besides the fact that it is non-compositional.
  The only one that doesn't have any of these problems is... dataflow

- it is still rather unreadable for beginners.. from best to worst, C, dataflow, FP, forth.

- it is potentially a lot of work connecting a far away source of data to where its needed, because of the manual
  plumbing. With forth this is also a distaster, and in C/dataflow this is relatively easy
  
All these can be improved together in the same way:

using a language that has underlying semantics of dataflow, can be edited as dataflow, but is represented/has automatic layout
that is a mix between dataflow and FP, i.e. automatically finding the sharing points and thus cutting up dataflow to remove crossing lines.

The algorithm would start at the output of the graph, and then create as much of a tree as possible, not including shared nodes.
Then, any shared nodes would be analyzed as what subtree they span, and attached as sequence to that subtree. 

A-B-C  -H
   -D  -H
 -E-F  -I
   -G  -I
   
so shared node H would be attached at the level of B, and I to E. 

Complications arise when some shared nodes overlap a bigger subtree than others, for example:

A-B-C  -H
   -D  -H
       -J
 -E-F  -I
   -G  -I
       -J

so J would attach to A here.
This can be dealt with in two ways:

1) keep H and I local to B and E, respectively, but both get a "pass-thru" link for J, which is attached to the whole tree after H and I
2) make H, I and J all local to A at the same level

I am tempted to think 2) is better, simply because long pass-thru links could get very unreadable. Of course, 2) makes stuff less local
than it can be, making the disconnected sharing harder to read.

though, what happens if people use values from very far away all over the place? this could make the graph a mess. It may have to be 1),
simply because that is what you'd do manually in FP as well.
 
-------------------------------------

one issue is sharing, if you allow an arbitrary subgraph of dataflow to be copy-pasted, then the resulting sub-box may have multiple outputs,
and the above dataflow-FP convertion only doesn't have crossing lines, assuming every box has 1 output (tree layout).
This could be solved by:
- forcing any multiple output node to be effectively a shared node
- having a tuple/list construct still, so that multiple outputs are converted to a tuple automatically. But to split up a tuple,
  you need to apply multiple selectors to it, making it shared anyway.
so these solutions amount to the same anyway.

Another question is wether sometimes shared splits can be avoided, if one tree (set) has multiple outputs that correspond 1:1 to some
inputs. This would be a single list in FP. This could be either a special case, or maybe the tuple approach above has merit, if
a tuple can automatically be taken from outputs and stuffed into inputs.


-----------------

so in terms of (re)factoring, dataflow is the ultimate refactoring language, since it has compositionality, and no plumbing. wow.
but how to make the equivalent of closures/macros for max code factoring work well in dataflow?

a = 5
map (_ + a) [1, 2, 3]

map(f, l) = l==nil ? nil
                   : cons(call(f, head(l)),
                          map(f, tail(l)))

how would a build in looping construct work, like, foreach?

most natural may be to make foreach out of 2 nodes: a splitter and a collector.

collect-closure-split-list
               -freevars

the question in tree format is where does the closure stop and freevars start... though I guess only the part of the tree directly
dependent on split is multiple-executed. This needs to be obvious somehow if any of freevars contains a side effecting operation.
We could just build a box with collect/split on the edges, but that requires people to think where to places boxes...

maybe there should be possible multiple splitters for one block, and filters, akin to list comprehensions...

but all that doesn't solve haveing a generic idea of a closure to pass to a function, or the fact that we don't even have functions...
In the abstractionless case, the user wants to copy some code where he needs to replace the middle, as opposed to the leaves.
so... we don't need closures? just a way to represent gaps in closed subgraphs. In dataflow, any subgraph can be folded into a node,
and thus shared. But what if a bunch of nodes are selected that happend to surround an unselected node? Sounds like that needs to
be supported. So if a user copies a chunk of code, pastes, makes it therefore shared, if he replaces parts of the code, that automatically
unshares/shrinks the shared part, even if the replaced part is in the middle.

If its in the middle, why should it become unshared rather than remain a shared change? The FP idea above was to keep things shared
by default, so any changes would be in both... you'd need to press a clone button to unshare manually any node. That is probably the best
approach.

How is such a graph represented internally?
In a sense, the surrounded unshared graph is both an input and an output... except dataflow to it is reversed. That doesn't help
representation any as that would also have to presented as a cycle when the shared area is folded.
worse, with replacement its possible to have another shared area with the unshared area!

S1-S2
  -U1-S3-U2-S2
     -U2

so S1-S2 "surrounds" the unshared bits, but S3 is completely disconnected from the rest of the sharing.

two possible approaches: either allow for a shared section to be any (disconnected) set of nodes, or split up
shared areas whenever middles get unshared. The first seems unpractical, since we want folding, and folded boxes
with internal unshared bits would need to show them inside the box somehow (we don't want cycles), and in the
above case that would be awkward because there is no easy way to show the relationship with S3 of that code.

So splitting up seems more logical. Any one shared area must be connected, such that it can be represented
as a folded box with ins and outs. If we had just:

S1-S2
  -U1-S2

(i.e. U1 was just inserted into a shared graph), then there would have to be a split because an output is also
an input (a cycle). How does this get detected/processed? Something along the lines of collecting nodes from the
root and splitting recursively.

what if such a gap graph get copied again? Then its a new shared area that includes everything, including sub-shared
areas. This is fine, sharing can be recursive.

What if a certain block is shared many times over, and only one copy removes one node from it? then that new smaller
areas should still be shared. This can cause some deep nesting of sharing, so nesting should have extremely lightweight
syntax.

What if someone copies too little, and later copies adjacent areas? This is only really an issue on the first copy,
as after that, the shared area will be a logical copy candidate. But yes, merging of touching shared areas that both
have the same usage locations would make sense.

So yes, this solves the typical code abstraction use. Copy-paste a loop, and change some of the internals. You'd get
two shared areas out of it probably, which funnily enough reflect the seperate split/collect areas. Use them again
and you could just select these 2 areas and copy paste em.

You can't name em/fold em as one, which is a shame. Maybe much like the builtin split/collect, there be a way to
graphically connect seperate blocks?
hmm.. much like split/collect could have some syntax automatically showing which nodes get multiple executed (part
of the the trail between the two), so could you have something similar for these shared areas... but somehow automatic?
of course if both end contain a split/collect, that's easy, but that won't always be so.

------------------------

Folded nodes could have a zoomable very small copy of the entire code inside them! So folding could be more
about shrinking, much like OverView.

Insertion points for pasting should be on lines, much like fp, and selection should be a paint-like thing much
like excel and above.

---------------------
graphical representation of the sharing "break":

there are some simple cases where there is no crossing lines... those should look like dataflow as much as possible.

for the more complex cases, can work with color. If red is always the first arg, then it will become easier to
match a red input with the first tree across the "break".

Traditionally shared vars are named vars... that helps identify what they are. but we don't want to get into the habit of
naming vars... automatic naming is required. This can be composed of:
* having each builtin op have names for its args, i.e. "cons" automatically can name its 2 links "head" and "tail"
* a representation of the connected tree, either as its root operation, or an entire subtree (can represent the entire
  tree by making it smaller and smaller in scale as links go on!)
* order color, as above
combining all of them, seeing a small red "tail:qsort" tag on an input helps much more keeping track

----------------------------

globals vs semi globals vs long links.

An easy way to do globals is to simply have named boxes that data can flow in and out of. Though what would be cool is
if those boxes could be local to a certain code scope, effectively making them dynamic vars.
Such boxes are also good because they reduce the amount of sharing going on.

But what of "global" values that are shared all over the place. They should not be copy pasted, because that would
create a new value, unless it was somehow "static". They could be connected thru a line, but that would create tremendous
long distance values. They could be put in globals, but why should a user decide something is to be a global? can we
somehow make this intuitively automatic?

Maybe we can make a singleton object box, that has the constructor code inside it. It can then be copy pasted anywhere,
and either be used as input or output. It can be named much like any shared block.

That's pretty good, still, what stops anyone from linking to it rather than copying it? It would have identical semantics.

(--- here starts PHASE 4 of the design ---)

linking vs copying? generally you copy when you want to provide different inputs, you link when want to use the value.
Even use of simple constants, people could link to it instead of copy. In both cases its shared, so same effect.

So if they're so similar, can't we use copying to replace linking automatically? so we wouldn't have breaks at all,
just shared areas. But that gets to be similar to original (non-FP) abstractionless idea... so why wouldn't this one
need "overrides"? Because it collapses the entire "function" to one node, and only that node is shared. Rather than assuming
that you'd want to share the entire "tree", you share a segment of the underlying dag. Replacing a leaf cuts up the shared
node as opposed to creating an override.

WOW. So the original override idea was created thinking too much from a tree/function_with_args
perspective, whereas thinking from a dag/FP/dataflow perspective makes it all simple again.

funnily enough the way we'd have to represent a shared subgraph is much the same as a shared tree with overrides.
they work great as args to higher order functions now, too!

What about boxes with multiple outputs. if I have:

A-[B-D]
 -C-[B-D]

and I want to reuse C-B, then the new block has two outputs going to A.

So here we really have 2 kinds of sharing: B, by virtue of being used twice, sharing the whole subtree that it builds up.
and C-B, which is a subgraph, with explicitly these nodes selected. Or whoever wants to reuse C-B can also just grab C, and
automatically get the whole subgraph with it until it is replaced (D). So the new tree is now:

A-[B-D]
 -[C-[B-D]]
E-[C-[B-D]]

so this means the multiple uses (outputs) of B isn't a problem, as they all have their own shared copy. So if you fold a box
with 2 outputs, it effectively ignores the output that's not going straight to the current parent, and shares that seperately!

so what now if D gets replaced in E, as was always the intention:

A-[B]-[D]
 -[C-[B]]-[D]
E-[C-[B]]-F

This forces B (and D) into a seperate shared area. Why not this instead:

A-[[B]-D]
 -[C]-[[B]-D]
E-[C]-[B]-F

That would result if the moment you shared the tree with C at the root, the shared area created is not the entire subtree, but
the subtree without any shared children (they are already shared). Which one is preferable? they are identical in complexity,
and what happens if you keep on sharing any of the subtrees is not that different. There should of course be intelligent rules for
merging of shared areas whenever possible. To implement the second option, you'd have to be careful that if you started sharing
a tree like A-[B]-C, that you'd make both copies [A]-[B]-[C], not just [A]-[B]-C. Hmm, but this means if a particular value you
are using is a long string of these shared blocks, every use of it gets its own copy of the whole structure: --- which is not bad
from a usage point of view, but uses unnecessary memory. So it has the advantage of less nesting, and easier to identify functional
blocks, but if many of the block in the chain contain single ops, then an overal shared area would have been more useful.
We'll have to see how small blocks commonly get with random cut&paste programming, also, if we decide single ops don't need sharing,
we are definitely better off with the first solution.

Is there any point in making [B] a shared area by itself if its a single operator? Yes, since they all originate from a single
original op, and you may expect to be able to replace that in all occurrences at once. No maybe, since its clutter, and the cases
where this loss of sharing is an issue are not going to be a big deal.

examples from the original paper:

2 * 3 + 1
drag, becomes [ 2 * 3 ] + [ 2 * 3 ]
changing the 3 to a 4... according to what I wrote above, this then becomes [ 2 * ] 4 + [ 2 * ] 4, even though [ 2 * 4 ] + [ 2 * 4 ]
was the original plan. Which one should be default is still a hard question. Incase the second copy is off-screen, the former may be
more clear that you are making a local copy since the layout changes to put your child outside the tree...
Shared change is of course a good default to stop inefficient factoring. 
We also may have to base it on which of the two ops is more common. 
Maybe we can simply give 2 visual targets for dragndrop/copynpaste, much like we have to do for replacing the root of a shared item or
just the entire shared item? That would however give 3 (!) different options when replacing a shared node in a shared parent area:
* replace the contents of the shared area (on the root node)
* replace the shared area in-place in parent (on the shared boundary)
* replace the shared area and cut loose from the parent. (on the line to its parent)
a non shared node, or a node deeper inside a shared area has just 2 possibilities:
* replace the node, shared (on the node)
* replace the node and cut (on the line)
which is consistent usage with the 3 way case. Visually, the cut one can show the are unshared as you hover over to predict what you will do.
But this all assumes dragging. Instead, we may want (as originally designed above) a cursor that provides an insertion point. This can still work,
we can allow both drag, and single click placing a cursor, both showing visually where the new code goes.
This kinda works with the original idea of having insertion, i.e. if the new code you paste is really a function (i.e. has inputs), then the old code
gets connected to its end. If the new code is a closed tree or constant, then since there's no attachment points, it replaces the old code entirely.
Though, there's not a lot of point in having paint-select anymore, as you now select by grabbing a tree's root, and you always get children... but hmm,
if thats the default, then you'd always be replacing rather than inserting. So maybe you shift-click to select a particular range, as insert is bound
to be a less frequent op than replace? That sounds good enough. This may indicate that as in the A-B-D example above, if you create a new sharing
area from a node down and make em seperate shared areas from areas that were already shared, it would be easier to include UI features to select just
the first block. 
So selecting... you have up to 3 different locations to select for a paste, but for a copy, all 3 amount to the same thing... it will always take the
entire shared area if there is one (if not, it would see that its already shared upon creating the new share), and the line location is the same too.
not necessarily a problem.

So back to the original paper.
We now have:
[ 2 * 4 ] + [ 2 * 4 ]
because we replaced the 3 right on its node (as per the 2 options above).
Now we add an unshared 5, which with overrides gave us [ 2 * { 4 } ] + [ 2 * { 5 } ] but now will look more like [ 2 * ] 4 + [ 2 * ] 5 or rather
+-[*-2]
  [ --]-4
 -[*-2]
  [ --]-5
(which would look very understandable in graphics)
and the [ 2 * { 4 } ] + [ 2 * { 4 } ] + [ 2 * { 5 } ] example (from originally 3 shared 4's) would be
+-+-[*-2]
    [ --]-[4]
   -[*-2]
    [ --]-[4]
 ---[*-2]
    [ --]-5
or
+-+-[[*-2]  ]
    [[ --]-4]
   -[[*-2]  ]
    [[ --]-4]
 ---[*-2]
    [ --]-5
this last one of course. This indeed preserves the sharing of 4, and has no scope problems to deal with like the overrides...
If you had [ 2 * 4 ] + [ [ 2 * 4 ] + [ 2 * 4 ] ] could you make it into [ 2 * { 4 } ] + [ [ 2 * { 5 } ] + [ 2 * { 5 } ] ] where the two 5's are shared?
Nope its either all or just 1. This is where the going up a level for the override as suggest UI feature in the original paper would come in.
I guess such a feature could be added, or it could be a 4th selection position, but that all gets kinda hairy. In most of these cases it can be fixed
by modifying just one copy and then making a copy of the whole [ 2 * 4 ] over the other.


---------------

evaluation of shared blocks:

any shared leaf block can be evaluated just once, on first eval. A block with inputs has to be evaluated again and again, except when its inputs are some leaf
nodes, and it occurs again in compbination with the same leaf nodes... this may occur if we choose the second option in dealing with recursive sharing, or even
just because the user created it so. 

-----------------------

ascii rep that is closer to C:

you can represent shared nodes as expressions with gaps, which can then be the root for its args which fill in the gaps. We still need variables to denote the
sharing. [ 2 * { 4 } ] + [ 2 * { 5 } ] becomes:

a = 2*_
a(4)+a(5)

or more fun:

a = 5; a+a;

becomes:

(_+_)(5);

you could use __ for a second arg, and ___ for a 3rd. But now we're dealing with sharing in 2 ways, with vars and number of _. The real version is of course:

a = 5;
(_+_)(a, a);

that explicitly deals with sharing, much like it would in the vpl. But now the + is unshared so it collapsed back to the same as in C. Which is right, because
the original didn't have the + shared, but the 5.

(can even do curried, without _ this way though.)

so this example:

A-[B]-[D]
 -[C-[B]]-[D]
E-[C-[B]]-F

becomes:

b = B
d = D
c = C(b)

A(b(d), c(d))
E(c(F))

interesting syntax, since 'b' and 'c' are functions.

------------------------------

functional programming

the whole language is rather functional. We don't want people to have to use recursion, as that requires named blocks.
So the higher order function primitives have to cover ALL iteration needs. map/filter/fold.

so

map f [1, 2, 3]
fold + [1, 2, 3] 0
filter (<3) [1, 2, 3]
[ x*2 | x <- [1, 2, 3], x<3 ]
[ x*y | x <- [1, 2, 3], y <- [2, 3, 4], x<y ]

would become:

map-f-iterate-[1, 3, 3]
fold-+-iterate-[1, 3, 3]
      -accumulate-0
filter-<-iterate-[1, 3, 3]
        -3
map-*-filter-<-iterate-[1, 3, 3]
     -2       -3
map-*-filter-<-iterate-[1, 3, 3]  // doesn't work in this syntax
     -?       -iterate-[2, 3, 4]
filter-<-iterate-[1, 3, 3]
        -iterate-[2, 3, 4]
      -*-iterate-[1, 3, 3]		// this is superfluous for simple filters
        -iterate-[2, 3, 4]


factorial:

[ fold-*-iterate-generate-1 ]
[       -accumulate-1    -- ] 7

or rather:

[ 'product'-generate-1 ]
[                   -- ] 7

where product is a labelled folded shared tree

quicksort: no escaping recursion here

[ if-nil----------------------------------------- ]
[   -nil                                          ]
[   -append-append-qsort-filter-<-iterater-tail-- ]
[                                -head----------- ]
[                 -head-------------------------- ] [8, 3, 5, 2, 7, 1]
[          --------qsort-filter->=-iterater-tail- ]
[                                -head----------- ]

but at least it will look neat graphically.
Could define a recursion operator that functions much like higher order functions:

[ rec-if-nil------------------------------------------- ]
[       -nil                                            ]
[       -append-append-recurse-filter-<-iterater-tail-- ]
[                                    -head------------- ]
[                     -head---------------------------- ] rec_in [8, 3, 5, 2, 7, 1]
[              --------recurse-filter->=-iterater-tail- ]
[                                    -head------------- ]

This doesn't allow mutual recursion yet...

in the graphical version, "rec_in [8, 3, 5, 2, 7, 1]" is repeated 6 times.
-> we can still layout all cases we can as DAG, just to save space!

how to to a mandelbrot - needs a while with multiple vars looping around in it.
- can require it to be done using recursion
- can be done with a fold that has multiple accumulators and inputs
  gotta see if thats any simpler looking than the recursive case

------------------

state...

=====================================================

(--- here starts PHASE 5 of the design ---)

So, the problem with the PHASE 4 (and consequently, PHASE 1) design is that some sharing pattern can lead to tremendously big trees
before folding. Take some huge ass data structure (or even just a faily big one, like a list of several elements), a selector, and
then a bunch of ops using the outcome of that in multiple places. Unfolded code will BALLOON very quickly. Now you can fold
(or better: shrink), but that requires user organization, makes it harder to go look at the folded code, and generally doesn't help
you understand your code with all those folded bits allover that you don't see the contents of (unless they have VERY good identifiers
attached).

So, looking back at PHASE 3, the error in thinking was that dataflow links and sharing would be different, which of course they aren't.
They are both shared uses of some code block. Seeing them as the same, makes the option of showing shared trees at the greatest tree
level they depend on impossible, as it would push all shared trees global. So they'd have to use the "pass thru" style. But this makes
any use of more global trees totally unreadable.

So PHASE 3 seems impossible now, and PHASE 4 seems unweildy. What can we do?

Well, if we take the PHASE 3 idea, but instead of trying to layout all code on one plane, only layout things for the current tree
(context sensitive), things change tremendously:

you would see your currently focused tree, and the trees shown as shared inputs to it are simply the ones it refers to in order of how
close to the root they are. Any shared trees used by other shared trees are then also listed in order of root depth, i.e. a tree 1 deep
from a tree thats 2 deep from the root would actually be listed before a tree 4 deep from the root directly.

This would create on giant context sensitive list of trees of ALL code below the focus, all ordered to how likely the user will want to
look at them relative to the focus. This actually lends itself to a more traditional vertical display, where the focus is a tree at the
top followed by all other trees. each tree could be prefixed on the left by its label. A further column or row would show all parent
nodes in a list, so a single click to those would bring the focus to a parent. A doubleclick inside any of the nodes in the current
display would simply bring the focus there.

Navigation would be quick and code would be very overviewable, with all relevant stuff near
the top: the display would have scrollbars, but there would hardly ever be a need to scroll down since the further you go down, the less
relevant it becomes. If you want to navigate very deep, instead of scroll+doubleclick, probably 2 or 3 doubleclicks would do the same.
It would almost be worth considering not having scrollbars, and instead relying on this principle. Or there could be a slight fisheye
mechanism, with trees further down printed in smaller size, and the bottom row being names of trees without their content.

Which bring us to: labels/identifiers. Really, it is unwise for clarity to try and do away with them, the readability the bring is
tremendous. So for shared tree the system would generate automatic identifiers (a, b, c... ?) and then you could very simply rename
them. So for a programmer working with a particular focus, it would become a habit to rename shared near the top items, but he doesn't
need to if he doesn't care. When inlined because of loss of sharing, these would be hidden, or maybe still shown smaller next to tree
items (the ones explicitly named). 

Auto generated names should not overlap with explicit names, and they should be shown differently graphically. Maybe they can be
generated from root function name they refer to rather than a/b/c ?

Some trees maybe be shared, but in the current focus they could be a single use. Which means they could be shown inline. This may be
unwise however, since this would inline potentially large function bodies. So probably by default not. Or maybe this can depend on
size, or on the fact wether they are named or not.

Depending on what editing operations will look like, since the format of code has now become a list of trees, it is now more
attractive to layout a single tree as horizontal as possible, since the list takes up the vertical dimension. This means code can
be shown to look more closely like traditional code, certainly expressions with operators would gain a lot from this.
Depending on the size of the tree this can't and shouldn't be done fully however, since the line would become too long to be readable.
There can be an auto algorithm that splits a certain node into a vertical layout because its children are a certain size, and hopefully
that would be good enough, though maybe sometimes human preference can be specified.

Can have an additional list on screen that shows the last N places doubleclicked upon, like auto bookmarks! It can even filter out
double-doubleclicks easily.

One issue: with this system, unshared trees connected to shared trees (with gaps, i.e. functions) would graphically hang of that shared
tree. Which means you'd still need some display of what's shared and whats not, or you'd need to make those "arguments" trees by themselves
even though they are not shared. In either case, it could cause this code to be far away from the root, depending on the size of the
function body which you may not care for. Err no, they displayed as children of the tag of the shared tree inside the parent tree,
which is perfect.
-> no, it is still somewhat of a problem, because the function tree has no name for the child. in previous designs this would be
displayed simply as a child of the function body, since everything was always inline, but that connection is now lost.
There are 3 solutions:
- showing the function body inline, like in past phases. This requires folding, something we have happily done away with.
- showing the argument behind the function body rather than the main tree. This really decouples the arguments from the rest,
  and allows a large function body to obscure the code. Not acceptable either
- auto-generate a second kind of identifier. This identifier is there purely to show how many inputs the function block has, and
  which are shared. Funnily enough if an arg happens to be shared, it will have a name, but that name can't necessarily be used
  as function arg because the function may be used more than once even inside the focus.
  Since of these names there are a limited amount, they can go by numbers or a/b/c in some kind of predefined format (a graphical
  box/color, or caps etc.)
Now, a case which wasn't explicitly considered in the above designs, is what happens when you create such an arg in a function with
internal shared blocks. If you insert the arg and shrink the shared area, it is still inside the outer function, which maybe is 
what you want it to be outside of. So inserting a tree with a "cut" now must specify somehow how many levels deep you want to cut,
and each level will get args, and passes thru to the deeper levels. So maybe the cut has to be a seperate action so that you can
keep applying it, or a cut is always as deep as possible?

Ordering of trees may need to be first sorted on which tree is local to what, then how close to the root they are, otherwise small
shared tree part of some functions' implementation risk getting interleaved with random stuff and not easy to find. Indentation
should be used to show locality.

what about the fact that all tag names are global? you'd soon run out of simple names. Should also use locality here somehow, though
of course the problem is that scope can easily change. Though I guess if there's a conflict you can easily append the parents name or something

a rec (or while) node forces the node to become shared, this way the shared node boundary can serve as the recursing scope, and args
can replace rec_in! What about map/filter/fold? -> it has the advantage that the entire iterator becomes one scope, no searching for the
iterator end... but unlike rec/while, it forces certain arguments to special semantics, which may not be nice. Overall seems to make things messier.

There is no easy way to connect the order of args as specified in the calling node, with the names when they have been renamed (maybe
initially they are #1, #2 etc). This can be done in the UI in some way, or on hover over. probably with labels in the caller?

---

so, what are the different paste options in this new system?
different options when replacing a shared node in a shared parent area:
1) replace the contents of the shared area (on the root node of shared)
2) replace the shared area in-place in parent (on the referring node)
3) replace the shared area and cut loose from the parent. (either allow selection just before node, or a seperate cut/unshare op that must be pressed before the replace)
a non shared node, or a node deeper inside a shared area has just possibilities 1) and 3)
inserting rather than replacing is still the same: this happens automatically if on copy there was a segment (from root to one child) rather than just
a regular tree. The replaced node will be appended to the end of the segment.

Hmm cutting can be a seperate operation because the moment you cut, the tree you cut off is now moved to the caller node, where it can then be overwritten normally.
So the double cut problem kinda goes away, since you'd first cut it out of the inner function, then out of the outer one. If the arg you are cutting is shared within
the function, you'd do the cut on the root node of the shared bit... the problem is that that cuts it from the function, but what if that shared item is used in more places?
We don't want it cut from the others. I guess the focus can help here, it would only be cut from its parent in the current focus. If it has multiple parents then it can
give an error and require the user to shift focus first. A similar rule can be used if a cut on a shared item is done in a function, and a sub-function also has a
reference to it, i.e. all children of the function get the cut applied first.

Only shame is that cuts now really feel a bit like abstraction: you are creating arguments. But args cannot be created on the first copy of the function, and args
are undone if only one copy remains, and if for one particular args, all arg values on all callers are the same. It can also move things from the arg into the function
body if for one arg the root of all vals are the same, etc.

Will need all sorts of auto refactorings... also not just root of arg val but leaves can sometimes be inlined. Two functions in a row can be concatenated if
always used together. Functions inlined if their only contents is a single op that takes all args.

-

So what happens if you cut (unshare), and that part of the tree contains an arg? then the unshared part must become a function, too.
It must therefore be possible to have function calls with missing args, that effectively are function values, and the receiver fills in the remaining args!
This... lacking args must also be possible for builtin ops.

[see WHILE 0..10 example]
The alternative is of course to cut it as a normal exp tree, and replace the arg by whatever the caller was supplying for it. This however changes the meaning
of the program greatly since the tree is now not multiple-executed anymore. To reach the original goal of the reusable tree, the programmer will now have to
unshare everything which is counter to the goals.
-> this is not legal, since it will also apply the cut to older users of the same code, and that will change their meaning without the user noticing!

An intermediate solution would be to unshare it as a function, but require any free vars to be args to the outer function. This is less flexible because the
outer function is now fixed to the free vars. If any of the inner functions get additional args, then so do all others, and the outer one too. that can work but
is fairly messy.
->?

This problem is somewhat caused by rec/while, because normal functions are purely compositional. (To some extend also IF, since there's an inherent lazyness in evaluation
there). Hmm, this has been a problem with every version of this design for if's inside functions with args getting unshared. This is most certainly a problem with
unpure operations, and even efficiency wise with pure ones. Lazy evaluation can fix IF, but it does not fix WHILE/REC, since effectively you're expecting normal
order there.

funnily enough the higher order functions don't have this problem... if you'd want to change the internals of a map-loop, you could unshare all you want, and
the head and tail of it would be seperate shared blocks if possible. But these can't do more generic loops like qsort & mandel

using rec_in, the problems with WHILE/REC go away, and it becomes a similar situation to higher order functions again. But this is messy for many vars.
It does have the advantage though of looping less vars around, those that are read only do not need to be a rec_in. It's also kinda neat that a tree with a loose
rec_in is kinda like a curried exp, or function value, except that its tied to a particular ordered arg to a loop.

Hmmm... much like foldr has iterate & accumulate, any while/rec should be able to name its rec_in's. That way, higher order functions become generic definitions
on top of rec, and making custom higher order functions becomes easy! Except... "iterate" actually does something, it doesn't just pass thru.

* how to make iterators same as rec_in renames?

* somehow make it such that a tree local to another can use its args? would reduce on duplication, but make parent tree less readable, you have to look at its shared
  children to see that it has args at all. And can't distinquish between own args and parent args in the child (beyond it receiving less args in the call than it has in the def).
  
* function calls with only args of the parent as args cannot be shared, since the shared function would take up just as much space. But they should still be executed once
  in most cases if copies exist. How does one identify these copies? What if the operation is a side effect, and shared eval is not expected?
  




->> see array programming for abstractionless.txt for more ideas on elimanting loops/recursion




STACK REVISITED
-> we dismissed stack languages above for their dup-swap mess. But with auto-refactoring, a lot of dup-swaps made by clumsy programmers can be
   auto-refactored. And stack languages are REALLY easy when it comes to refactoring and compositionaly... having less problems with loops and
   higher order functions. Also, there may be ways to make the stack more readable...
   ("Cat" is worth looking at... its like Joy with Hindley-Milner).
   
   Stack more readable: if you represent a list of stack ops like dataflow with lines, the special case compared to normal dataflow is that
   every output goes to exactly one input. This guarantees non-overlapping lines, since every op always consumes the top N (or the right N)
   lines, and adds more lines to the top (or right). This can still be represented rather flat, with ops going purely left-right.
   Can actually represent dup & swap almost like their line equivalents so its even more readable... "almost" in the sense that they are
   still ops in the language.
   Can color code lines to types.
   You would still edit like a list of ops, so lines get rearranged automatically, so you get immediate feedback if ops take too many (extra input lines
   to the current function) or too few (extra output lines from the current function) args that you still may want to add.
   You now would have explicit function blocks, but they are more like smalltalk blocks or scopes, in that you just indicate what part you want
   looped or passed to a higher order function, not a function definition. You're still not allowed to create your own functions, these
   are auto-refactored.
   Hmm.. these higher order functions would need special syntax to work, otherwise the lines don't connect up.
   if(a>0) a++ else a--
   dup 0 > [++] [--] if
   a lazy version of if would require much more dup-swap:
   dup -- swap dup ++ swap 0 > if
   int a = 1; while(a<10) a++;
   1 [dup 10 <] [++] while
   0 [+] fold
   fold is very easy, while can work, if is the most problematic (lack of matching ins/outs). So this hints towards an array aproach here too.
   But we still need IF... maybe in the a different format somehow.
   If using instruction masks? IF op just eats conditional, and then every following instruction can just be marked as true/false only. hmm
   will be just as messy with unmatched in-outs
   
   problem with factoring stack code is that is not as compositional as it looks. In some cases, if you want to make a local replacement
   inside a list of ops then the 2 sides of it can be made into individual shared blocks: a b c 1 d e f where you want to change 1,
   makes a b c and d e f seperate blocks. This may already not be desirable if they comprise one algorithm together or so, but is bearable
   (is it? with many args, this can result in a function with many seperate bits in between the args.. not so good).
   It is harder when there is structure, such as introduced by blocks (in ifs etc): [a [b [c 1 d] e] f] this cannot be shared properly!
   Would have to do blocks where left & right can be factored individually... no, makes issues with splitting up algorithms worse.
   To re-attain the goal of no-copy paste code, you'd have to reintroduce placeholders, which is very un-forth like.
   shared = [a [b [c X d] e] f]; shared(1) ?
   In theory, could replace X by an op that gets the value from the stack, below the args to shared. But then it would have to be burried
   there just before the call to shared, which gets messy. Though, this only really happens when user is creating a "function", and 
   auto refactoring can undo a lot of damage. The "burrying" instructions can be removed by moving the arg earlier in the code if possible.
   The arg has to be on the top of the stack and the burried in the general case, because of complicated control flow that may generate
   the other args. Similarly it has to be lowest on the stack inside the function because it would otherwise interfere with the code inside
   the function.
   This feels more like "creating an argument to a function" and not abstractionless, but oh well.
   
   That brings up the general topic of stack order.. the user influences some of that by the way they write code. Automatic refactoring
   should sometimes reorder args to reduce dup-swap inside the function, which may then push swaps to the caller, which may then have to
   be refactored too. Besides recursion, this can be done nicely from the leaves out...
   
   how do these stack order refactorings work?
   
LEE: mental continuity for SWAP op by using colors, icons...
the solution of breaking up into seperate functions may not be so bad, small functions are nice and how forth tends to work. also will
be easier to refactor this way.
The solution of lifting out and inserting stack ops could be ok too, depends on the refactoring power. Stack ops are also much more visual
and easier to follow. It may actually be that both solutions often amount to the same thing.
If this works, big advantage of course is that it doesn't require identifiers for args. All relatively simple.
LEE: horizontal IF probably best
LEE: can represent shared functions inline much like orginal abstractionless idea -> but that causes too much code on screen.




-----
fun idea: depending on how the structure of the language pans out, there'd be no problem doing this with an editor that looks
like ascii-syntax. Simply layout an ascii representation from an internal tree-layout, and track the mapping tree -> src locations.
When the user inserts a char/deletes a char, simply find the corresponding tree, and reparse it including the changed code.
If it can't be parsed (as will often be the case during editing, simply show the raw ascii with say a red background, and replace
the tree node with a "raw ascii" node. You can of course not run/typecheck until all such nodes are gone. 

If you doubleclick, it will select the current hierarchical node, or the parent of the current selection if you doubleclick on
that.

if we conclude that "graphical" syntax is not the way to go, that could open up the possibilities for a more complex
base language... can this system support a more complex language like Bear?
In particular, can it deal with state vs eager/normal/lazy?

What requires things to be normal order anyway? When passed to a normal order builtin. A for() loop in C can be seen as
a function that takes a normal order argument, so if you want to abstract over it, it needs a macro. Same with if.
So this can work simply by perculating these properties up. In a more functional language, normal order args are modelled
by function values. But this means creating functions explicitly again... though its not as explicit, as these functions
are always inline, and you can't determine the argument (they are given by the higher order function), so it is equal to
the body of a loop. Though once you allow function values in general, they can be stored in data structures, giving you
back explicit function.. not sure if this is an issue. If it has strong type inference, then this can only be allowed
if the usage of the data structure is passing it somewhere which eventually ends up in a builtin that has a normal order arg.

How do we differentiate normal order vs lazy? Normal order almost by definition has an argument (except when its sideeffecting).
It is lazy vs eager for 0-arg exps. To be correct, if an exp has a side effect (or just is expensive, or is used inside and IF), it should be lazy when its 
abstracted out. 

For a side effecting language, default when abstracting out should be normal order. If it is then used twice inside the body
afterwards, this is ambiguous. Or is it? if it has args, its normal order. If it has no result, and is used twice, that must
mean its normal order. If it has a result, but no args, that is the ambiguous case. Like pop(stack), you might want to reuse
the result, or you might want to pop twice.

if it has no args, a result, any number of uses, and no side effects... its eager.

Worse, if you have:

a = pop(stack)+1

and you somehow determine that it needs to be eager, then if you parametrize 1, it becomes normal order again!
So that means any side effecting code needs to be default normal order to reduce on suprises. It can't even be
lazy because it would give the same change in semantics.... not true, if it were properly lazy, the pop(true)
part of the function would be evaluated only once since it doesn't depend on function args.. or at least it
would have to be done that way. But it is kind of a non-argument since lazy and state is not a good idea anyway.

This means we need to make the entire language default normal order, or abandon side effects. Of course, most
expressions in normal order can be optimized to eager, but not all. What if you DO want the eager behaviour
for pop(stack) ? There would be no way to express that except some explicit variable or annotation. So
with state you gain less plumbing, but in return you'd have to inspect every use of it to see how often you
want the side effect to happen.. besides, full normal order may have the same evaluation order confusion as
lazyness.. so seems like this would be pointless.

Would it help to seperate side effects from results, disallow functions like pop(), but instead have top()
(which gives a result, and can be repeated (actually no, see below)), and drop() (which give no result, and therefore is clearly
normal order when repeated)? It does resolve the ambiguity, but it gives new problems, in that reordering
the execution order of these two now gives unexpected results.

so making a single shared block into a function not only can change the amount of evaluation (unless all is
normal order) it can also change the order of evaluation (for eager code).

So side effects are out. But that totally makes it useless for the mainstream.
Lee agrees side effects MUST be in.

as unreadable as stack languages are, they don't have this problem, since duplicate use of values is dealt
with completely seperately from factoring, hence any copy and paste is all normal order. Is there any way
we can replicate this behaviour with a more sensible syntax? Not really, only with variables (explicit
abstraction) or lines (not mainstream/readable).

A fully normal order language would be fun, too. It would just need annotations for certain expressions to
be "cached" for optimisation or semantics purposes. However this could never be mainstream. 

also, the real solution could be:

b = pop(stack)
a(x) = b+x

lifting out any sub-tree that doesn't depend on the new arg. This creates redundant factoring,
so maybe should only be done on side effecting trees. The redundancy is confusing though.

-

note that the problem above is not just with writing side effects, its also with reading, i.e.

a = top(stack)+1; ...a...; push(stack); ...a...;

this also means that any solutions trying to not allow functions that both have a side effect
and return a value cannot work, as there is no other way to read state.

Only if you can guarantee there's no writing of state in between reading it, would it be safe

-

Besides the stack example, there's also potentially data structure construction to consider
equally:

[1,2,3]

that will mean something different when that code is copied, wether the result is shared
or just the code is shared, in a side effecting language.


One solution would be to have a normal order language, but to have a special marking
that denotes "create a cell here that caches the result in case of multiple evaluations",
lets say <> for now. So it would look like:

a = < pop(stack) + 1 >; ... a ... a ...

Which means shared result. But if the user actually wanted to pop twice, he could change it to:

a = pop(stack) + 1; ... < a > ... < a > ...

actually, just this would suffice:

a = pop(stack) + 1; ... a ... a ...

this would now truely make it a macro system. Probably not as intuitive for existing programmers,
but maybe better than having to have completely different UI facilities for code sharing and dataflow sharing.

Now in the first case, if 1 is abstracted out there is still a semantics change.. IF 1 was within the <> in 
the first place. So maybe that means it needs to surround the side effecting op directly.

so in essense this is like allowing the user to change from a default "a" to "a()", except that now it also
works preventing the semantics change when arguments are added outside of the side effecting expression.

can we turn this around? since you want default cached, and really all pure code you'd conceptually want
cached too even though it doesn't matter, have an annotation for repeated evaluation instead?
Yes.. except that now it be weird that 

a(x) = pop(stack) + x; ... a(2) ... a(3) ...

by default only pops the stack once! That is not semantics programmers of any language are used to. So maybe
this can be a forced abstraction of pop(stack) ? a "side effect abstraction" ? So, if you had these:
- default "a" for side effects with return values, "a()" for void side effects
- manual change from "a" to "a()" (only works if contains side effects, because otherwise would revert to "a" immediately)
- automatic "side effect abstraction" when "a" -> "a(x)" (not when "a()" to "a(x)", and only for the side effects not dependent on x)
- manual inlining of "side effect abstraction", or rather, the same "s" to "s()", which will cause the inlining automatically
you could program with side effects pretty comfortably.

can promote builtin side effecting functions to not return values when possible, to further reduce ambiguity.

any side effects (and/or symbols using them) should be marked graphically.

what of the scope of these cache cells? This is a problem in general, what happens if the user starts copying these
shared blocks outside of the current "function" they are in? Ideally, you define any form of abstraction as not being
"inside" anything, but since "a" means, evaluate just once, that is essentially lazy evaluation if its not part of
any scope, which may give an order of evaluation problem! Though, for any block of code, all its dependents can be shown
in order of evaluation (previous idea was in terms of closeness to the root). Not entirely, if-then-else can screw that
up, but maybe the amount of evaluation order problems with this is not likely to be great and relatively predictable?

"a" is secretly a function too, since it often relies external arguments, like "stack". The difference is, that
all uses of "a" call it with the same arguments, thus explaining it single evaluation. So if the body gets copied 
somewhere where stack is not available, it simply becomes a real function, and stack an arg (ignore "side effect abstraction"
for now). Similarly with any exp thats dependent on an arg thats not shared yet that gets copied outside of the scope,
immediately becomes a function. But what should the arg to the new caller be? ideally its also stack, which means now
whatever the value of the original caller was needs to first be made shared, and then possibly a function also if it
depends on anything, and repeat. So this essentially will result in a chain of nested function calls at the new call site,
that reconstruct whatever value the original copy operation relied on. If now the user replaces it with something else,
this then results in a reverse chain reaction of inlinings. Worse, if theres many callers, the the functions being abstracted
need to essentially cover the union of all those call sites.
So that is a total mess. So when pasting, any free vars need to be replaced by args with default values, or maybe just with
an error value that is required to be replaced to be able to run the code.
It should however still be able to paste uses of such vars in "called" functions, as that would simply add arguments all
the way down the chain. 
If the language had multiple return values, there may be a case for allowing limited amount of use of free vars up the chain,
if its after the call and no if-thens etc interfere.

So this language can have assignment, in terms of something equivalent to a[1] = x and a.y = x, but no single var assignment?
Single var assignment is mostly needed for loops, so it is essential that loops get replaced by something else. This may not
be possible in the general case, and people may dislike having to get into crazy functional/APL list just to get their
algorithm working.
- looping a function onto itself? / accumulators

- local symbols?
yes, by default symbols are as local as possible and can have duplicate names with names elsewhere. Just requires that
whenever a new use is added that it searches for clashes. Symbols that are purely local to the current root should be
show differently.
Local should be defined as all copies being directly in the root, or in local functions from the root, NOT in simply reachable from
the root.

- macro style use?
essentially, we have normal order defs, but what about args? if the arg is a void exp, its again a given, but we have no way
to see this nor to change it if its eager. So maybe needs special syntax, i.e. f({ pop(stack) }). Again it be default 
f(pop(stack)), but you could change this with the same key as a() above. This would have to be implemented as a closure or
macro. So essentially the moment you type "pop(stack)" inside of f, it may not know what stack is and mark it as an error
(or make it an arg to f if all callers have it?), but then as soon as the exp is lifted out, it will be available locally.
So what now if the exp depends on both a local and a parent var? It could either simply be allowed, or the closure can be
lifted to an anonymous function. Anonymous functions would be quite special, since they'd be the only thing that can
access parent variables, or maybe all functions should be allowed to be local if possible? That be fine, and would help
with naming anyway, and can always be made more global. Just need consistent syntax, so maybe instead of f({ pop(stack) })
it would have to be f(() = pop(stack)). Or maybe a() = pop(stack) can become a = { pop(stack) } ? That is actually nice
because now a -> a() is not a special UI op anymore! But sadly it also doesn't show that () is always normal order... 
though I guess any {} def could always be shown as a() = {} ? That's probably best, just now the arg version becomes
f((x) = { pop(stack) }) which is not entirely consistent, but oh well.
Hmm is maybe not such a good idea for {} to mean normal order, since {} used statement concatenator may not be normal
order (e.g. { print 1; 2 }+3), and a(x) = pop(stack) IS normal order.
so back to the () = syntax uniformly. Can use {} for statement concat, or ";" exclusively.
How close to C syntax is this supposed to be anyway? Is not going to be super close for many reasons, so can take
some liberties.


global variables are simply made by creating any data structure, sharing its creation code everywhere and using
update in-place ops on it.


inclusion polymorphism? since this will be a closed world system, is this even relevant? could add pattern matching
for data structures to ease the pain of switch statements. Everything needs to be an exp in the end, so doing something
at the function level doesn't make sense... will have to be a pattern case-of. I guess MM support would be nice, which
would require a case-of for multiple values. Also would need to allow superclass-subclass substitution...

-----
issue: if you copy/paste a code block right next to itself, and the system abstracts it out to a shared block, and 
you then go make an unshared change, how does the system know to which of the 2 blocks it belongs?

[ 1+2*3 ]
[ a*a ] with a = [ 1+2 ]
then a = [ 1+[ 3 ] ] ????   [ a(2)*a(3) ] or the other way around??

Sounds like you somehow need to indicate which copy you are referring to.
Since there is already an ambiguity between shared and unshared changes, a mechanic that fixes both at once would be great.
For example, you could make it such that if you zoom in with the shared use as its root (in this case "a"), and then touch 
the body of a, you'll get an unshared change. It will show this by showing it with an "unshared" background color.

So basically, the scope of the changes relies on the current zoom! That is pretty intuitive, manipulatable, and does not
require special keys for any of the above cases! Also, just by the mere fact that it is abstracted out, you KNOW that
it is shared elsewhere, just not in this tree!

The problem with this however is that by default most functions being called from a block of code will show up unshared,
meaning you need to zoom into them to make shared changes. This puts more weight on the zooming feature, meaning it
says even more firmly, you are making changes to code in context. This may not be so bad.

----

for state: normally FP is annoying because state requires excessive plumbing. But here we can do the plumbing
automatically, and treat variables that are both input/output or passed on from further away differently.

Thing is, to solve e.g. the above pop(stack) ambiguity, it needs to generate an entirely new variable, otherwise
its not possible to indicate which you want. Assignment also can't be a construct since that would start making
uses of variables as something other than merely names, and impede sharing/unsharing.

But stateless sharing can certainly work this way, once you start creating functions, you may want to share an
exp from the root function all the way down to a leaf function. This shared block then has to be weaved thru all
function calls. We initially had this situation as simply a shared block sitting outside both functions, but that
is not realistic, as it may depend on variables in the root function. (this kind of plumbing is ok, since
it is MUCH less work plumbing only thru args, as opposed to args + retvals).

so what happens if you drag from a leaf to a root? in theory, it should be weaved thru the return values, as it
may depend on locals in the leaf. Frankly return values can be seen symmetric to arguments and also need to be
auto abstracted. This isn't possible in the general case, as when the call goes thru a conditional, or the
destination is before the caller, or there are multiple callers and it be ambiguous which one to use, etc.
Also, with multiple uses of "state" like this, you'd have to know what other uses to link it up with...
which sounds near impossible in more complex cases. (yup, not going to allow this in the most
general case, see above for more detail).


Generally, for state, it would be fine to have the language be purely functional, but data structures are
mutable. This mirrors how already a lot of OO programs are written, without globals, but with lots of
environment objects passed around. Array/list/vector functions can generally be functional, but can be
optimized to be in-place where need be. This still gives problems with side effects mentioned above,
however.


------------

focus based editing needs to be transitive, e.g. if I have the Pythagorean theorem:

sqrt(a+a)
a = b*b
b = 3

if I change 3 in the above focus, both sides of the triangle change, but if I then focus on one
of the two "a"'s in the root, and change b, then b should only change for one of them.

sqrt(a(3)+a(4))
a(x) = b(x)*b(x)
b(x) = x

which then immediately gets refactored to:

sqrt(a(3)+a(4))
a(x) = x*x

Which is fine.

