Structs
On 05/27/2010 05:29 PM, Brendan Eich wrote:
On May 27, 2010, at 2:18 PM, Jonas Sicking wrote:
On Thu, May 27, 2010 at 12:05 PM, Brendan Eich <brendan at mozilla.com> wrote:
If structs are anything like value types, then I am interested in participating. I'm particularly interested in working through the details of how arithmetic of integer like quantities would work.
The struct-array idea for WebGL avoids the typed array aliasing design, and consequent byte-order leakage. But we are not envisioning new operator overloading or literal syntax. At most a[i].x would be optimized to a very fast, packed member read or write, after scaling i and adding the offset of x. Any read would use ES's number type, I believe.
The other thing that Khronos really wanted to avoid was for a[i] in the above expression not to create an object which is to be initialized and GCed. Though maybe you're including that in the scaling+offset algorithm.
Yes, I wrote a[i].x on purpose. Just a[i] with no .x or .y, etc. after would reify a JS object.
What would the results of the following be:
a[i] === a[i]
Or the following(*):
b=a[i]; ...; b == a[i]
Or the following(*):
a[0]=a[1]; ...; a[0] === a[1]
I hope that the answer is the same in all three cases. If the answer is the true in all three cases, then this is a value time in the ECMA 334 sense of the word. If the answer is false in all three cases, then I assert that many will find that to be surprising, particularly in the first expression.
- Sam Ruby
(*) where "..." is a series of statements that do not affect the value of b or a.
On May 27, 2010, at 3:32 PM, Sam Ruby wrote:
What would the results of the following be:
a[i] === a[i]
Or the following(*):
b=a[i]; ...; b == a[i]
Or the following(*):
a[0]=a[1]; ...; a[0] === a[1]
I hope that the answer is the same in all three cases. If the
answer is the true in all three cases, then this is a value [type]
in the ECMA 334 sense of the word. If the answer is false in all
three cases, then I assert that many will find that to be
surprising, particularly in the first expression.
Good questions.
You're right that if we have value types then structs, even if
schematically defined rather than declared, and used only to optimize
primitive-typed field access, are value types.
In particular, since typed arrays are fixed length but with mutable
elements, a[i] === a[i] implies memoizing when reifying a[i] as an
object. The element structs are not extensible, however -- they're
sealed in ES5 terms. So while you can set a[i].x = ..., you cannot set
a[i].w where no such w was defined in the schema.
b = a[i] just copies the memoized reference to the reified object, so
b == a[i] and of course b === a[i].
a[0] = a[1] could also be made to work with sealed objects reflecting
the struct elements of the arrays. The right-hand side reifies or
finds the memoized object reflecting a[1], the packed struct. The
assignment to a[0] then reads property values from the object (fat
ugly numbers!) and writes into the packed struct at a[0] according to
the schema.
So it still seems to me one could avoid equating value types and these
schema structs. They are distinct proposals, but you could tie them
together. I think that would be a mistake at this point, especially
since WebGL folks have no need for operators and literals (as far as I
know).
On May 27, 2010, at 3:47 PM, Brendan Eich wrote:
Good questions.
You're right that if we have value types then structs, even if
schematically defined rather than declared, and used only to
optimize primitive-typed field access, are value types.
Argh, I meant "could be value types." Because I then went on to argue
that sealed objects with writable properties reflecting the element
structs, memoized to avoid a fresh object identity on each a[i], could
be used too.
It pays, especially in the strawman space, to avoid coupling
proposals. But it would be great to have structs of the schematic
kind, and value types with operators and literals (whatever the data
representation, structs or arrays or number pairs), in hand. Then we
could see whether it makes sense to unify.
On 05/27/2010 06:47 PM, Brendan Eich wrote:
On May 27, 2010, at 3:32 PM, Sam Ruby wrote:
What would the results of the following be:
a[i] === a[i]
Or the following(*):
b=a[i]; ...; b == a[i]
Or the following(*):
a[0]=a[1]; ...; a[0] === a[1]
I hope that the answer is the same in all three cases. If the answer is the true in all three cases, then this is a value [type] in the ECMA 334 sense of the word. If the answer is false in all three cases, then I assert that many will find that to be surprising, particularly in the first expression.
Good questions.
You're right that if we have value types then structs, even if schematically defined rather than declared, and used only to optimize primitive-typed field access, are value types.
In particular, since typed arrays are fixed length but with mutable elements, a[i] === a[i] implies memoizing when reifying a[i] as an object. The element structs are not extensible, however -- they're sealed in ES5 terms. So while you can set a[i].x = ..., you cannot set a[i].w where no such w was defined in the schema.
b = a[i] just copies the memoized reference to the reified object, so b == a[i] and of course b === a[i].
a[0] = a[1] could also be made to work with sealed objects reflecting the struct elements of the arrays. The right-hand side reifies or finds the memoized object reflecting a[1], the packed struct. The assignment to a[0] then reads property values from the object (fat ugly numbers!) and writes into the packed struct at a[0] according to the schema.
... and the value of a[0] === a[1] is?
So it still seems to me one could avoid equating value types and these schema structs. They are distinct proposals, but you could tie them together. I think that would be a mistake at this point, especially since WebGL folks have no need for operators and literals (as far as I know).
It is not a matter of whether they "need" === operators to be defined, but rather a matter of defining what the value of === operators is to be.
Nor, am I suggesting that === be overloadable.
I am suggesting that if a[0]===a[1] is to be true, and one can assume that a[n] is a series of octets, then we are talking about strncmp like behavior. And if we can assume strncmp, then I see no reason to preclude implementations from using strncpy.
/be
- Sam Ruby
On May 27, 2010, at 5:21 PM, Sam Ruby wrote:
a[0] = a[1] could also be made to work with sealed objects reflecting the struct elements of the arrays. The right-hand side reifies or
finds the memoized object reflecting a[1], the packed struct. The
assignment to a[0] then reads property values from the object (fat ugly
numbers!) and writes into the packed struct at a[0] according to the schema.... and the value of a[0] === a[1] is?
With any kind of mutable object reflecting the packed struct, the
answer will be false unless we have value types. See
strawman:value_types#hard_cases
search for ===.
Value types were conceived of as shallowly frozen, but we could try to
relax that.
So it still seems to me one could avoid equating value types and
these schema structs. They are distinct proposals, but you could tie them together. I think that would be a mistake at this point, especially since WebGL folks have no need for operators and literals (as far
as I know).It is not a matter of whether they "need" === operators to be
defined, but rather a matter of defining what the value of ===
operators is to be.Nor, am I suggesting that === be overloadable.
I am suggesting that if a[0]===a[1] is to be true, and one can
assume that a[n] is a series of octets, then we are talking about
strncmp like behavior. And if we can assume strncmp, then I see no
reason to preclude implementations from using strncpy.
If you want this, whether === is overloadable, and WebGL (lots of
folks, let's say) want structs in the array to be mutable, then we run
into the conflict recorded in the wiki.
''Jason: === must be overloadable. Mark: no, Value Types mean === can
do a recursive, cycle-tolerant structural comparison. Jason: not
convinced that complexity is warranted.
...
Decision logic:
if (=== is overloadable) { The need for egal goes up; if (egal is added (it is not overloadable)) {0, -0} and NaN issues go away; } else { The need for egal goes down, because === is well-defined as structural recursive etc. comparison; } Value Type means “shallow frozen”.''
Say we add egal, so one can tell the difference between a[0] and a[1]
even if they happen to contain the same bits, because that difference
might matter due to one of them being mutated to have different bits.
Then do the objections to unfrozen value types go away? Cc'ing Mark.
On 05/27/2010 08:36 PM, Brendan Eich wrote:
On May 27, 2010, at 5:21 PM, Sam Ruby wrote:
a[0] = a[1] could also be made to work with sealed objects reflecting the struct elements of the arrays. The right-hand side reifies or finds the memoized object reflecting a[1], the packed struct. The assignment to a[0] then reads property values from the object (fat ugly numbers!) and writes into the packed struct at a[0] according to the schema.
... and the value of a[0] === a[1] is?
With any kind of mutable object reflecting the packed struct, the answer will be false unless we have value types. See
strawman:value_types#hard_cases
search for ===.
And I previously argued that false would be unexpected, that's how I came to the conclusion that structs in ECMA 262, just like in ECMA 334, should be value types.
Value types were conceived of as shallowly frozen, but we could try to relax that.
So it still seems to me one could avoid equating value types and these schema structs. They are distinct proposals, but you could tie them together. I think that would be a mistake at this point, especially since WebGL folks have no need for operators and literals (as far as I know).
It is not a matter of whether they "need" === operators to be defined, but rather a matter of defining what the value of === operators is to be.
Nor, am I suggesting that === be overloadable.
I am suggesting that if a[0]===a[1] is to be true, and one can assume that a[n] is a series of octets, then we are talking about strncmp like behavior. And if we can assume strncmp, then I see no reason to preclude implementations from using strncpy.
If you want this, whether === is overloadable, and WebGL (lots of folks, let's say) want structs in the array to be mutable, then we run into the conflict recorded in the wiki.
''Jason: === must be overloadable. Mark: no, Value Types mean === can do a recursive, cycle-tolerant structural comparison. Jason: not convinced that complexity is warranted. ...
Decision logic:
if (=== is overloadable) { The need for egal goes up; if (egal is added (it is not overloadable)) {0, -0} and NaN issues go away; } else { The need for egal goes down, because === is well-defined as structural recursive etc. comparison; } Value Type means “shallow frozen”.''
Say we add egal, so one can tell the difference between a[0] and a[1] even if they happen to contain the same bits, because that difference might matter due to one of them being mutated to have different bits.
I don't follow that.
If a[0] and a[1] have the same bits, why would a[0] === a[1] ever be false?
Then do the objections to unfrozen value types go away? Cc'ing Mark.
I am not assuming that === be overloadable. Nor am I presuming that value types are mutable.
Some people believe that the {0, -0} and NaN behaviors are a historical wart, to be avoided in all future work. Others believe that consistency here is important. I see merit in both sides, and in any case believe that the issue can be made to go away without requiring an egal method. At most, all that is requires is the addition of methods to determine NaN-ness and Zero-ness.
Pseudo-code:
function "==="(a,b) { if (strcmp(a,b)) { return a.isZero() && b.isZero(); } else { return !a.isNaN() } }
My assumptions are that "overloading" the operators for value types, at some point in time, and based on double-dispatch approach, will be seen as a good idea as a possible candidate for inclusion in some version of ECMAScript. Furthermore, I am assuming that whatever is designed for value types should apply to structs -- unless there is a compelling reason not to. As you put it: 'having a value types proposal that composes well with any "struct and struct array schema" proposal would be great.'
What I see in structs is the potential to define a 128 bit quantity that could be efficiently schlepped around. I don't see a need to modify one -- instead if you want a new value, you create one. But I don't believe that the end result of "a=b" should result in a situation where "a !== b' for structs any more than it does today for either Numerics today or for Objects (with the one notable exception being NaN... meh).
Further, I see in structs as a data type with no legacy. And one where neither arithmetic nor comparison functions are super-duper-ultra performance critical. By that I mean that the isZero and isNaN calls above are tolerable.
A combination of double-dispatch and existing ES objects could enable the creation of an infinite precision integer library -- written in pure ES.
A combination of double-dispatch and structs could enable the creation of 128 bit decimal objects. Again, this could be written in pure ES. Should it prove popular, vendors may chose to optimize this, and perhaps even standardize the library.
Or not. In fact, there need not be only one true decimal library. If others have different ideas on how decimal should be implemented, they would be free to create their own.
/be
- Sam Ruby
On May 27, 2010, at 7:12 PM, Sam Ruby wrote:
Say we add egal, so one can tell the difference between a[0] and a[1] even if they happen to contain the same bits, because that difference might matter due to one of them being mutated to have different bits.
I don't follow that.
If a[0] and a[1] have the same bits, why would a[0] === a[1] ever be
false?
Because mutability means a[0] and a[1] do not have the same identity,
and code that must care about identity (security, hashing, anything)
must be able to tell the difference between the two structs.
If a[0] and a[1] are immutable value types, then no worries. But
structs in ECMA-334 (C#) and in the WebGL alternative we are trying to
develop are mutable.
Then do the objections to unfrozen value types go away? Cc'ing Mark.
I am not assuming that === be overloadable. Nor am I presuming that
value types are mutable.
You wrote "... I came to the conclusion that structs in ECMA 262,
just like in ECMA 334, should be value types" which I claim means
value types can be mutable.
Short on time, I'll stop here because it seems to me you are presuming
mutability (for some value types, anyway).
On Thu, May 27, 2010 at 5:36 PM, Brendan Eich <brendan at mozilla.com> wrote:
On May 27, 2010, at 5:21 PM, Sam Ruby wrote:
a[0] = a[1] could also be made to work with sealed objects reflecting
the struct elements of the arrays. The right-hand side reifies or finds the memoized object reflecting a[1], the packed struct. The assignment to a[0] then reads property values from the object (fat ugly numbers!) and writes into the packed struct at a[0] according to the schema.
... and the value of a[0] === a[1] is?
With any kind of mutable object reflecting the packed struct, the answer will be false unless we have value types. See
strawman:value_types#hard_cases
search for ===.
Value types were conceived of as shallowly frozen, but we could try to relax that.
So it still seems to me one could avoid equating value types and these
schema structs. They are distinct proposals, but you could tie them together. I think that would be a mistake at this point, especially since WebGL folks have no need for operators and literals (as far as I know).
It is not a matter of whether they "need" === operators to be defined, but rather a matter of defining what the value of === operators is to be.
Nor, am I suggesting that === be overloadable.
I am suggesting that if a[0]===a[1] is to be true, and one can assume that a[n] is a series of octets, then we are talking about strncmp like behavior. And if we can assume strncmp, then I see no reason to preclude implementations from using strncpy.
If you want this, whether === is overloadable, and WebGL (lots of folks, let's say) want structs in the array to be mutable, then we run into the conflict recorded in the wiki.
''Jason: === must be overloadable. Mark: no, Value Types mean === can do a recursive, cycle-tolerant structural comparison. Jason: not convinced that complexity is warranted. ...
Decision logic:
if (=== is overloadable) { The need for egal goes up; if (egal is added (it is not overloadable)) {0, -0} and NaN issues go away; } else { The need for egal goes down, because === is well-defined as structural recursive etc. comparison; } Value Type means “shallow frozen”.''
Say we add egal, so one can tell the difference between a[0] and a[1] even if they happen to contain the same bits, because that difference might matter due to one of them being mutated to have different bits. Then do the objections to unfrozen value types go away? Cc'ing Mark.
Given that we have added egal as a separate reliable non-overloadable operator, I would then have less objection to allowing === to be overloaded. But should we add egal?
I see this as much like the function vs lambda question. I long for a true lambda, as a clean alternative to function. I long for egal, as a clean alternative to ===. However, the argument against lambda is at least as valid as an argument against egal: The functionality represented by lambda or egal is already closely approximated by an existing construct in the language. Adding another similar but subtly different construct to the language adds more confusion and complexity than utility. Imagine how much harder it would be to teach beginning programmers not just about function vs constructors vs methods, but also about these vs lambda. Likewise, imagine teaching beginning programmer not just about === vs ==, but about both of these vs egal.
The downside of egal by this argument is just as bad as the downside of lambda. The upside of lambda is much greater than the upside of egal, since === is a much less broken egal than function is a broken lambda.
That said, I'm not sure I understand why this should gate anything in this thread. Value types should be frozen (shallowly immutable) regardless, or other things break, e.g., they could no longer be transparently passed by copy. C# got this wrong, and paid a semantic complexity price we must avoid. Non-frozen structs should not be value types. Frozen structs could be value types, or could be wrapped in value types or something. Given these constraints on mutability and value types, I don't see that it matters whether the reliable comparison is named === or egal.
Btw, please no one suggest that egal be spelled "====". Thanks.
On May 27, 2010, at 10:43 PM, Mark S. Miller wrote:
Given that we have added egal as a separate reliable non- overloadable operator, I would then have less objection to allowing
=== to be overloaded. But should we add egal?I see this as much like the function vs lambda question. I long for
a true lambda, as a clean alternative to function. I long for egal,
as a clean alternative to ===. However, the argument against lambda
is at least as valid as an argument against egal: The functionality
represented by lambda or egal is already closely approximated by an
existing construct in the language. Adding another similar but
subtly different construct to the language adds more confusion and
complexity than utility. Imagine how much harder it would be to
teach beginning programmers not just about function vs constructors
vs methods, but also about these vs lambda. Likewise, imagine
teaching beginning programmer not just about === vs ==, but about
both of these vs egal.
I agree. We can consider egal on its own, but it did come up last fall
when we were discussing value types, in particular whether === is
overloadable, even stipulating immutable value types. And something
like an identity test would seem to be necessary (as a patch to a
mitigate a mistake) if we were to allow === to be overloaded for
mutable structs-as-values.
Problem solved by requiring shallowly frozen value types. Which means
they cannot be the "structs" we've talked about this week as the array-
of-structs alternative to typed array views on a byte buffer. This is
the bone of contention Sam and I were gnawing on.
The downside of egal by this argument is just as bad as the downside
of lambda. The upside of lambda is much greater than the upside of
egal, since === is a much less broken egal than function is a broken
lambda.
We've also talked about reforming function, a process begun by ES5
strict mode, which won't culminate in function reaching the TCP
nirvana of lambda. But some of us believe that TCP, especially return
in a lambda contained by a function returning from the function (or
throwing, if the lambda escaped and outlived the function's
activation) is a net loss, not a gain.
Digression (bear with me, I hope it is worthwhile):
I wager lambda is attractive to Smalltalk block fans, but Smalltalk
does not have first-class methods-as-funargs you can extract and pass
around. Alan Kay quote:
"I could hardly believe how beautiful and wonderful the idea of LISP
was. I say it this way because LISP had not only been around enough to
get some honest barnacles, but worse, there were deep flaws in its
logical foundations. By this, I mean that the pure language was
supposed to be based on functions, but its most important components
-- such as lambda expressions, quotes, and conds -- were not functions
at all, and instead were called special forms. Landin and others had
been able to get quotes and cons in terms of lambda by tricks that
were variously clever and useful, but the flaw remained in the jewel.
In the practical language things were better. There were not just
EXPRs (which evaluated their arguments), but FEXPRs (which did not).
My next questions was, why on Earth call it a functional language? Why
not just base everything on FEXPRs and force evaluation on the
receiving side when needed?
I could never get a good answer, but the question was very helpful
when it came time to invent Smalltalk, because this started a line of
thought that said 'take the hardest and most profound thing you need
to do, make it great, an then build every easier thing out of it."
[end quote]
This criticism of LISP seems a bit off, insofar as "special form"
means only what you can't write as a macro; it does not mean "non-
function". LISP lambda as I understand it is based on one definition
of "function" (en.wikipedia.org/wiki/Lambda_calculus, en.wikipedia.org/wiki/Function_(mathematics)
-- more learned students of programming languages and foundations of
mathematics should jump in if I'm wrong).
What's more, what Kay said about Smalltalk does not seem accurate
either: FEXPR^H^H^H^HSmalltalk blocks are not the most profoundly
primitive building block (ahem) on which the rest of Smalltalk is
built. Messaging and classes are more primitive. Classes with (non-
extractable) methods cannot be built on blocks. (Again I hope others
more expert than I am will correct me if I'm mistaken.)
Blocks in Smalltalk are brilliant, don't get me wrong. They're just
not the most profoundly primitive piece of the language, out of which
everything else was built.
I may be misreading Kay -- if he means FEXPRs imply messaging and
classes, and sending a message (possibly bearing quoted code as an
actual parameter) to a class instance is the most primitive building
block, then I agree. But I don't see how FEXPRs imply classes and
messaging in full (setting aside blocks for the moment).
Taking all this together, including the loveliness of blocks as
message selector arguments for building control structures, it seems
to me that lambdas aren't needed in JS, even if you don't find return
in a lambda either unwinding an outer function or failing to be
confusing and likely to be misused by non-experts. JS has first class
functions, which can be used to build classes and many other patterns,
at some cost in verbosity and (currently, mitigated by Object.freeze)
loss of integrity.
And TCP is not the highest good, or even an unmixed good in a language
that did not start out following it, and stick to it the whole way
through its evolution.
As Allen has noted, what we miss most about Smalltalk blocks, which
seemed attractive at first in the lambda proposal, is the very concise
block syntax. This conciseness may yet be achieved, or approximated at
any rate, purely via shorthand syntax (strawman:shorter_function_syntax
). If we get concise syntax, but not TCP, then lambda is even less
attractive.
End of digression. I agree that if given function (with incremental
reform under way), JS doesn't need lambda. Given ===, perhaps JS
doesn't need egal.
We should talk about egal separately -- it keeps coming up when we
turn to value types. It came up when decimal was being proposed for
ES3.1 -- remember Object.eq?
That said, I'm not sure I understand why this should gate anything
in this thread. Value types should be frozen (shallowly immutable)
regardless, or other things break, e.g., they could no longer be
transparently passed by copy. C# got this wrong, and paid a semantic
complexity price we must avoid. Non-frozen structs should not be
value types. Frozen structs could be value types, or could be
wrapped in value types or something.
Agreed. Sam?
Given these constraints on mutability and value types, I don't see
that it matters whether the reliable comparison is named === or egal.
Quite agreed.
I brought up egal, based on our value types discussion from last fall,
to buttress the case against mutable structs as value types. By
assuming mutable structs as value types and then showing how they
require a true identity relation operator, I hoped to shoot down
mutable structs as value types. On second thought, this was not the
best way to take on the problems inherent in mutable structs as value
types :-P.
C# is an Ecma and ISO standard, but that doesn't mean we should follow
it in this regard. In its defense, C#, like C++, is a static language
and so can take steps that JS cannot to preserve or recover
transparency for features such as pass by value and operator
overloading. It is not a good precedent for JS in this regard.
Btw, please no one suggest that egal be spelled "====". Thanks.
Amen!
[+DanHHIngalls]
On Fri, May 28, 2010 at 10:20 AM, Brendan Eich <brendan at mozilla.com> wrote:
On May 27, 2010, at 10:43 PM, Mark S. Miller wrote:
Given that we have added egal as a separate reliable non-overloadable operator, I would then have less objection to allowing === to be overloaded. But should we add egal?
I see this as much like the function vs lambda question. I long for a true lambda, as a clean alternative to function. I long for egal, as a clean alternative to ===. However, the argument against lambda is at least as valid as an argument against egal: The functionality represented by lambda or egal is already closely approximated by an existing construct in the language. Adding another similar but subtly different construct to the language adds more confusion and complexity than utility. Imagine how much harder it would be to teach beginning programmers not just about function vs constructors vs methods, but also about these vs lambda. Likewise, imagine teaching beginning programmer not just about === vs ==, but about both of these vs egal.
I agree. We can consider egal on its own, but it did come up last fall when we were discussing value types, in particular whether === is overloadable, even stipulating immutable value types. And something like an identity test would seem to be necessary (as a patch to a mitigate a mistake) if we were to allow === to be overloaded for mutable structs-as-values.
Problem solved by requiring shallowly frozen value types. Which means they cannot be the "structs" we've talked about this week as the array-of-structs alternative to typed array views on a byte buffer. This is the bone of contention Sam and I were gnawing on.
The downside of egal by this argument is just as bad as the downside of lambda. The upside of lambda is much greater than the upside of egal, since === is a much less broken egal than function is a broken lambda.
We've also talked about reforming function, a process begun by ES5 strict mode, which won't culminate in function reaching the TCP nirvana of lambda. But some of us believe that TCP, especially return in a lambda contained by a function returning from the function (or throwing, if the lambda escaped and outlived the function's activation) is a net loss, not a gain.
Digression (bear with me, I hope it is worthwhile):
I wager lambda is attractive to Smalltalk block fans, but Smalltalk does not have first-class methods-as-funargs you can extract and pass around. Alan Kay quote:
"I could hardly believe how beautiful and wonderful the idea of LISP was. I say it this way because LISP had not only been around enough to get some honest barnacles, but worse, there were deep flaws in its logical foundations. By this, I mean that the pure language was supposed to be based on functions, but its most important components -- such as *lambda *expressions, quotes, and *conds *-- were not functions at all, and instead were called special forms. *Landin *and others had been able to get quotes and cons in terms of lambda by tricks that were variously clever and useful, but the flaw remained in the jewel. In the practical language things were better. There were not just EXPRs (which evaluated their arguments), but FEXPRs (which did not). My next questions was, why on Earth call it a functional language? Why not just base everything on FEXPRs and force evaluation on the receiving side when needed?
I could never get a good answer, but the question was very helpful when it came time to invent Smalltalk, because this started a line of thought that said 'take the hardest and most profound thing you need to do, make it great, an then build every easier thing out of it." [end quote]
This criticism of LISP seems a bit off, insofar as "special form" means only what you can't write as a macro; it does not mean "non-function". LISP lambda as I understand it is based on one definition of "function" ( en.wikipedia.org/wiki/Lambda_calculus, en.wikipedia.org/wiki/Function_(mathematics) -- more learned students of programming languages and foundations of mathematics should jump in if I'm wrong).
What's more, what Kay said about Smalltalk does not seem accurate either: FEXPR^H^H^H^HSmalltalk blocks are not the most profoundly primitive building block (ahem) on which the rest of Smalltalk is built. Messaging and classes are more primitive. Classes with (non-extractable) methods cannot be built on blocks. (Again I hope others more expert than I am will correct me if I'm mistaken.)
Blocks in Smalltalk are brilliant, don't get me wrong. They're just not the most profoundly primitive piece of the language, out of which everything else was built.
I may be misreading Kay -- if he means FEXPRs imply messaging and classes, and sending a message (possibly bearing quoted code as an actual parameter) to a class instance is the most primitive building block, then I agree. But I don't see how FEXPRs imply classes and messaging in full (setting aside blocks for the moment).
Brendan pointed me at < www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_II.html> as
the source of this quote. Given that context, I do not believe Kay was speaking about what we now think of as Smalltalk blocks. Rather, I think Kay was referring to Smalltalk-72's use of dynamic scoping, inspired by Lisp 1.5, and (like Lisps of the day) the combination of dynamic scoping and delayed evaluation to support call-by-name-like functionality, in turn supporting control abstraction.
Like Lisp 1.5, Smalltalk-72 suffered from the name-capture issues that result from dynamic scoping. But I don't know that people worried too much about this at the time -- especially given their focus on children programming in the small. Smalltalk-72's delayed evaluation was actually built on its delayed parsing, going even farther than Lisp in what, in retrospect, we may now judge to be the wrong direction.
Fortunately, Smalltalk-72 helped inspire Actors, which combined lexical scoping and objects, which inspired Scheme, which kept the lexical scoping but lost the objects. Smalltalk-76 put lexical scoping and objects back together again, introduced the modern Smalltalk block syntax and semantics, and got rid of the delayed parsing and evaluation. I don't know whether the scoping reforms in Smalltalk-76 were directly influenced by Actors, or were indirectly influenced via Scheme. On these issues, Smalltalk-80 is essentially identical to Smalltalk-76. Smalltalk-76's blocks even had a scoping bug under block recursion that remains unfixed AFAIK in the main Squeak distribution.
I am cc'ing Dan Ingalls who lived through all of this and was responsible for much of it. Dan, how far off am I?
On 05/28/2010 01:20 PM, Brendan Eich wrote:
That said, I'm not sure I understand why this should gate anything in this thread. Value types should be frozen (shallowly immutable) regardless, or other things break, e.g., they could no longer be transparently passed by copy. C# got this wrong, and paid a semantic complexity price we must avoid. Non-frozen structs should not be value types. Frozen structs could be value types, or could be wrapped in value types or something.
Agreed. Sam?
There are so many undefined terms in that paragraph that I don't know what I would be agreeing to.
For example, I don't know why the word "shallowly" was inserted there. Was that just reflex, or is there an actual requirement to allow object references inside a struct? Looking at the syntax that Brendan put out for discussion purposes, it isn't clear to me how one would do that.
const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32", z:"u32", r:"u8", g:"u8", b:"u8", a:"u8"})); let a = new TA(...);
Mark mentions passed by copy. What happens if I pass a[1] as an parameter on a method call? Does something semantically different happen if the struct is frozen vs non-frozen? Is that complexity worth it?
Putting that aside for the moment, my more specific questions is: under what conditions would it ever be possible for a[1]===a[2] to be true?
There is much wrapped in that simple question. I'm inferring a lot from the discussion: "typed arrays" have a fixed number of elements, each of which has a fixed length. It should be possible for implementations to store entire arrays in contiguous storage.
As such, a[1] and a[2] in a typed array can never be the same object. Which means that they can never be ===, much less egal. By contrast, they could conceivably be the same object in a "classic" Array, depending on how they were constructed.
To facilitate discussion, I toss out the following:
var a = new Array(); a[0] = "a"; a[1] = "ab"; a[0] += "b";
if (a[0] === a[1]) { ... }
To the casual developer, I will assert that the fact that these strings are treated as being equal is an indication that they have the same value (i.e., sequence of bits) and not an indication that they occupy the same storage location (which could conceivably be true, but that's not generally something the implementation directly exposes).
The real question here: are what is currently being called a struct more like a bit string with convenient methods of accessing slices, or are they more like objects where no matter how close the sequence of bits are, two objects in different locations are never the same.
It might very well be that the requirements are such that the final conclusion will reluctantly be that it isn't worth trying to make === have a sane definition for these structs. That just isn't something I would expect as a starting position.
- Sam Ruby
On May 28, 2010, at 1:40 PM, Sam Ruby wrote:
On 05/28/2010 01:20 PM, Brendan Eich wrote:
That said, I'm not sure I understand why this should gate anything
in this thread. Value types should be frozen (shallowly immutable) regardless, or other things break, e.g., they could no longer be transparently passed by copy. C# got this wrong, and paid a semantic complexity price we must avoid. Non-frozen structs should not be
value types. Frozen structs could be value types, or could be wrapped in value types or something.Agreed. Sam?
There are so many undefined terms in that paragraph that I don't
know what I would be agreeing to.
Ok -- sorry about that. Mark and I are definitely on the same page
here, and used to the lingo.
For example, I don't know why the word "shallowly" was inserted
there. Was that just reflex, or is there an actual requirement to
allow object references inside a struct?
Mark was referring to value types, not the recent struct idea.
Object.freeze (ES5) is shallow, it does not try to walk an object
graph spanning tree freezing whatever it can reach (the Ice-9
disaster!). Henry Baker's egal does shallow bit comparison. That's the
key idea. So the freezing need not be deep.
Looking at the syntax that Brendan put out for discussion purposes,
it isn't clear to me how one would do that.const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32",
z:"u32", r:"u8", g:"u8",
b:"u8", a:"u8"})); let a = new TA(...);Mark mentions passed by copy. What happens if I pass a[1] as an
parameter on a method call? Does something semantically different
happen if the struct is frozen vs non-frozen? Is that complexity
worth it?
In the sketchy proposal for structs so far, a[i] reifies an object
reflecting the i'th element of a, not a new struct type. The array
still can be used to read and write packed, machine-typed, GPU-
friendly data, specifically via reads and writes of a[i].x, a[j].y, etc.
But if you extract a[i] and pass it around, you are definitely
allocating an object to represent the struct element, and however fast
it is to allocate a new object, you're not on the ultra-high-
performance path that WebGL wants, and uses today via typed arrays.
This is not a loss compared to the typed-arrays-as-views-of-one-shared-
byte-buffer idea because there's no way with typed arrays to view or
extract the whole element as a struct instance.
So far, I've been talking about the "structs defined by runtime type
descriptors" idea we've entertained as an alternative for the WebGL
use-case satisfied by typed arrays.
If you don't insist that structs are value types, then there's no
issue here. These structs are mutable, but not extensible; you can
write as well as read their members.
Your question about about frozen vs. non-frozen, by freezing the
structs, breaks the WebGL use-case in order to make structs be value
types. That's no good for the WebGL use-cases, but I'll answer your
question anyway: if the struct were a value type, then it would be
frozen, and loading a[i] to pass as a parameter to a function call
would be free to copy the struct data rather than pass an object
reference, if doing so were more efficient.
This freedom to copy is not just an optimization win (sometimes,
depending on size and other considerations). It's a different type
model for programmers to count on. More below.
Putting that aside for the moment, my more specific questions is:
under what conditions would it ever be possible for a[1]===a[2] to
be true?
BTW, === and other operators are not wanted for WebGL structs. Again
you can't even reference the aggregate element (struct) type via typed
arrays -- you can just load and store primitive types in members of
the struct elements via aliasing typed array "views" of the underlying
bytes.
There is much wrapped in that simple question. I'm inferring a lot
from the discussion: "typed arrays" have a fixed number of elements,
each of which has a fixed length. It should be possible for
implementations to store entire arrays in contiguous storage.
Right.
As such, a[1] and a[2] in a typed array can never be the same
object. Which means that they can never be ===, much less egal. By
contrast, they could conceivably be the same object in a "classic"
Array, depending on how they were constructed.
That's right, an array of references to objects in the heap can have
two elements referring to the same object. Object identity is by
address or equivalent "safe pointer".
With anything like struct or value types, the array elements are not
safe pointers into the object heap -- they are bits stored in bytes or
larger units stored adjacent to one another for a[i] and a[i+1].
Identity is by bit-string value.
To facilitate discussion, I toss out the following:
var a = new Array(); a[0] = "a"; a[1] = "ab"; a[0] += "b";
if (a[0] === a[1]) { ... }
To the casual developer, I will assert that the fact that these
strings are treated as being equal is an indication that they have
the same value (i.e., sequence of bits) and not an indication that
they occupy the same storage location (which could conceivably be
true, but that's not generally something the implementation directly
exposes).
Notice how it is critical that primitive string instances cannot be
mutated. That is, suppose you could assign to an element of a string
via a[1][1] = "c", and thereby cause a[1] to be "ac".
Well, no big deal, right? a[1] was === "ab", and it is now ===
"ac". Except there could be other references to a[1], passed to a
function a second before we observed "ab", or shared via copying
references to other vars or properties. Are those references expecting
"ac", or will they be surprised not to have received what looks like
an immutable string with chars "ab"?
You can make both kinds of strings work, and various programming
languages have chosen one way or the other. JS has immutable-seeming
strings, however. We are not looking to make an incompatible change to
string semantics. But more crucially, we believe the ability to mutate
any shared value (not a referenced object) is likely to result in
more bugs than otherwise.
The obvious cases of numeric value types seem to be excluded, but they
prove the rule. There's no exception for JS strings. Why should there
be one for structs?
Object mutation is the name of the game in JS if you program
imperatively by mutating the store, and local variables are not
enough. But mutable objects are a source of bugs. Mutation is a bitch.
Yet at least with objects, which have "reference type" semantics, it
is a known and accepted part of the deal.
String, number, boolean, and possibly decimal, rational, etc. are
"value types". You can't change someone's value received via an
argument "behind their back." No one can call foo(x = 42) and for
function foo(a) { setTimeout(alert, 0, a); } have other than 42 be
alerted, even if the caller of foo(x = 42) immediate does
x.mutatingAdd(1). There's no way to implement mutatingAdd on a
primitive number. Any such method on a Number object instance that's
automatically wrapped around 42 at the point of the method call won't
be able to change the primitive value either.
But structs as sketched for WebGL array-of-struct use cases must be
writable. Therefore they can' t be value types.
The real question here: are what is currently being called a struct
more like a bit string with convenient methods of accessing slices,
or are they more like objects where no matter how close the sequence
of bits are, two objects in different locations are never the same.
WebGL structs are the latter, and operators are not wanted anyway.
Value types are the former, which is why I keep urging you not to mix
up the very recent "schematic structs for WebGL (instead of typed
array views)" idea with the value types proposal at strawman:value_types
-- the way to prototype Decimal or any other new numeric type is not
as a "struct" of the proposed WebGL-array-of-structs kind, but as a
"value type".
Sure, value types might use structs-as-proposed (assuming they make
a future spec) to store the bits being compared by ===. But that would
be up to the value type implementation, and the frozen restriction
would apply, overriding any writable by default status that structs as
hypothesized have.
It might very well be that the requirements are such that the final
conclusion will reluctantly be that it isn't worth trying to make
=== have a sane definition for these structs. That just isn't
something I would expect as a starting position.
It is a "starting point" given the value-types-are-
(shallowly-)immutable premise recorded in the wiki'd discussion from
last fall. But that may not be shared premise. We need to agree that
objects are the only reference types, and therefore value types may be
copied (not passed by reference) safely without fear of someone
changing the received value's bits "behind your back".
On May 28, 2010, at 3:12 PM, Dan Ingalls wrote:
[+DanHHIngalls]
Help! You caught me walking out the door to Germany. Will try to
answer in a day or two ;-)
Hi Dan, thanks for replying -- be gentle, I probably sounded like some
fresh kid talking nonsense about the greats from before his time (and
I'm not young! Thanks to Mark for reciting history back to
Smalltalk-72 and making me feel young, anyway ;-)). My copy of HOPL-II
was at work, so I did not check the date of Alan's quote, and probably
utterly wrongly assumed Smalltalk-80 blocks were the FEXPR successors.
You guys are my heroes for Smalltalk (all vintages, but I only know
80, used briefly on Sun workstations when I was in grad school; sadly,
my BYTE magazine with the balloons on the cover succumbed to mildew
years ago). Peace,
Jason Orendorff pointed out
wiki.mozilla.org/Jsctypes/api#User-defined_types
from the "ctypes" (libffi) bindings for SpiderMonkey docs. Excerpt
below. Ctypes is overkill (and unsafe to boot!) but the ability to
generate array and struct types is pretty much what I've been sketching.
/be
....
new ctypes.ArrayType(t, n) - Return the array type t[n]. If t is not a
type or t.size is undefined or n is not a size value (defined below),
throw a TypeError. If the size of the resulting array type, in bytes,
would not be exactly representable both as a size_t and as a
JavaScript number, throw a RangeError.
A size value is either a non-negative, integer-valued primitive
number, an Int64 object with a non-negative value, or a UInt64 object.
(Array types with 0 elements are allowed. Rationale: C/C++ allow them,
and it is convenient to be able to pass an array to a foreign
function, and have it autoconverted to a C array, without worrying
about the special case where the array is empty.)
new ctypes.StructType(name, fields) - Create a new struct type with
the given name and fields. fields is an array of field descriptors, of
the format
[ { field1: type1 }, { field2: type2 }, ... ]
where fieldn is a string denoting the name of the field, and typen is
a ctypes type. js-ctypes calculates the offsets of the fields from its
encyclopedic knowledge of the architecture's struct layout rules. If
name is not a string, or any typen is such that typen.size is
undefined, throw a TypeError. If the size of the struct, in bytes,
would not be exactly representable both as a size_t and as a
JavaScript number, throw a RangeError.
(Open issue: Specify a way to tell ctypes.StructType to use #pragma
pack(n).)
These constructors behave exactly the same way when called without the
new keyword.
Brendan Eich wrote:
There's an issue with float32 -- IIRC it does not project into float64 (AKA double). All the other types (uint32, what I schematically named "u32" above, etc.) do.
What? The set of all float32's is a proper subset of the set of all float64's.
When devising scalar types for ES4, I had a devil of a time with int64's. Those (and uint64's) do not form a subset of float64's.
WaldemarBrendan:
To clear up something that misled me when reading the first few messages of this struct thread:
I assume that the proposal is that:
const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32", z:"u32", r:"u8", g:"u8", b:"u8", a:"u8"})); let a = new TA(...);
b = a[i]; a[i].x += 1;
Now b.x also reflects the new value of a[i].x; i.e. b is an alias, not a mutable copy. The same thing would happen if one had mutated b.x.
That leads to a couple plausible meanings of b === a[i]:
-
Pointer equality: True if and only if mutating one's contents would mutate the other's.
-
Recursive value equality: True if and only if all of their constituent elements are ===. I hope that you can't have things like aliases themselves as first-class values inside structs, as that could create cycles (although you could still make === work in that case if you really wanted to).
Bit-by-bit equality is not desirable in the presence of NaN's and signed zeroes. If we go with choice 2, the struct {x:17, y:NaN} should not be === to {x:17, y:NaN}. Using bit equality, it might or might not be ===, which then could be used to discriminate among different kinds of NaN's. There is currently nothing in the language with that power and I'd prefer to keep it that way.
WaldemarOn 06/01/2010 09:15 PM, Waldemar Horwat wrote:
Brendan:
To clear up something that misled me when reading the first few messages of this struct thread:
I assume that the proposal is that:
const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32", z:"u32", r:"u8", g:"u8", b:"u8", a:"u8"})); let a = new TA(...);
b = a[i]; a[i].x += 1;
Now b.x also reflects the new value of a[i].x; i.e. b is an alias, not a mutable copy.
That's a valid choice, but one that can't be applied 100% consistently. For example, wouldn't the following create a mutable copy?
a[0] = b;
I'll also assume that all aliases will pin the entire array for the purposes of garbage collection.
I'll still maintain that the choice that ECMA 334 takes, namely that the assignment to b in the example above, makes a mutable copy is a valid choice.
- Sam Ruby
On Jun 1, 2010, at 5:48 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
There's an issue with float32 -- IIRC it does not project into
float64 (AKA double). All the other types (uint32, what I
schematically named "u32" above, etc.) do.What? The set of all float32's is a proper subset of the set of all
float64's.When devising scalar types for ES4, I had a devil of a time with
int64's. Those (and uint64's) do not form a subset of float64's.
Ok, thanks -- I was misremembering your devil-of-a-time.
On Tue, Jun 1, 2010 at 6:29 PM, Sam Ruby <rubys at intertwingly.net> wrote:
On 06/01/2010 09:15 PM, Waldemar Horwat wrote:
I assume that the proposal is that:
const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32", z:"u32", r:"u8", g:"u8", b:"u8", a:"u8"})); let a = new TA(...);
b = a[i]; a[i].x += 1;
Now b.x also reflects the new value of a[i].x; i.e. b is an alias, not a mutable copy.
That's a valid choice, but one that can't be applied 100% consistently. For example, wouldn't the following create a mutable copy?
a[0] = b;
The way this works in js-ctypes is that b = a[i]; causes b to be a struct object that aliases part of the array a; and a[0] = b; copies an element from a[i] to a[0].
Neither creates a mutable copy in a new buffer.
I'll also assume that all aliases will pin the entire array for the purposes of garbage collection.
This is how js-ctypes does it.
I'll still maintain that the choice that ECMA 334 takes, namely that the assignment to b in the example above, makes a mutable copy is a valid choice.
I would expect a[0].x = 3; to modify a[0], not a temporary copy of a[0]. How do you propose to make that work in ES?
-j js-ctypes: wiki.mozilla.org/Jsctypes/api
On 06/02/2010 03:52 AM, Jason Orendorff wrote:
I'll still maintain that the choice that ECMA 334 takes, namely that the assignment to b in the example above, makes a mutable copy is a valid choice.
I would expect a[0].x = 3; to modify a[0], not a temporary copy of a[0]. How do you propose to make that work in ES?
I'll note that that is not the way strings work today:
a = "abc'; a[0] = 'x';
That being said, I'll agree that a[0].x = 3 would be a handy thing to have. (The clumsy alternative would be to require users to do a[0] = new TA(...);).
-j js-ctypes: wiki.mozilla.org/Jsctypes/api
Thanks for the pointer to js-ctypes! I was unaware of this previously.
- Sam Ruby
On Wed, Jun 2, 2010 at 7:02 AM, Sam Ruby <rubys at intertwingly.net> wrote:
On 06/02/2010 03:52 AM, Jason Orendorff wrote:
I'll still maintain that the choice that ECMA 334 takes, namely that the assignment to b in the example above, makes a mutable copy is a valid choice.
I would expect a[0].x = 3; to modify a[0], not a temporary copy of a[0]. How do you propose to make that work in ES?
I'll note that that is not the way strings work today:
a = "abc'; a[0] = 'x';
Strings are immutable, so I'm not sure they're a good guide here.
Mike
On Jun 2, 2010, at 4:02 AM, Sam Ruby wrote:
On 06/02/2010 03:52 AM, Jason Orendorff wrote:
I'll still maintain that the choice that ECMA 334 takes, namely that the assignment to b in the example above, makes a mutable copy is a valid choice.
I would expect a[0].x = 3; to modify a[0], not a temporary copy of a[0]. How do you propose to make that work in ES?
I'll note that that is not the way strings work today:
a = "abc'; a[0] = 'x';
That being said, I'll agree that a[0].x = 3 would be a handy thing
to have. (The clumsy alternative would be to require users to do
a[0] = new TA(...);).
It's not a matter of "handy", although WebGL people would agree. If
the struct is mutable then implementations can't copy it if it has
reference semantics, and programmers have to copy it to avoid
"behind my back" changes. That's why we seemed to agree last fall (and
it works for decimal) that value types should be shallowly frozen.
There's no issue if we separate value types from structs-for-WebGL,
but perhaps that is still premature. What I'd like to get back to is
"value types are shallowly frozen", though. Otherwise we are
introducing a new optimization and user-programming hazard to the
language, beyond what objects as reference types created.
-j js-ctypes: wiki.mozilla.org/Jsctypes/api
Thanks for the pointer to js-ctypes! I was unaware of this previously.
Noted on this list previously: esdiscuss/2010-May/011310 :
On Jun 2, 2010, at 7:50 AM, Brendan Eich wrote:
There's no issue if we separate value types from structs-for-WebGL,
but perhaps that is still premature. What I'd like to get back to is
"value types are shallowly frozen", though. Otherwise we are
introducing a new optimization and user-programming hazard to the
language, beyond what objects as reference types created.
Sam pointed out in private mail that (my interpretation here)
regardless of value types being frozen, the structs for WebGL idea has
aspects of value types -- the structs in a typed array are allocated
in-line, that's the whole point -- and of reference types via element
extraction reifying a "view object" by which you can mutate the packed
data.
So either we lose this refactoring equivalence:
b = a[i]; b.x = 42; assert(a[i].x === 42);
This assertion botches with Sam's proposed semantics.
Or else we lose the other equivalence, if we reify a struct-view- object on element extraction (rvalue):
a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42);
This example is just like the one above, but uses a[j] for some valid
index j, instead of a b temporary. Note that with objects (reference
types), specifically a plain old Array of object instances, the
assertion holds. But with the sketchy struct semantics I've been
proposing, typed-array-of-struct elements behave like value types when
assigned to (lvalues), so this assertion botches .
Structs can't be fully (mutable) value types without breaking one
equivalence. Yet they can't be reference types or we lose the critical
in-line allocation and packing that WebGL needs, but this leaves them
in limbo, breaking another equivalence.
On Jun 1, 2010, at 6:15 PM, Waldemar Horwat wrote:
b = a[i]; a[i].x += 1;
Now b.x also reflects the new value of a[i].x; i.e. b is an alias,
not a mutable copy. The same thing would happen if one had mutated
b.x.That leads to a couple plausible meanings of b === a[i]:
- Pointer equality: True if and only if mutating one's contents
would mutate the other's.
That is a good point. Since WebGL demands a[i] be stored as packed
machine types in-line in a vector of such structs, this is really
asking whether b aliases a[i]. We could make this work.
- Recursive value equality: True if and only if all of their
constituent elements are ===.
This is what Sam suggested, citing ECMA-334.
I hope that you can't have things like aliases themselves as first- class values inside structs, as that could create cycles (although
you could still make === work in that case if you really wanted to).
The only types WebGL wants inside structs are machine scalar types,
ints and floats of several sizes. But if we take a broader view, we
could try to define structs that can contain (typed arrays of) (other)
structs, so long as everything bottomed out in primitive types.
The complexity of object-reference-typed struct members does not seem
worth it.
Bit-by-bit equality is not desirable in the presence of NaN's and
signed zeroes. If we go with choice 2, the struct {x:17, y:NaN}
should not be === to {x:17, y:NaN}. Using bit equality, it might or
might not be ===, which then could be used to discriminate among
different kinds of NaN's. There is currently nothing in the
language with that power and I'd prefer to keep it that way.
Ok.
On 06/02/2010 02:03 PM, Brendan Eich wrote:
On Jun 2, 2010, at 7:50 AM, Brendan Eich wrote:
There's no issue if we separate value types from structs-for-WebGL, but perhaps that is still premature. What I'd like to get back to is "value types are shallowly frozen", though. Otherwise we are introducing a new optimization and user-programming hazard to the language, beyond what objects as reference types created.
Sam pointed out in private mail that (my interpretation here) regardless of value types being frozen, the structs for WebGL idea has aspects of value types -- the structs in a typed array are allocated in-line, that's the whole point -- and of reference types via element extraction reifying a "view object" by which you can mutate the packed data.
So either we lose this refactoring equivalence:
b = a[i]; b.x = 42; assert(a[i].x === 42);
This assertion botches with Sam's proposed semantics.
"proposed" is a bit more than I had intended. My intent was merely to inquire if the usage of the word "struct" in this context matches the usage of that term in another ECMA standard that I was familiar with. I seem to have implied much more than that, for which I apologize.
Or else we lose the other equivalence, if we reify a struct-view-object on element extraction (rvalue):
a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42);
This example is just like the one above, but uses a[j] for some valid index j, instead of a b temporary. Note that with objects (reference types), specifically a plain old Array of object instances, the assertion holds. But with the sketchy struct semantics I've been proposing, typed-array-of-struct elements behave like value types when assigned to (lvalues), so this assertion botches .
FWIW, I believe that in C#, both assertions would fail.
Structs can't be fully (mutable) value types without breaking one equivalence. Yet they can't be reference types or we lose the critical in-line allocation and packing that WebGL needs, but this leaves them in limbo, breaking another equivalence.
/be
- Sam Ruby
Brendan Eich wrote:
So either we lose this refactoring equivalence:
b = a[i]; b.x = 42; assert(a[i].x === 42);
This assertion botches with Sam's proposed semantics.
Or else we lose the other equivalence, if we reify a struct-view-object on element extraction (rvalue):
a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42);
This example is just like the one above, but uses a[j] for some valid index j, instead of a b temporary. Note that with objects (reference types), specifically a plain old Array of object instances, the assertion holds. But with the sketchy struct semantics I've been proposing, typed-array-of-struct elements behave like value types when assigned to (lvalues), so this assertion botches .
Structs can't be fully (mutable) value types without breaking one equivalence. Yet they can't be reference types or we lose the critical in-line allocation and packing that WebGL needs, but this leaves them in limbo, breaking another equivalence.
Aargh, what fun! This is one of the more fascinating aspects of not having type annotations.
Possibly the most ECMAScripty approach would be to break the first equivalence by making the step that converts lvalues to rvalues convert struct references into copies of structs. In ES5 terms this would be the GetValue internal call. Struct references would not be first class any more than lvalues are.
Waldemar
On Jun 2, 2010, at 11:14 AM, Brendan Eich wrote:
On Jun 1, 2010, at 6:15 PM, Waldemar Horwat wrote:
b = a[i]; a[i].x += 1;
Now b.x also reflects the new value of a[i].x; i.e. b is an alias, not a mutable copy. The same thing would happen if one had mutated b.x.
That leads to a couple plausible meanings of b === a[i]:
- Pointer equality: True if and only if mutating one's contents would mutate the other's.
That is a good point. Since WebGL demands a[i] be stored as packed machine types in-line in a vector of such structs, this is really asking whether b aliases a[i]. We could make this work.
I cannot think of a case where b should not be an alias to a[i] if b is not an alias to a[i] then the code
a[i].x += 1;
Could not modify a[i] itself, effectively we've created an immutable type.
Honestly i think the semantics of the current definition are effectively something along the lines of the following: /* Given: const TA = Array.newTypedArray(fixed_length, Object.newStructType({x:"u32", y:"u32", z:"u32", r:"u8", g:"u8", b:"u8", a:"u8"})); */ Array.newTypedArray = function(length, elementGenerator) { var result = {}; Object.defineProperty(result, "length", {value: length, writable: false, configurable: false, enumerable: false}); for (var i = 0; i < length; i++) { result[i] = elementGenerator(); } return result; }
Object.newStructType = function(description) { return function() { var result = {}; var value = 0; for (var property in description) { if (!description.hasOwnProperty(property)) continue; Object.defineProperty(result, property, { get:function(){ return value; }, set:function(coercer) { return function(v){ value = coercer(v); }; }(description[property]) }); } result.preventExtensions(); return result; } }
function coercer(typeDescription) { switch (typeDescription) { case "u32": return function (v){ return [[v ToUInt32]]; } ... } }
The delayed/lazy reification of the object storage in this case would be an implementation optimisation, and would therefore not require any additional changes to semantics of the language.
- Recursive value equality: True if and only if all of their constituent elements are ===.
This is what Sam suggested, citing ECMA-334.
I dislike this concept -- it feels too much like we're trying to bring in a new sense of strict equality.
I hope that you can't have things like aliases themselves as first-class values inside structs, as that could create cycles (although you could still make === work in that case if you really wanted to).
The only types WebGL wants inside structs are machine scalar types, ints and floats of several sizes. But if we take a broader view, we could try to define structs that can contain (typed arrays of) (other) structs, so long as everything bottomed out in primitive types.
The complexity of object-reference-typed struct members does not seem worth it.
I feel this proposal is incredibly complicated for basically the single use case of WebGL. The majority of use cases for typed arrays are for homogenous element types, and I believe most of what WebGL wants could be achieved simply by use of homogeneously typed arrays that were stepped in a way to avoid aliasing (this would not be complicated from either implementation or specification PoV). The usability would likely be better than the current webgl model, it wouldn't require aliasing, and it would require such a complicated addition to the language.
On Jun 2, 2010, at 4:43 PM, Oliver Hunt wrote:
I cannot think of a case where b should not be an alias to a[i] if b is not an alias to a[i] then the code
Should be: I cannot think of a case where b should not be an alias to a[i]. If b is not an alias to a[i] then the code: ...
Hooray for punctuation!
Oliver Hunt wrote:
If b is not an alias to /a[i]/ then the code
a[i].x += 1;
Could not modify /a[i]/ itself, effectively we've created an immutable type.
That does not logically follow. See my previous message for why.
WaldemarOn Jun 2, 2010, at 3:50 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
So either we lose this refactoring equivalence: b = a[i]; b.x = 42; assert(a[i].x === 42); This assertion botches with Sam's proposed semantics. Or else we lose the other equivalence, if we reify a struct-view- object on element extraction (rvalue): a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42); This example is just like the one above, but uses a[j] for some
valid index j, instead of a b temporary. Note that with objects
(reference types), specifically a plain old Array of object
instances, the assertion holds. But with the sketchy struct
semantics I've been proposing, typed-array-of-struct elements
behave like value types when assigned to (lvalues), so this
assertion botches . Structs can't be fully (mutable) value types without breaking one
equivalence. Yet they can't be reference types or we lose the
critical in-line allocation and packing that WebGL needs, but this
leaves them in limbo, breaking another equivalence.Aargh, what fun! This is one of the more fascinating aspects of not
having type annotations.Possibly the most ECMAScripty approach would be to break the first
equivalence by making the step that converts lvalues to rvalues
convert struct references into copies of structs. In ES5 terms this
would be the GetValue internal call. Struct references would not be
first class any more than lvalues are.
For an expression a[i]j.x, the ES specs all evaluate a[i], call
GetValue on it if a Reference, then make a new Reference with the
result as base and 'x' as the propertyName. Only then, depending on
whether a[i].x occurs as the left-hand side of assignment or in a RHS
expression, e.g., would we PutValue or GetValue.
For what you suggest (which was Sam's thought, inspired by C#), we
would need to preserve the a[i] struct reference instead of calling
GetValue and copying to a fresh struct value. We'd need to factor the
internal methods so that the value of the expression to the left of .
followed by Identifier could remain a struct reference (presumably the
same goes for [ instead of .), while any other struct reference result
would be "dereferenced" by creating a copy.
IINM this was Oliver's point in reply. He was advocating the alias
alternative, where structs reify as "view objects", fat pointers or
aliases. Either way could be made to work.
Which is more ECMAScripty is not clear to me, due to both equivalences
being useful for arrays of objects (arrays of true reference types). I
do not think GetValue, an internal spec method, supports the user's
principle of least astonishment. I rather suspect that implicit struct
copies will be surprising, given their mutability (yet no aliasing of
the original element of the array whence they came).
On Jun 2, 2010, at 10:26 PM, Brendan Eich wrote:
For an expression a[i]j.x,
Sorry, obvious typo adding a 'j' there (keyboard malfunction, and hard
to see in the font I'm looking at!). No confusion in the rest:
On Jun 2, 2010, at 4:43 PM, Oliver Hunt wrote:
I feel this proposal is incredibly complicated for basically the
single use case of WebGL. The majority of use cases for typed
arrays are for homogenous element types, and I believe most of what
WebGL wants could be achieved simply by use of homogeneously typed
arrays that were stepped in a way to avoid aliasing (this would not
be complicated from either implementation or specification PoV).
The usability would likely be better than the current webgl model,
it wouldn't require aliasing, and it would require such a
complicated addition to the language.
It would help for the WebGL use case to be simplified to unify typed
arrays and array buffers, eliminating aliasing and requiring monotyped
arrays.
But IIRC WebGL folks other than yourself (who should please speak
up ;-) can't take the hit of, e.g. (r << 24) | (g << 16) | (b << 8) |
a and then a 32-bit store via assignment to the "rgba" element of what
is ideally a typed array of {x, y, z: uint32; r, g, b, a: uint8}
structs (to abuse syntax further), but which per your suggestion above
would have to be a uint32 array with x, y, z, and rgba elements stored
in sequence.
A sufficiently optimizin JS VM could perhaps recognize all those
shifts and ORs and issue byte stores instead. I'm assuming issuing the
narrower stores beats all the shifty ALU ops and then one 32-bit store.
This could all be "optimization", an implementation issue. Except
WebGL needs speed, and this need is evident in the semantics of
proposals such as typed arrays.
Ignoring all such optimization guarantees, the shift and OR code is
ugly and verbose. So there's a usability argument to be made for
arrays of structs, if not typed arrays as proposed.
As your amusing ES5 metaprogramming code shows, we could support
arrays of structs using objects, and leave optimization out of the
spec. In doing this we'd be preserving the "b = a[i]; b.x = 42;
assert(a[i].x === 42)" behavior.
We would break "a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42)" for
j != i and j < a.length. But perhaps that is not a useful equivalence.
Or we could require an explicit copy operation to assign from a[i] to
a[j], or throw an error (but the last seems gratuitous since one can
introduce a temporary b as workaround).
Implicit struct copy seems worse than reified object as fat-pointer.
Implicit copying could be quite expensive, even if optimized away for
"a[i].x" but not "b = a[i]".
Bottom line: it still seems to me that the WebGL use-case is C-like
(GL was a C API back in the day), and wants arrays (C vectors) of
structs that are allocated in-line and packed full of machine types,
and possibly even sub-structs or sub-typed-arrays. Vlad averred as
much recently.
The current typed array views into array buffers, besides reminding
Tucker of Common Lisp "displaced arrays", smell of Fortran common
blocks holding overlayed vectors. Surely we can do better!
On Jun 3, 2010, at 2:19 PM, Brendan Eich wrote:
On Jun 2, 2010, at 4:43 PM, Oliver Hunt wrote:
I feel this proposal is incredibly complicated for basically the single use case of WebGL. The majority of use cases for typed arrays are for homogenous element types, and I believe most of what WebGL wants could be achieved simply by use of homogeneously typed arrays that were stepped in a way to avoid aliasing (this would not be complicated from either implementation or specification PoV). The usability would likely be better than the current webgl model, it wouldn't require aliasing, and it would require such a complicated addition to the language.
It would help for the WebGL use case to be simplified to unify typed arrays and array buffers, eliminating aliasing and requiring monotyped arrays.
But IIRC WebGL folks other than yourself (who should please speak up ;-) can't take the hit of, e.g. (r << 24) | (g << 16) | (b << 8) | a and then a 32-bit store via assignment to the "rgba" element of what is ideally a typed array of {x, y, z: uint32; r, g, b, a: uint8} structs (to abuse syntax further), but which per your suggestion above would have to be a uint32 array with x, y, z, and rgba elements stored in sequence.
My proposal would have you do something along the lines of: buffer = new ArrayBuffer({x, y, z: uint32; r, g, b, a: uint8} ) // using your abused syntax
producing an object where you would do: buffer.x[i] = ...; buffer.y[i] = ...; ... buffer.r[i] = ...; buffer.g[i] = ...;
etc
This could all be "optimization", an implementation issue. Except WebGL needs speed, and this need is evident in the semantics of proposals such as typed arrays.
Ignoring all such optimization guarantees, the shift and OR code is ugly and verbose. So there's a usability argument to be made for arrays of structs, if not typed arrays as proposed.
I'm not sure where this shift/or stuff comes from?
As your amusing ES5 metaprogramming code shows, we could support arrays of structs using objects, and leave optimization out of the spec. In doing this we'd be preserving the "b = a[i]; b.x = 42; assert(a[i].x === 42)" behavior.
We would break "a[j] = a[i]; a[j].x = 42; assert(a[i].x === 42)" for j != i and j < a.length. But perhaps that is not a useful equivalence. Or we could require an explicit copy operation to assign from a[i] to a[j], or throw an error (but the last seems gratuitous since one can introduce a temporary b as workaround).
Implicit struct copy seems worse than reified object as fat-pointer. Implicit copying could be quite expensive, even if optimized away for "a[i].x" but not "b = a[i]".
Bottom line: it still seems to me that the WebGL use-case is C-like (GL was a C API back in the day), and wants arrays (C vectors) of structs that are allocated in-line and packed full of machine types, and possibly even sub-structs or sub-typed-arrays. Vlad averred as much recently.
The current typed array views into array buffers, besides reminding Tucker of Common Lisp "displaced arrays", smell of Fortran common blocks holding overlayed vectors. Surely we can do better!
My intention was that you end up with a "struct of arrays" rather than an "array of structs" -- internally the implementation would be using flat storage in the form requested. The exposed accessor for a given field array would be something akin to
return toJSValue((MachineType)(data + index * structSize + fieldOffset))
On Jun 3, 2010, at 2:42 PM, Oliver Hunt wrote:
My proposal would have you do something along the lines of: buffer = new ArrayBuffer({x, y, z: uint32; r, g, b, a: uint8} ) //
using your abused syntaxproducing an object where you would do: buffer.x[i] = ...; buffer.y[i] = ...; ... buffer.r[i] = ...; buffer.g[i] = ...;
Hack-a-rama! :-P
It still seems to me, whether you use overlayed typed arrays that
users never intentionally use to alias the same given machine-typed
fields, or a "transpose" of such overlayed typed arrays: monotyped
arrays that stride with the right skews to access the members; that we
are just perpetrating the lowest-level hack possible to satisfy the
immediate use-case, without counting usability costs or considering
future-hostility to obvious evolutionary pathways.
This could all be "optimization", an implementation issue. Except
WebGL needs speed, and this need is evident in the semantics of
proposals such as typed arrays.Ignoring all such optimization guarantees, the shift and OR code is
ugly and verbose. So there's a usability argument to be made for
arrays of structs, if not typed arrays as proposed.I'm not sure where this shift/or stuff comes from?
If you could say "use a uint32 vector" but every fourth element was
not x, y, or z but rather a quadruple of bytes (rgba), then these
shifts and ORs would be needed. The idea would be to simplify typed
arrays to monotyped, non-aliasing vectors of machine types.
My intention was that you end up with a "struct of arrays" rather
than an "array of structs" -- internally the implementation would be
using flat storage in the form requested. The exposed accessor for
a given field array would be something akin toreturn toJSValue((MachineType)(data + index * structSize +
fieldOffset))
a.x[i] instead of a[i].x doesn't really express this computation, but
never mind. It still seems hard to nest structs or typed arrays in
structs (in typed arrays), or otherwise future-proof this design.
Vlad had WebGL wish-list examples like this:
typedef struct { float32_t x, y, z; } vec3_t;
typedef struct { uint8_t r, g, b, a; } color_t;
struct { vec3_t vertex; vec3_t normal; color_t color; } vertex_element_t;
Making people code accesses to such data in some Fortran-ish way is
just awful. I would rather we do something like a subset of Mozilla's
jsctypes, designed by Jason Orendorff, mentioned previously:
wiki.mozilla.org/Jsctypes/api#User-defined_types
Some careful thought went into this design. Jason should weigh in here.
On 05/27/2010 03:05 PM, Brendan Eich wrote:
I'll note that with newStructType one could define a 128 bit quantity for representing a complex number. That coupled with something like the following proposal would enable a complete set of complex arithmetic operators to be supported:
esdiscuss/2009-January/008535
On 05/27/2010 03:05 PM, Brendan Eich wrote: > On May 27, 2010, at 11:38 AM, Sam Ruby wrote: > >>> Well-supported arrays of structs (as schemas) might be a satisfactory >>> compromise. Consensus emerging around that solution. >> >> Structs in ECMA-334 are value types, and consist of members that also >> are value types. Would structs in future revisions of ECMA-262 share >> these characteristics? >> >> Is it fair to assume that there would end up being a more richer set >> of primitives to select from when composing a struct than simply >> "object", "boolean", "number", "string" and the like? Again, ECMA-334 >> defines the following: >> >> http://en.csharp-online.net/ECMA-334:_11.1.5_Integral_types >> >> Would something similar be envisioned for ECMA-262? > > No, we did not envision adding more primitive types, type annotations, > conversion rules, and first-class struct declarations. > > Something more like > > const TA = > Array.newTypedArray(fixed_length, > Object.newStructType({x:"u32", y:"u32", z:"u32", > r:"u8", g:"u8", b:"u8", a:"u8"})); > let a = new TA(...); > > ... a[i].x ... I'll note that with newStructType one could define a 128 bit quantity for representing a complex number. That coupled with something like the following proposal would enable a complete set of complex arithmetic operators to be supported: https://mail.mozilla.org/pipermail/es-discuss/2009-January/008535.html - Sam Ruby