typed objects and value types
That is very helpful, thanks! I agree with a lot with what you say. But to expand on my reply to your blog post, I have one fundamental concern: I think the idea that value types should be objects would require stretching the notion of object too far, and violate one of the fundamental properties that objects currently have. Namely, objects have generative identity.
Other properties, like the ability to use an object as keys in weak collections, follow from that. If we suddenly allowed objects that do not have identity, but a structural notion of equality, then we could not allow these objects into weak collections. That would break a rather fundamental assumption, and make a distinction between different sorts of objects that does not exist before.
It would also have other unpleasant consequences. For example, if value types yield objects, then these objects could be proxied. But a proxy could not simulate their structural notion of equality, at least not right now. So we'd need to extend the proxy API. Being objects, you could also use value type instances as prototypes, which seems a rather useless feature as well, but would potentially cause a lot of pain for VMs.
(For what's it worth, all these arguments also apply to typed objects and their interpretation as fat pointers, which are essentially a special case of value type.)
Either way, I think we should refrain from introducing a completely new, hybrid form of value that behaves neither like a primitive nor like a proper object. To me, it seems a much more elegant and economic model to extend the notion of primitive type. That is, make value types primitive-like, use the existing wrapping semantics, let them have an implicit prototype link, and typeof different from "object" (whether something fixed or actually user-defined is debatable).
On Wed, Apr 2, 2014 at 5:01 PM, Andreas Rossberg <rossberg at google.com>wrote:
Other properties, like the ability to use an object as keys in weak collections, follow from that. If we suddenly allowed objects that do not have identity, but a structural notion of equality, then we could not allow these objects into weak collections. That would break a rather fundamental assumption, and make a distinction between different sorts of objects that does not exist before.
I never had a strong preference either way, and making typed objects and value objects "objects" simplifies the spec vastly. But weak maps semantics really pushes the needle towards value and type objects not being objects.
Dmitry
On Wed, Apr 2, 2014 at 7:32 AM, Niko Matsakis <niko at alum.mit.edu> wrote:
I just wanted to let people on es-discuss know about two of my recent blog posts concerning typed objects. The first is a kind of status report:
smallcultfollowing.com/babysteps/blog/2014/04/01/typed-objects-status-report
and the second details some (preliminary) thoughts on how one could build on typed objects to support user-defined value types:
smallcultfollowing.com/babysteps/blog/2014/04/01/value-types-in-javascript
Just a quick local editorial comment for now:
... points[0].x = 1; ... But assigning to a field of a value object like temp has no effect, and hence the assignment is lost.
Please assume (and state if you feel it is needed) that all example code is strict code. Introducing abstractions, like value types, that cause assignments such as this to fail would be insane if the clients of these abstractions were sloppy. If the client code were sloppy, these failures would be silent as you state, and the code would proceed on execution paths that assume success. By assuming that client code is sloppy, you unnecessarily burden, distract, and mislead the reader, as not one in a thousand truly understands the semantics of sloppy mode.
Substantive comments later. Quick reaction: I like it!
I really appreciate you guys making === (not just ==) work for value types. I think that drastically improves the usability compared to earlier proposals.
I can see Andreas's concerns regarding what that means for their object-ness though. But hopefully that can be worked out while still keeping a useful ===.
I just got to the Appendix B at the end of your article, so I see you were aware of the strictness issue. The semantics of a failed assignment to a field of a value type should indeed just be the semantics of a failed assignment: if the assignment is performed by sloppy code, the failure is silent. If the failed assignment is performed by strict code, it throws.
Commenting on this same region:
print(points[0].x); // 0 to start
points[0].x = 1;
print(points[0].x); // still 0
points[0] = {x: 1, y: 2};
print(points[0].x); // now 1
There's no reason why we couldn't extend the grammar to handle this
case, in the same way that a.f() has different semantics than f = a.f; f();. That is, use the [Reference Specification Type] to treat
field assignment differently, so that the semantics of:
points[0].x = 1;
are the same as:
points[0] = { x: 1, y: points[0].y }
For my own education, here's a rough draft of how the [runtime semantics of assignment] might look:
AssignmentExpression[In, Yield] : LeftHandSideExpression[?Yield] = AssignmentExpression[?In, ?Yield]
- Let lref be the result of evaluating LeftHandSideExpression
- ReturnIfAbrupt(lref)
- If Type(lref) is Reference, then a. Let baseref be GetBaseReference(lref) b. If baseref is a reference to a value type, then i. Let name be GetReferencedName(lref) ii. Let oldval be GetValue(baseref) iii. Let rref be the result of evaluating AssignmentExpression iv. ReturnIfAbrupt(rref) v. Let rval be GetValue(rref) vi. Let newval be a value type identical to oldval except that field name is set to rval vii. Let status be PutValue(baseref, newval) viii. ReturnIfAbrupt(status) ix. Return rval etc
Note that in step 3a I need to get a reference to the base value, rather than using GetValue(lref). So the Reference specification type would have to cover two levels of indirection rather than one. --scott
[Reference Specification Type]: people.mozilla.org/~jorendorff/es6-draft.html#sec-reference-specification-type) [runtime semantics of assignment]: people.mozilla.org/~jorendorff/es6-draft.html#sec
quick feedback
- paper: all Cartesian examples forget to retrieve/assign this.x and
this.y, i.e.
Cartesian . prototype .toPolar = function() {var r = Math.sqrt(x*x + y*y);}... I can guess those are initial own properties but after reading all examples without context I started doubting about it - no idea why types shouldn't be initialized either with or without
new, which is the most natural way to write objects in JS since kinda ever (new Point() VS Point() ... how about it's just the same as it is for new Object() VS Object() ... if String comes out as argument I call it shenanigans) - I'd love to see this stuff in ASAP but I keep wondering why nobody is planning to show the type in these properties descriptors ... as discussed before, so that ...
var a = new Int32Array(1);
a[0] = 1;
Object.getOwnPropertyDescriptor(a, '0');
// produces
Object {
value: 1,
writable: true,
enumerable: true,
configurable: true,
type: int32 // <========= this !
}
Overall I appreciate the work and the progress behind this topic, good stuff.
Best
I think the idea that value types should be objects would require stretching the notion of object too far, and violate one of the fundamental properties that objects currently have.
Certainly this was the idea, though I didn't think of it is as violating, but rather... stretching. :)
Other properties, like the ability to use an object as keys in weak collections, follow from that. If we suddenly allowed objects that do not have identity, but a structural notion of equality, then we could not allow these objects into weak collections.
Interesting point. I hadn't thought about weakmaps (though it seems like an obvious next step from thinking about maps in general).
Either way, I think we should refrain from introducing a completely new, hybrid form of value that behaves neither like a primitive nor like a proper object.
I will ponder this for a bit and write another e-mail shortly. :) I want to try and draw up what I see as the family of alternatives. I feel like to some extent the question of "primitive or object?" is perhaps a distraction, though I made it sort of central to the blog post for some reason. The real question is to derive the properties that user-defined values must have, and then we can see whether it seems to fit better into the system as a whole to stretch primitives or to stretch values.
Along those lines, user-defined values (uvalues) probably ought to have the following characteristics:
- uvalues are (shallowly) immutable
- uvalues have no "generative identity", and are compared using "by-value" ===
- uvalues have a prototype linked to their def'n
- the syntax 'uvalue.foo' can be used to access members of this prototype
- when transmitted between realms, uvalues retain this link
- uvalues cannot go into weak maps, but can go into other maps/sets
Does this sound about right?
Clearly I left open the question of what typeof yields (which I don't
feel especially strongly about, actually, I don't have much intution
for what typeof means to people) and whether uvalue.foo works
via a wrapper or some other mechanism.
Niko
On 4/2/2014 8:01 AM, Andreas Rossberg wrote:
That is very helpful, thanks! I agree with a lot with what you say. But to expand on my reply to your blog post, I have one fundamental concern: I think the idea that value types should be objects would require stretching the notion of object too far, and violate one of the fundamental properties that objects currently have. Namely, objects have generative identity.
...
Either way, I think we should refrain from introducing a completely new, hybrid form of value that behaves neither like a primitive nor like a proper object. To me, it seems a much more elegant and economic model to extend the notion of primitive type. That is, make value types primitive-like, use the existing wrapping semantics, let them have an implicit prototype link, and typeof different from "object" (whether something fixed or actually user-defined is debatable).
It seems like we're trying to have our cake and eat it too; we want these binary data objects to act both like primitives and objects at the same time. We want them to be comparable by value, but we also want them to be mutable. The second article proposes doing this by drawing a line in the sand between mutable object-like versions of a Type and immutable primitive-like versions of a Type. I think this is problematic because it seems like a kludge to get something kinda-sorta like what we want but not really, and also because it introduces two things that look very similar but that act completely different when it comes to identity.
const Color = new StructType({ red: uint8, green: uint8, blue: uint8 });
const ColorV = Color.valueType();
const blue = Color({ blue: 255 });
blue === Color({ blue: 255 }); // false
const bluev = Color({ blue: 255 });
bluev === ColorV({ blue: 255 }); // true
blue === bluev; // false
// I would expect to end up needing helpers like:
function eq(a, b) {
var TypeV = a.constructor.valueType();
return TypeV(a) === TypeV(b); // eww
}
eq(blue, bluev); // true
What I really want to be able to do is something like:
const red = Object.freeze(Color({ red: 255 })); // immutable
const blue = Object.freeze(Color({ blue: 255 })); // immutable
let myColor = Color({ red: 255 }); // mutable
myColor === red; // true
myColor.red = 0;
myColor.blue = 255;
myColor === blue; // true, OOPS new identity!
Making Color produce either objects or primitives is going to break
strong existing invariants. If it makes objects then it breaks
invariants about object identity. If it makes primitives then it breaks
the invariant that primitives don't have mutable state. For either, it
breaks the invariant that for any two values A and B, the result of
doing A === B will never change. I don't think we can/should violate
any of the above invariants and so that leaves us with two options:
reject mutable value types, or give up on using === to structurally
compare them. Choosing the latter seems to naturally lead to the idea of
introducing a structural comparison operator.
If we introduced a new configurable equality operator we might be able
to find a path out of this mess. For the purpose of example, let's name
this operator EQ for now. I imagine EQ is to === as for-of is to
for-in. With for-in and for-of, they both are about iterating over an
object, but they define "iteration" very differently. With === and EQ,
they both are about comparing the equality of an object, but they define
"equality" differently.
Since === already has identity covered, EQ should focus on structural comparisons. I would expect to see it working something like:
[5, 10, 20] EQ [5, 10, 20]; // true
({ x: 10 } EQ { x: 10 }); // true
({ x: 20 } EQ { x: 10 }); // false
true EQ true; // true
// from above
let myColor = Color({ red: 255 }); // mutable
myColor === red; // false
myColor EQ red; // true
myColor.red = 0;
myColor.blue = 255;
myColor EQ blue; // true
This also opens up the door to how comparing different numeric value types can work:
0L === 0; // false
0L EQ 0; // true
I know it's cringe-worthy, adding more operators for something that is ostensibly already well covered by JS, with its already two equality operators and another (or two?) definitions of equality used in the spec. But really, all the existing operators are inextricably linked with identity and having nothing to do with structural comparison, so this really is a different kind of thing.
On Wed, Apr 2, 2014 at 10:37 AM, Niko Matsakis <niko at alum.mit.edu> wrote:
I think the idea that value types should be objects would require stretching the notion of object too far, and violate one of the fundamental properties that objects currently have.
Certainly this was the idea, though I didn't think of it is as violating, but rather... stretching. :)
I find your arguments about why uvalues can't simply be an extension of primitive values convincing. I find Andreas' arguments about why uvalues can't simply be new kinds of objects convincing.
Although I am not suggesting that we import the entire taxonomy used by E, it can help inform the discussion. It is shown by the diagram at < erights.org/elang/kernel/auditors/images/prop-tree.gif> within <
erights.org/elang/kernel/auditors>. (Note that this paper is stale
regarding E itself, but is still probably the best starting point for understanding the distinctions we might want to borrow into JS.) Depending of whether uvalues may or may not contain non-uvalue objects, uvalues correspond either to E's PassByCopy or DeepPassByCopy.
The key point for the current discussion is that both are Selfless, i.e., identity-free, and compare for equality based purely on internal structure. They are also Transparent, which means "non-encapsulating" in the sense of being reproducible purely from client visible data, which is distinct from the implementation-level "opaque" notion your typed objects article.
Together, these ensure that, for a deep uvalue x,
x === unserialize(serialize(x))
for some suitable serialization into a pure data form. This allows deep uvalues to be passed between vats either by copying or by pointer sharing at the implementation's choice, without any observable difference. Between vats within an address space passing by pointer sharing is important for performance. Between vats in different address spaces or on different machines, copying is necessary. It's nice that this difference can be made non-semantic, but see below.
Other properties, like the ability to use an object as keys in weak collections, follow from that. If we suddenly allowed objects that do not have identity, but a structural notion of equality, then we could not allow these objects into weak collections.
Interesting point. I hadn't thought about weakmaps (though it seems like an obvious next step from thinking about maps in general).
Maps and Sets can already use non-identity values as keys with structural equality, i.e., the current primitive values. Thus, Maps and Sets should be able to use uvalues as well.
WeakMaps cannot use uvalues as keys for the same reason they can't use the current primitive values.
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
Either way, I think we should refrain from introducing a completely new, hybrid form of value that behaves neither like a primitive nor like a proper object.
Whatever words we use, the concept we're all trying to define here has some things in common with the current primitive values, and some things in common with current objects.
I will ponder this for a bit and write another e-mail shortly. :) I want to try and draw up what I see as the family of alternatives. I feel like to some extent the question of "primitive or object?" is perhaps a distraction, though I made it sort of central to the blog post for some reason. The real question is to derive the properties that user-defined values must have, and then we can see whether it seems to fit better into the system as a whole to stretch primitives or to stretch values.
Along those lines, user-defined values (uvalues) probably ought to have the following characteristics:
- uvalues are (shallowly) immutable
+1. Whether they are shallow or not, or even whether this question extends the taxonomy, is important to discuss further.
- uvalues have no "generative identity", and are compared using "by-value" ===
+1.
- uvalues have a prototype linked to their def'n
- the syntax 'uvalue.foo' can be used to access members of this prototype
+1.
- when transmitted between realms, uvalues retain this link
A uvalue is deeply uvalue only when its prototype and all the methods found there are uvalues. Although this is the case for E, the implications for JS are, ahem, tricky. If they're not deep in this way, then they can't be passed transparently between vats, address spaces, and machines.
- uvalues cannot go into weak maps, but can go into other maps/sets
+1.
On Wed, Apr 2, 2014 at 8:26 PM, Mark S. Miller <erights at google.com> wrote:
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
It is unclear to me how WeakMaps can ever use typed objects as keys, since typed objects do not have their unique identity. Or rather, the only possible semantics I see is "hold to the value forever". So for example:
w = new WeakMap(); w[Point(buffer, 0)] = "abc"; // 1
w[Point(buffer,0)] = ? // 2
I believe that in line 2, result should always be "abc" (since Point(buffer, 0) === Point(buffer, 0)). This is something we can spec, but it is probably not behavior we want.
Dmitry
On Wed, Apr 2, 2014 at 9:11 PM, Dmitry Lomov <dslomov at chromium.org> wrote:
On Wed, Apr 2, 2014 at 8:26 PM, Mark S. Miller <erights at google.com> wrote:
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
It is unclear to me how WeakMaps can ever use typed objects as keys, since typed objects do not have their unique identity. Or rather, the only possible semantics I see is "hold to the value forever". So for example:
w = new WeakMap(); w[Point(buffer, 0)] = "abc"; // 1
w[Point(buffer,0)] = ? // 2
I believe that in line 2, result should always be "abc" (since Point(buffer, 0) === Point(buffer, 0)). This is something we can spec, but it is probably not behavior we want.
(to put it another way, for "normal" objects/thin pointers, the user cannot construct the new identical thin pointer out of thin air, therefore garbage collection is possible; whereas the typed objects/fat pointers are user-reconstructable at any point of time - so garbage collection is impossible).
You can't recreate a given fat pointer out of thin air. You need at least the same backing buffer, or something (e.g., another fat pointer) which retains the same backing buffer.
The backing buffer has generative identity, so the WeakMap should hold on to the backing buffer weakly[1], but it should use the entire fat pointer to compare for key equality.
[1] In the normal ephemeron sense of "weakly".
Note that I am only pointing out that all this is possible, and is the least surprising extension of the semantics of the various objects involved. If implementors say this isn't worth it and would rather just reject typed objects as keys, I'm ok with that, as long as we all agree to reject.
On Wed, Apr 2, 2014 at 9:36 PM, Mark S. Miller <erights at google.com> wrote:
You can't recreate a given fat pointer out of thin air. You need at least the same backing buffer, or something (e.g., another fat pointer) which retains the same backing buffer.
The backing buffer has generative identity, so the WeakMap should hold on to the backing buffer weakly[1], but it should use the entire fat pointer to compare for key equality.
[1] In the normal ephemeron sense of "weakly".
Note that I am only pointing out that all this is possible, and is the least surprising extension of the semantics of the various objects involved. If implementors say this isn't worth it and would rather just reject typed objects as keys, I'm ok with that, as long as we all agree to reject.
Point taken, yes, there are conditions that ensure that this typed object can never be recreated, but I am not sure this will in practice be the least surprising extension (from the practical point of view of "why my WeakMap still holds on to these values? Oh it is because there is this one typed object that shares the buffer with all these other objects"). From practical standpoint, it feels like once we have made GC observable (through WeakMaps), we just can't have objects (weak map keys) with non-generative identity.
So as an implementor, I feel that we should reject typed objects as weak map keys.
Dmitry
On Wed, Apr 2, 2014 at 1:02 PM, Dmitry Lomov <dslomov at chromium.org> wrote:
On Wed, Apr 2, 2014 at 9:36 PM, Mark S. Miller <erights at google.com> wrote:
You can't recreate a given fat pointer out of thin air. You need at least the same backing buffer, or something (e.g., another fat pointer) which retains the same backing buffer.
The backing buffer has generative identity, so the WeakMap should hold on to the backing buffer weakly[1], but it should use the entire fat pointer to compare for key equality.
[1] In the normal ephemeron sense of "weakly".
Note that I am only pointing out that all this is possible, and is the least surprising extension of the semantics of the various objects involved. If implementors say this isn't worth it and would rather just reject typed objects as keys, I'm ok with that, as long as we all agree to reject.
Point taken, yes, there are conditions that ensure that this typed object can never be recreated, but I am not sure this will in practice be the least surprising extension (from the practical point of view of "why my WeakMap still holds on to these values? Oh it is because there is this one typed object that shares the buffer with all these other objects"). From practical standpoint, it feels like once we have made GC observable (through WeakMaps), we just can't have objects (weak map keys) with non-generative identity.
So as an implementor, I feel that we should reject typed objects as weak map keys.
Reiterating, I am fine with this conclusion. I even buy the perspective in which this is less surprising than my proposal.
However, I have to quibble with "observable". WeakRefs make GC observable. WeakMaps do not. Running out of memory, or failing to, is not an observation, since the spec never says when this might or might not occur anyway. In fact, the spec as written is only faithfully implementable on a machine with an unbounded amount of memory ;). (I regret the missed opportunity to discuss test262 and infinite machines on April 1)
On Wed, Apr 2, 2014 at 10:17 PM, Mark S. Miller <erights at google.com> wrote:
However, I have to quibble with "observable". WeakRefs make GC observable. WeakMaps do not. Running out of memory, or failing to, is not an observation, since the spec never says when this might or might not occur anyway. In fact, the spec as written is only faithfully implementable on a machine with an unbounded amount of memory ;). (I regret the missed opportunity to discuss test262 and infinite machines on April 1)
:) Agreed on all counts. "Observable" in a sense of "from practical standpoint, programmers will try to reason about it, so we should not make their reasoning hard" (or "harder than necessary") [not a quantifiable argument indeed!].
Dmitry
What missing expressive capability does this give developers, aside from overloading of "equality"?
Obviously, being able to directly represent an int33 (for example) is useful, but I'm not seeing the motivation behind user-defined value objects, yet, aside from overloading equality.
On Wed, Apr 2, 2014 at 3:43 PM, Kevin Smith <zenparsing at gmail.com> wrote:
What missing expressive capability does this give developers, aside from overloading of "equality"?
Obviously, being able to directly represent an int33 (for example) is useful, but I'm not seeing the motivation behind user-defined value objects, yet, aside from overloading equality.
Deep uvalues (uvalues whose members are all primitives or deep
uvalues) can be unobservably cached by the UA and reused, because
they're immutable and lack reference identity. This allows for any
number of optimizations, like what was outlined in the blog post
linked in the OP (you can invisibly move a "foo"+"bar" expression
out of a loop, but you can't move an object construction; you can
move a deep uvalue construction).
As noted in the blog post, Brendan's original proposal had fuller
operator overloading, the benefits of which should be obvious. It
also has user-definable numeric suffixes, for "simple" types of
uvalues whose constructor can take a single numeric value. This lets
CSS, for example, define a set of uvalues for all of its dimensions,
so authors can write element.css.width += 50px; rather than
element.style.width = +element.style.width + 50 + "px"; /* sure hope the value was in px! */
On 2 April 2014 19:37, Niko Matsakis <niko at alum.mit.edu> wrote:
Along those lines, user-defined values (uvalues) probably ought to have the following characteristics:
- uvalues are (shallowly) immutable
- uvalues have no "generative identity", and are compared using "by-value" ===
- uvalues have a prototype linked to their def'n
- the syntax 'uvalue.foo' can be used to access members of this prototype
- when transmitted between realms, uvalues retain this link
- uvalues cannot go into weak maps, but can go into other maps/sets
Does this sound about right?
It does.
Clearly I left open the question of what typeof yields (which I don't feel especially strongly about, actually, I don't have much intution for what
typeofmeans to people) and whetheruvalue.fooworks via a wrapper or some other mechanism.
Yes, I understand. I agree that typeof is a somewhat dubious feature. However, there are certain invariants that people seem to rely on regarding typeof, e.g. that testing for type 'function' identifies callables, Not sure what type 'object' is used for in the wild.
I also remember one TC39 discussion from 1+ year ago where we concluded that new future types like int64 should have their own typeof result.
On 2 April 2014 20:26, Mark S. Miller <erights at google.com> wrote:
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
I'm not sure I agree with your notion of identity on pointers here. It doesn't make sense to me, semantically, to characterise structural (fat) pointers as having identity, just because their pointees have. (Or in other words, tuples do not acquire identity just because some of their components are values with identity. This is mixing up domains.)
In any case, from the low-level perspective of the GC, a fat pointer definitely is an object separate from the buffer it points to (though it strongly references this buffer). Weak maps currently work by making the references to their keys weak. But that would simply be incorrect in the case where a key is a fat pointer -- because of its structural equality, an equivalent pointer can be recreated. It is unclear to me how weak maps could (efficiently) be extended to support this particular notion of reverse transitive weakness that you allude to.
- uvalues have a prototype linked to their def'n
- the syntax 'uvalue.foo' can be used to access members of this prototype
+1.
- when transmitted between realms, uvalues retain this link
A uvalue is deeply uvalue only when its prototype and all the methods found there are uvalues. Although this is the case for E, the implications for JS are, ahem, tricky. If they're not deep in this way, then they can't be passed transparently between vats, address spaces, and machines.
How can a prototype possibly be a fully structural value in JavaScript? Especially, since the language has this great feature of making all functions objects with identity?
On the other hand, I also don't see how uvalues can work without a prototype link. So it seems that it's simply impossible to make them transparently transferable between vats in the way you desire (and I would, too). It seems that JavaScript is actively hostile to that idea.
On Thu, Apr 3, 2014 at 8:26 AM, Andreas Rossberg <rossberg at google.com>wrote:
On 2 April 2014 20:26, Mark S. Miller <erights at google.com> wrote:
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
I'm not sure I agree with your notion of identity on pointers here. It doesn't make sense to me, semantically, to characterise structural (fat) pointers as having identity, just because their pointees have. (Or in other words, tuples do not acquire identity just because some of their components are values with identity. This is mixing up domains.)
In any case, from the low-level perspective of the GC, a fat pointer definitely is an object separate from the buffer it points to (though it strongly references this buffer). Weak maps currently work by making the references to their keys weak. But that would simply be incorrect in the case where a key is a fat pointer -- because of its structural equality, an equivalent pointer can be recreated. It is unclear to me how weak maps could (efficiently) be extended to support this particular notion of reverse transitive weakness that you allude to.
They can, in theory - for the sake of argument, let's define "weak reference to fat pointer" to be "weak reference to its backing store + offset" (this is probably even fairly efficiently implementable). The question is, is this a desired semantics, and as we discussed above, it is clearly not.
"True JS object" with generative identity seems to be a nice demarcation line for any notion of weak reference that we have (admittedly, we do not have the notion of weak reference per se in the spec, only weak maps, but willy-nilly the programmers will reason of them in terms of weak references).
- uvalues have a prototype linked to their def'n
- the syntax 'uvalue.foo' can be used to access members of this prototype
+1.
- when transmitted between realms, uvalues retain this link
A uvalue is deeply uvalue only when its prototype and all the methods found there are uvalues. Although this is the case for E, the implications for JS are, ahem, tricky. If they're not deep in this way, then they can't be passed transparently between vats, address spaces, and machines.
How can a prototype possibly be a fully structural value in JavaScript? Especially, since the language has this great feature of making all functions objects with identity?
On the other hand, I also don't see how uvalues can work without a prototype link. So it seems that it's simply impossible to make them transparently transferable between vats in the way you desire (and I would, too). It seems that JavaScript is actively hostile to that idea.
Correct, yes - I think Mark says the same thing when he says "the implications are tricky". "Passing transparently between vats, address spaces, and machines" must be a non-goal for value objects as proposed. That's also how I read "uvalues have a prototype ... when transmitted between realms, uvalues retain this link" - among other things, it means that uvalues from different realms are never equal (since their prototype links are different by virtue of belonging to different realms).
Dmitry
On 3 April 2014 09:00, Dmitry Lomov <dslomov at chromium.org> wrote:
On Thu, Apr 3, 2014 at 8:26 AM, Andreas Rossberg <rossberg at google.com> wrote:
On 2 April 2014 20:26, Mark S. Miller <erights at google.com> wrote:
We could specify that WeakMaps can use typed objects as keys. The current discussion in your article confuses semantics with implementation when it speaks of typed objects comparing structurally. Typed objects have identity, characterized by the four-tuple you explain. Just as all the bits of a thin pointer are significant when comparing two thin pointers to see if they point at the same semantic object identity, so are all the bits in your fat pointer significant. Don't get confused by the bits in the pointer to the fat pointer, when the fat pointer happens to be boxed.
I'm not sure I agree with your notion of identity on pointers here. It doesn't make sense to me, semantically, to characterise structural (fat) pointers as having identity, just because their pointees have. (Or in other words, tuples do not acquire identity just because some of their components are values with identity. This is mixing up domains.)
In any case, from the low-level perspective of the GC, a fat pointer definitely is an object separate from the buffer it points to (though it strongly references this buffer). Weak maps currently work by making the references to their keys weak. But that would simply be incorrect in the case where a key is a fat pointer -- because of its structural equality, an equivalent pointer can be recreated. It is unclear to me how weak maps could (efficiently) be extended to support this particular notion of reverse transitive weakness that you allude to.
They can, in theory - for the sake of argument, let's define "weak reference to fat pointer" to be "weak reference to its backing store + offset" (this is probably even fairly efficiently implementable). The question is, is this a desired semantics, and as we discussed above, it is clearly not.
You'd have to unwrap the whole fat pointer, right? So it would be "weak reference to backing store + offset + opacity + weak reference to prototype or type descriptor + dimensions", or something like that. The fact that there are two weak references in there seems to be asking for trouble already. The entry would die if one of those references dies.
But in any case, since I view fat pointers as just an instance of uvalues, I was considering the more general case of nested uvalues with references to a backing store at (potentially) arbitrary depth. But I see that Mark wasn't talking about that general case, so I suppose that wouldn't matter.
"True JS object" with generative identity seems to be a nice demarcation line for any notion of weak reference that we have (admittedly, we do not have the notion of weak reference per se in the spec, only weak maps, but willy-nilly the programmers will reason of them in terms of weak references).
Obviously, I agree. Although an argument could be made to also allow symbols.
On Thu, Apr 3, 2014 at 9:40 AM, Andreas Rossberg <rossberg at google.com>wrote:
You'd have to unwrap the whole fat pointer, right?
Yes, of course. I simplified a bit.
On Thu, Apr 3, 2014 at 3:00 AM, Dmitry Lomov <dslomov at chromium.org> wrote:
On Thu, Apr 3, 2014 at 8:26 AM, Andreas Rossberg <rossberg at google.com> wrote:
A uvalue is deeply uvalue only when its prototype and all the methods found there are uvalues. Although this is the case for E, the implications for JS are, ahem, tricky. If they're not deep in this way, then they can't be passed transparently between vats, address spaces, and machines.
How can a prototype possibly be a fully structural value in JavaScript? Especially, since the language has this great feature of making all functions objects with identity?
On the other hand, I also don't see how uvalues can work without a prototype link. So it seems that it's simply impossible to make them transparently transferable between vats in the way you desire (and I would, too). It seems that JavaScript is actively hostile to that idea.
Correct, yes - I think Mark says the same thing when he says "the implications are tricky". "Passing transparently between vats, address spaces, and machines" must be a non-goal for value objects as proposed. That's also how I read "uvalues have a prototype ... when transmitted between realms, uvalues retain this link" - among other things, it means that uvalues from different realms are never equal (since their prototype links are different by virtue of belonging to different realms).
You can never compare uvalues from different realms: both uvalues need to be in the same realm before you can do the comparison. All you need is that equal uvalues in realm A become equal uvalues in realm B when passed over. This is exactly how existing primitives work, right?
On 3 April 2014 14:57, C. Scott Ananian <ecmascript at cscott.net> wrote:
You can never compare uvalues from different realms: both uvalues need to be in the same realm before you can do the comparison. All you need is that equal uvalues in realm A become equal uvalues in realm B when passed over. This is exactly how existing primitives work, right?
Unfortunately, that only works for primitives because the respective constructor/wrapper class is known to exist in all realms, it is known to be "the same" everywhere, and because the language can hence apply some magic to "rebind" it when primitives cross realm boundaries. For user-defined values, no such guarantees exist. Hence, no similar rebinding magic can be applied, and values have to hold on to their original constructor.
Le 3 avr. 2014 à 07:33, Andreas Rossberg <rossberg at google.com> a écrit :
Not sure what type 'object' is used for in the wild.
I sometimes use it for distinguishing between "Object-like" objects and primitive values (usually together with an additional test for excluding null).
For that purpose, typeof applied to uvalues should not produce "object".
On Thu, Apr 3, 2014 at 3:09 PM, Andreas Rossberg <rossberg at google.com>wrote:
Unfortunately, that only works for primitives because the respective constructor/wrapper class is known to exist in all realms, it is known to be "the same" everywhere, and because the language can hence apply some magic to "rebind" it when primitives cross realm boundaries. For user-defined values, no such guarantees exist. Hence, no similar rebinding magic can be applied, and values have to hold on to their original constructor.
Wouldn't it be a security risk to send the constructor and prototype between realms? Since the uvalue only consists of primitives and other uvalues, it can be serialized to an object consisting of objects and primitives. This object could then be passed into the constructor of an equivalent uvalue, just like the original uvalue was created. This would unfortunately add an extra step in passing uvalues between realms, but it might just be a minor inconvenience.
Marius Gundersen
On Thu, Apr 3, 2014 at 9:09 AM, Andreas Rossberg <rossberg at google.com> wrote:
On 3 April 2014 14:57, C. Scott Ananian <ecmascript at cscott.net> wrote: Unfortunately, that only works for primitives because the respective constructor/wrapper class is known to exist in all realms, it is known to be "the same" everywhere, and because the language can hence apply some magic to "rebind" it when primitives cross realm boundaries. For user-defined values, no such guarantees exist. Hence, no similar rebinding magic can be applied, and values have to hold on to their original constructor.
I'm waving my hands here, but it seems to me that a two-part solution could work:
- by default, objects passed between realms retain their original "uvalue type id" (realm id + id of uvalue in that realm") but become new anonymous uvalue types with no prototype (although field names and types are preserved.) Equality between types in different realms work, because they share the same "uvalue type id", and if the value is passed back into the original realm the prototype can be re-linked.
This lets uvalues become new "opaque types" when passed between realms.
- a "cross-realm registry" that allows user code to opt-in to associate a given local prototype with a certain "uvalue type id". For cooperating codebases, this lets "Colors" be manipulated as "Colors" in both realms. Since the fundamental nature of the uvalue is structural equality on the contents + "uvalue type id", reassociating a prototype with a "uvalue type id" is just a convenience.
2a) One alternative here is to make a general feature out of the map
from "uvalue type id" to prototype. Given a uvalue, setting
uvalue.__proto__ will change the prototype used for all uvalues of
that type. This is rather frightening in its power. Obviously,
1.__proto__ should be made read-only! But on the other hand, it
surfaces the conceptual framework already underlying method dispatch
on primitives.
2b) Another alternative is to define a special "transferable uvalue"
type, which (unlike other uvalues) inherits from [Transferable].
Transferable uvalues register a string or symbol as their "handle"
when the type is created. This is used to link up uvalues between
realms. As described in (1), uvalues with a Symbol as their handle
are transferable, but are treated as opaque uvalues in other realms.
There is no rebinding of the prototype after the type is created.
(Presumably the 'handle' is an optional parameter to the uvalue type
constructor?)
2c) Like 2b, but all uvalues are transferable, and are created by
default with a new Symbol as their handle. You can transfer the
Symbol across realms if you want/need to link up uvalues between
realms (or use the option to specify a String handle to trade-off
namespace security for convenience).
--scott
On 3 April 2014 15:30, Marius Gundersen <gundersen at gmail.com> wrote:
On Thu, Apr 3, 2014 at 3:09 PM, Andreas Rossberg <rossberg at google.com> wrote:
Unfortunately, that only works for primitives because the respective constructor/wrapper class is known to exist in all realms, it is known to be "the same" everywhere, and because the language can hence apply some magic to "rebind" it when primitives cross realm boundaries. For user-defined values, no such guarantees exist. Hence, no similar rebinding magic can be applied, and values have to hold on to their original constructor.
Wouldn't it be a security risk to send the constructor and prototype between realms?
Objects could always cross realm boundaries, that is nothing new. In the browser, the usual access checks apply.
Since the uvalue only consists of primitives and other uvalues,
They also (necessarily) carry an implicit reference to their type descriptor (an internal object), which in turn links to the prototype (or equivalently, the other way round). Otherwise, methods, wrapping, etc would not work.
On 3 April 2014 15:39, C. Scott Ananian <ecmascript at cscott.net> wrote:
On Thu, Apr 3, 2014 at 9:09 AM, Andreas Rossberg <rossberg at google.com> wrote:
On 3 April 2014 14:57, C. Scott Ananian <ecmascript at cscott.net> wrote: Unfortunately, that only works for primitives because the respective constructor/wrapper class is known to exist in all realms, it is known to be "the same" everywhere, and because the language can hence apply some magic to "rebind" it when primitives cross realm boundaries. For user-defined values, no such guarantees exist. Hence, no similar rebinding magic can be applied, and values have to hold on to their original constructor.
I'm waving my hands here, but it seems to me that a two-part solution could work:
[...]
Without going into details: Yes, it should be possible to build some form of registry mechanism (similarly as for symbols) that supports mapping and round trips of uvalues. However, that could be rather involved (maybe needing membranes?), and hence would best be an abstraction on top. I see no particular reason to build it into the core value type semantics -- we don't do anything like that for objects either.
Given that Color is a struct type object, is there a more direct way of getting the common prototype of all the array type objects with different dimensions with this element type, than by calling getPrototypeOf(Color.arrayType.prototype)?
On 04/02/2014 07:32 AM, Niko Matsakis wrote:
I just wanted to let people on es-discuss know about two of my recent blog posts concerning typed objects. The first is a kind of status report:
smallcultfollowing.com/babysteps/blog/2014/04/01/typed-objects-status-report
and the second details some (preliminary) thoughts on how one could build on typed objects to support user-defined value types:
smallcultfollowing.com/babysteps/blog/2014/04/01/value-types-in-javascript
We've been tossing things like this around for a while. I'd personally love to have some notion of value types along with int64 and friends (after having the proposals out for 15 years). Note that these generally got bogged down on one of four issues:
-
Handling of NaN's and ±0
-
Direction of binary operator coercion (if x is an int64, is x+1 an int64 or a regular double Number? What about x+0.5?)
-
Built-in int64 vs. providing a library that allows folks to implement int64 (whichever one was proposed, folks wanted the other one)
-
Cross-realm behavior
Waldemar
Note that "value object" proposal does not address int64.
Note that "value object" proposal does not address int64.
FWIW, I'm too interested in solving the int64 problem somehow. The last I heard about int64 was here: www.slideshare.net/BrendanEich/value-objects2. How does the int64 story move forward?
int64 and uint64 are integral parts of value objects, first proofs of concept even. Sorry I fell behind on the SpiderMonkey bug, I'm hoping others will take it over (cc'ed; and for V8) and revive it. It has obvious use-cases in Node.js and anything that tiles the OS syscall surface.
I just wanted to let people on es-discuss know about two of my recent blog posts concerning typed objects. The first is a kind of status report:
smallcultfollowing.com/babysteps/blog/2014/04/01/typed-objects-status-report
and the second details some (preliminary) thoughts on how one could build on typed objects to support user-defined value types:
smallcultfollowing.com/babysteps/blog/2014/04/01/value-types-in-javascript
Niko