Specifying template strings
On Wed, Jul 9, 2014 at 11:25 AM, Axel Rauschmayer <axel at rauschma.de> wrote:
I find the specification of template strings still a bit difficult to understand:
– The abbreviations TV and CV are used 12.2.9, but defined in 11.8.6.1.
Did you mean TV and TRV?
This is no different than:
- String SV and CV
- Number MV
Which are all defined in Chapter 11
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
– It’d be nice if untagged template strings and tagged templates could be mentioned closer together. They are basically the same thing and this structure seems to be dictated by grammar. To me, template strings are more like tagged templates without a tag. Would it make sense to specify them that way?
(no response, I'll let Allen address these)
On Jul 9, 2014, at 12:41 PM, Rick Waldron wrote:
On Wed, Jul 9, 2014 at 11:25 AM, Axel Rauschmayer <axel at rauschma.de> wrote: I find the specification of template strings still a bit difficult to understand:
– The abbreviations TV and CV are used 12.2.9, but defined in 11.8.6.1.
Did you mean TV and TRV?
This is no different than:
- String SV and CV
- Number MV
Which are all defined in Chapter 11
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
Because a Tagged Template is a call. Would it be clearer if there was a note that highlighted the fact that the actual TemplateLiteral provides the argument list for the call?
– It’d be nice if untagged template strings and tagged templates could be mentioned closer together. They are basically the same thing and this structure seems to be dictated by grammar. To me, template strings are more like tagged templates without a tag. Would it make sense to specify them that way?
They actually aren't the same thing at all. A tagged template literal is a kind of call. A untagged template literal is essentially a string interpolation operation. Different precedence, different semantics.
(no response, I'll let Allen address these)
The spec. isn't a feature level tutorial. Instead, it is organized around the syntactic and semantic layers of the language and each major feature typically impacts various different parts of the spec.
New lexical (token) level syntax and related semantics go into clause 11. Template Literals (without the tag) are a kind of PrimaryExpression so they belong in 12.2. A tagged template is a kind of MemberExpression so it belongs in 12.3.
On Wed, Jul 9, 2014 at 5:02 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
Because a Tagged Template is a call. Would it be clearer if there was a note that highlighted the fact that the actual TemplateLiteral provides the argument list for the call?
This is one of many places where the spec would be greatly improved by a line or two of explanatory text. Here is an attempt:
12.3.7
A tagged template is an expression such as xthe secret words are ${a} ${b}. It looks like a string, but it is a kind of function call. It
is evaluated like x(callSite, a, b) where callSite is a call site
object (see 12.2.9.2.2), an Array of strings that supplies the textual
parts of the template.
12.2.9.2.2
Call site objects are used in the evaluation of tagged templates (12.3.7). They contain the string parts of a template. Since this information is always the same for any given template, a single call site object is created for each tagged template in a program, it is frozen to prevent modifications, and it is reused each time the tagged template is evaluated.
There are so many algorithms with no explanatory text at all that it seems like it must be a deliberate style choice. Is it?
Speaking as a user of the spec, a little prose can be a big help. In regions of the spec that don't have any, I often feel like I'm reading uncommented assembly code, reverse-engineering all higher-level structure.
On Jul 9, 2014, at 21:41 , Rick Waldron <waldron.rick at gmail.com> wrote:
On Wed, Jul 9, 2014 at 11:25 AM, Axel Rauschmayer <axel at rauschma.de> wrote: I find the specification of template strings still a bit difficult to understand:
– The abbreviations TV and CV are used 12.2.9, but defined in 11.8.6.1.
Did you mean TV and TRV?
This is no different than:
- String SV and CV
- Number MV
Which are all defined in Chapter 11
Yes, I meant TV and TRV. I’d prefer these names to be longer. The short names are not very self-explanatory and finding their definitions is difficult, too (different chapter!).
Axel
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
Because a Tagged Template is a call. Would it be clearer if there was a note that highlighted the fact that the actual TemplateLiteral provides the argument list for the call?
Yes. Naively, I’d expect the result of a TemplateLiteral to be a string. But I previously didn’t understand the difference between static semantics and runtime semantics. The indirection makes things more difficult to follow, but I assume it helps with writing the specification (given proxies, generators etc.).
Axel
On Jul 10, 2014, at 7:10 AM, Jason Orendorff wrote:
On Wed, Jul 9, 2014 at 5:02 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
Because a Tagged Template is a call. Would it be clearer if there was a note that highlighted the fact that the actual TemplateLiteral provides the argument list for the call?
This is one of many places where the spec would be greatly improved by a line or two of explanatory text. Here is an attempt:
12.3.7
A tagged template is an expression such as x
the secret words are ${a} ${b}. It looks like a string, but it is a kind of function call. It is evaluated like x(callSite, a, b) where callSite is a call site object (see 12.2.9.2.2), an Array of strings that supplies the textual parts of the template.12.2.9.2.2
Call site objects are used in the evaluation of tagged templates (12.3.7). They contain the string parts of a template. Since this information is always the same for any given template, a single call site object is created for each tagged template in a program, it is frozen to prevent modifications, and it is reused each time the tagged template is evaluated.
Most of this is already in Note 2 of 12.2.9.22 with somewhat different wording. (but the section reference in Rev25 to tagged templates is wrong).
In general, I'm happy to receive bug reports suggesting the addition of informative material such as the above. It's one way that I discover where things need additional clarification. But also see my comments below.
There are so many algorithms with no explanatory text at all that it seems like it must be a deliberate style choice. Is it?
Yes. The experience with Ecma-262 (and other standards) is that redundant descriptive text (whether marked as normative or informative) is frequently misleading. And any redundancy introduces the possibility of internal specification inconsistency. Prior editions had more such descriptive text and there have been several situations where a buggy or incompatible implementation of a feature was traced back to somebody misinterpreting a vague descriptive statement rather than following the details of the an algorithm.
In general, we've tried to reduce the amount of such descriptive in the ES6 spec. Often this was done by taking summary descriptions that had formerly been formatted as normative text and converting them into informative notes. (This happened most frequently in the descriptions of some of the standard library methods). Sometimes developers moving fast read the prose and based upon that assume that they know what they need to implement without fully understanding the details of the algorithmic specification,.
Speaking as a user of the spec, a little prose can be a big help. In regions of the spec that don't have any, I often feel like I'm reading uncommented assembly code, reverse-engineering all higher-level structure.
I agree. But good informative notes needs to clarify the normative text rather than just provide redundant and simplified alternative descriptions
But, please file bugs on things like this, and tell me what you think needs such clarification.
On Jul 10, 2014, at 7:17 AM, Axel Rauschmayer wrote:
On Jul 9, 2014, at 21:41 , Rick Waldron <waldron.rick at gmail.com> wrote:
On Wed, Jul 9, 2014 at 11:25 AM, Axel Rauschmayer <axel at rauschma.de> wrote: I find the specification of template strings still a bit difficult to understand:
– The abbreviations TV and CV are used 12.2.9, but defined in 11.8.6.1.
Did you mean TV and TRV?
This is no different than:
- String SV and CV
- Number MV
Which are all defined in Chapter 11
Yes, I meant TV and TRV. I’d prefer these names to be longer. The short names are not very self-explanatory and finding their definitions is difficult, too (different chapter!).
TV and TRV for the template literal lexical components serve the similar purpose to that of SV and CV for string literals. They are intentionally defined using similar formulations and they belong in Chapter 11 because are part of the definition of the lexical mapping of source characters to token values.
I agree that the naming (this goes all the way back to SV and CV in the first edition) is not ideal and there is at least one bug on file suggesting an alternative restructuring of this part of the specification. However, it's a relatively low priority cleanup that may end up not getting done for this edition.
Yes, I meant TV and TRV. I’d prefer these names to be longer. The short names are not very self-explanatory and finding their definitions is difficult, too (different chapter!).
TV and TRV for the template literal lexical components serve the similar purpose to that of SV and CV for string literals. They are intentionally defined using similar formulations and they belong in Chapter 11 because are part of the definition of the lexical mapping of source characters to token values.
I agree that the naming (this goes all the way back to SV and CV in the first edition) is not ideal and there is at least one bug on file suggesting an alternative restructuring of this part of the specification. However, it's a relatively low priority cleanup that may end up not getting done for this edition.
Sure, this is definitely in the domain of fine-tuning. IMO, even making them a bit longer would already help.
On Jul 10, 2014, at 7:31 AM, Axel Rauschmayer wrote:
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
Because a Tagged Template is a call. Would it be clearer if there was a note that highlighted the fact that the actual TemplateLiteral provides the argument list for the call?
Yes. Naively, I’d expect the result of a TemplateLiteral to be a string. But I previously didn’t understand the difference between static semantics and runtime semantics. The indirection makes things more difficult to follow, but I assume it helps with writing the specification (given proxies, generators etc.).
Static semantics is about things about or which can be derived from just the source code, independent of the actual execution of the code Runtime semantics is about what actually happens when the program executes and typically has dependency upon the runtime state of the program.
Yes. Naively, I’d expect the result of a TemplateLiteral to be a string. But I previously didn’t understand the difference between static semantics and runtime semantics. The indirection makes things more difficult to follow, but I assume it helps with writing the specification (given proxies, generators etc.).
Static semantics is about things about or which can be derived from just the source code, independent of the actual execution of the code Runtime semantics is about what actually happens when the program executes and typically has dependency upon the runtime state of the program.
Yes. It’s quite clever how this attaches behavior to grammar rules (which initially confused me – I’m more used to runtime semantics referring directly to syntax).
It seems to me that part of the challenge of writing and reading the spec is that it has the complexity and structure of code, but without the support of code-focused tools such as IDEs. In an IDE, I’d simply click on an identifier to jump to its definition. Refactoring would also be simple(r).
On Jul 10, 2014, at 9:03 AM, Axel Rauschmayer wrote:
Yes. Naively, I’d expect the result of a TemplateLiteral to be a string. But I previously didn’t understand the difference between static semantics and runtime semantics. The indirection makes things more difficult to follow, but I assume it helps with writing the specification (given proxies, generators etc.).
Static semantics is about things about or which can be derived from just the source code, independent of the actual execution of the code Runtime semantics is about what actually happens when the program executes and typically has dependency upon the runtime state of the program.
Yes. It’s quite clever how this attaches behavior to grammar rules (which initially confused me – I’m more used to runtime semantics referring directly to syntax).
It seems to me that part of the challenge of writing and reading the spec is that it has the complexity and structure of code, but without the support of code-focused tools such as IDEs. In an IDE, I’d simply click on an identifier to jump to its definition. Refactoring would also be simple(r).
Yup. Ideally the spec. would be a hyper document with built-in browsings tools. And by that I'm think of something much more elaborate than even the hyperlinks that JSON puts into the HTML versions.
A general gripe of mine, even on the web and with html we are moistly emulating linear paper-oriented documents.
On Thu, Jul 10, 2014 at 10:37 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
Most of this is already in Note 2 of 12.2.9.22 with somewhat different wording. (but the section reference in Rev25 to tagged templates is wrong).
Oh, yes. I actually looked at that recently, because the person implementing template strings for SpiderMonkey (Lakshmi Narasimha Guptha Munuhur Rajagopal) was confused by the spec as written, and I had to clarify for him.
Step 1 did not strike me as totally clear. NOTE 2 is the only reason I didn't bring a question to this list. :)
This particular bit of spec is just specially weird. The language in step 1 is (necessarily) unique. It might be helpful to rewrite that step with even more explicit language, so that a note is less necessary to understanding. I guess the issue for me was that I had never thought of ES grammar productions as having identity before. Is there a place in the spec that explicitly says that each time code is parsed, a new tree of constructions is generated, distinct from other parses of the same code? If so, a link from here to there would help.
There are so many algorithms with no explanatory text at all that it seems like it must be a deliberate style choice. Is it?
Yes. The experience with Ecma-262 (and other standards) is that redundant descriptive text (whether marked as normative or informative) is frequently misleading. And any redundancy introduces the possibility of internal specification inconsistency.
OK. I respect these concerns.
This specific decision is really hard on implementers, who I think are the spec's target audience (?). Please consider mitigations for the observed problems other than defaulting to no explanation for anything.
Prior editions had more such descriptive text and there have been several situations where a buggy or incompatible implementation of a feature was traced back to somebody misinterpreting a vague descriptive statement rather than following the details of the an algorithm.
Thanks for clarifying this. I didn't know such things had ever happened.
Certainly there have often been implementation bugs due to underspecification. I'm very grateful for the increased precision in the spec under your editorship.
Speaking as a user of the spec, a little prose can be a big help. In regions of the spec that don't have any, I often feel like I'm reading uncommented assembly code, reverse-engineering all higher-level structure.
I agree. But good informative notes needs to clarify the normative text rather than just provide redundant and simplified alternative descriptions
I'll take whatever I can get.
On Jul 10, 2014, at 1:43 PM, Jason Orendorff wrote:
On Thu, Jul 10, 2014 at 10:37 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
Most of this is already in Note 2 of 12.2.9.22 with somewhat different wording. (but the section reference in Rev25 to tagged templates is wrong).
Oh, yes. I actually looked at that recently, because the person implementing template strings for SpiderMonkey (Lakshmi Narasimha Guptha Munuhur Rajagopal) was confused by the spec as written, and I had to clarify for him.
Step 1 did not strike me as totally clear. NOTE 2 is the only reason I didn't bring a question to this list. :)
Looking at it just now, Step 1 really only makes sense relative to understanding step 12. Putting in an explicit forward reference.
This particular bit of spec is just specially weird. The language in step 1 is (necessarily) unique. It might be helpful to rewrite that step with even more explicit language, so that a note is less necessary to understanding. I guess the issue for me was that I had never thought of ES grammar productions as having identity before. Is there a place in the spec that explicitly says that each time code is parsed, a new tree of constructions is generated, distinct from other parses of the same code? If so, a link from here to there would help.
that's why it says "source code corresponding to TemplateLiteral". It's the actual source code that we are identifying as a call site. Except for explicit cover grammars each source code unit is only parsed once. (except for eval, and we'll probably have to say something about module loaders in this regard...) but that is mostly implicit in the spec. Chapter 5 talks a bit about correspondences between source code and corresponding grammar productions. the spec. isn't very explicit about parse trees and/or ASTs. This all could be...don't know how much can be done for ES6.
BTW, eval brings up an interesting question:
let t = "taghello, ${world}.";
eval(t);
eval(t);
or new Function(t)(); new Function(t)();
Do each of these pairs compile into different call sites or are they the same call site? I'm inclined to say different. That each parse of an input string whither via eval, the function constructor, or module/script loading represent a logically distinct input into the ES environment.
There are so many algorithms with no explanatory text at all that it seems like it must be a deliberate style choice. Is it?
Yes. The experience with Ecma-262 (and other standards) is that redundant descriptive text (whether marked as normative or informative) is frequently misleading. And any redundancy introduces the possibility of internal specification inconsistency.
OK. I respect these concerns.
This specific decision is really hard on implementers, who I think are the spec's target audience (?). Please consider mitigations for the observed problems other than defaulting to no explanation for anything.
It would be really nice to have a guide to reading the ES6 spec. specifically target for implementors. I gave a talk something like that for ES5 when I was at Microsoft. But finding the time to write that is going to be hard (at least for me) until ES6 is done. Any volunteers out there? I'd be happy to work with somebody on this...
Bug reports are a good way to let me know that something is confusing and needs a note or something. Even better is proposing what it should say, like you did for this.
I find the specification of template strings still a bit difficult to understand:
– The abbreviations TV and CV are used 12.2.9, but defined in 11.8.6.1.
– Tagged templates are explained via EvaluateCall(tagRef, TemplateLiteral, tailCall). I think it would be easier to understand if it used GetTemplateCallSite.
– It’d be nice if untagged template strings and tagged templates could be mentioned closer together. They are basically the same thing and this structure seems to be dictated by grammar. To me, template strings are more like tagged templates without a tag. Would it make sense to specify them that way?
Axel