String identity template tag
On Mon, Dec 10, 2018 at 7:08 PM Isiah Meadows <isiahmeadows at gmail.com> wrote:
It'd be way easier to construct simple template tags if there was a built-in identity tag
Wholeheartedly agree, a couple of months ago I considered posting something very similar, both for utility reasons and in hopes that it would be an optimization target (being a standard operation).
I find the name identity unilluminating, though, partially because it's
not quite the same meaning as the usual "identity" function (function identity(x) { return x; }), though it's close. assemble?
-- T.J. Crowder
I'm not married to identity, and I agree the name is probably not
ideal. I'm more concerned about functionality, though.
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
Why not String.tag or .tagged?
While we're at it, is there any good reason not to have something like this:
String.template = (template : String, taggerFn=String.identity/tag/tagged :
Function) => (keys : Array | Object) => taggerFn(template, (keys is Array)
? ...keys : keys)
// apologies for pseudo-semi-functional code
// having keys be an object allows template to be filled by key name rather
than just index
This would make templates closer to the traditional usage, where the template comes first and is later passed values to be filled in with. Having the taggerFn as an argument allows for things like Isiah's escape-then-apply tagging examples.
Those names a little too generic for my liking here. What about
String.expand(template, ...params)?
And also, let's not try to bake a traditional template engine into the JS spec - syntactic template strings already work well enough.
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
On Wed, Dec 12, 2018 at 6:59 AM Isiah Meadows <isiahmeadows at gmail.com> wrote:
Those names a little too generic for my liking here. What about
String.expand(template, ...params)?
I like it.
And also, let's not try to bake a traditional template engine into the JS spec - syntactic template strings already work well enough.
Agreed.
-- T.J. Crowder
Yeah, String.expand is the nicest one I've seen so far.
What is the intuition behind "expand"? What is being expanded, and what is it expanding into?
The template is being expanded as if the template itself is untagged. The point of this is a template tag that just does the default untagged behavior of coercing all expressions to strings and joining the whole thing together.
On Wed, Dec 12, 2018 at 5:24 PM Isiah Meadows <isiahmeadows at gmail.com>
wrote:
The template is being expanded as if the template itself is untagged.
Does this mean that you describe what tagged templates do, or what untagged templates do, as "expanding" something? If so, what is the intuition behind that prior usage of "expand"?
The point of this is a template tag that just does the default untagged behavior of coercing all expressions to strings and joining the whole thing together.
I certainly agree that the name should suggest equivalence to the default untagged behavior. I just never would have thought to describe that behavior as "expanding" something. What is it expanded into?
I mean equivalence to untagged behavior, if that helps.
FWIW, as stated previously, I'm not married to the name.
I agree with Mark, and I wonder why String.tag is not the obvious choice
here, since every interpolation is also coerced as String
Random suggestions:
String.cooked, which pairs well with already existingString.rawString.vanillaString.plainnull, i.e., using a null (or undefined) value as tag before a template literal is equivalent to using no tag. (Con: not polyfillable)
To me, String.tag seems more descriptive of the syntactic location
(the template tag) than the semantics it carries.
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
I like String.cooked, especially considering String.raw already
basically does this, just using template.raw instead of template.
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
In general, I think method names should be verbs in the imperative
tense (okay, mood if you like linguistic distinctions), which would
argue for cook rather than cooked. (String.raw is an unfortunate
exception to this rule, which has largely been used throughout the
standard library. Another exception is Reflect.ownKeys. There are
probably others as well, but they are exceptions, not the norm.)
I like cook. It's assemble, but with more flavor.
The good news here, though, is we're all talking about a name, which suggests that in general people like the taste of the idea. There don't seem to be concerns that it's half-baked.
(I'll stop now.)
-- T.J. Crowder
FWIW, I've used same logic for something like a no-op for i18nstrings [1]
so, considering it has easily use cases with mapped interpolations too, I
think it's more than natural to have that in core.
It's also backward compatible/polyfillable so this is kinda a no-brainer, name a part, of course.
why not copy python's list-zip static-function ( www.programiz.com/python-programming/methods/built-in/zip)?
i'm also against the spread-operator signature.
// python-inspired list-zip static-function
// List.zip(list1, list2)
str = List.zip(
templateList,
argList
// slice-out argList-padded undefined
).slice(0, -1).join("\n");
// List.zip only zips up to length of list1
// padding with undefined
List.zip([1, 2, 3], ["one", "two"]);
// [1, "one", 2, "two", 3, undefined]
List.zip([1, 2], ["one", "two", "three"]);
// [1, "one", 2, "two"]
On Thu, Dec 13, 2018 at 1:03 PM kai zhu <kaizhu256 at gmail.com> wrote:
why not copy python's list-zip static-function
The result isn't a string, it's an array you'd then have to join with
.join(""). Not that zip isn't a useful function too... (At least,
presumably it is, it shows up in libs a lot. I don't recall having had to
do it outside a tag function.)
i'm also against the spread-operator signature.
Good point to raise. (FWIW: It's "rest," not "spread;" and it's not an operator.)
An argument in favor of using a rest parameter is it aligns with
String.raw. (It also makes cook a valid tag function, though a
pointless one to use in that way as the result is what you'd get from an
untagged template.)
An argument against using a rest parameter (taking an array instead) is
that, to my mind anyway, the primary use case for this function is as a
tool within other general-purpose tag functions (like Isiah's debug and
trust examples). In a general-purpose tag function, since you don't know
how many substitution values you're going to get, you're likely to have
used a rest parameter, meaning you already have an array. Passing the array
directly is more efficient, surely, than spreading it and having cook
gather it up into a rest parameter. (That said, if engines don't already
aggressively optimize calls using spread with an array that has the default
iterator to functions using a perfectly-matching rest parameter list,
presumably they will at some point, or investigations have proved it's not
worth the trouble.)
I'm not bothered either way.
-- T.J. Crowder
I like String.cooked best. While I agree that method names should generally be verbs, I suggest this rule should not be used for template literal tag names. Rather, the tag name should generally be descriptive, naming the language being parsed or the way it is interpreted or what the value of the template literal expression will be. Most often, it should name the language being parsed. By contrast with "raw", "cooked" is the right name for this language --- the language of escaped characters within a normal string literal.
Historical note: Template literals derive from E's quasi-literals www.erights.org/elang/grammar/quasi-overview.html . Template literal tags are E's quasi-parsers. We usually named quasi-parsers according to the language they were quasi-parsing. This is natural for most JS uses. See erights/quasiParserGenerator
On Thu, Dec 13, 2018 at 5:56 PM Mark Miller <erights at gmail.com> wrote:
I like String.cooked best. While I agree that method names should generally be verbs, I suggest this rule should not be used for template literal tag names.
Fair enough, but the primary use of this function is not using it as a tag function, but rather as a normal function. It doesn't make sense to use it as a tag function, the result is the same as an untagged template. The only use case I could see for using it as a tag function is if you were selecting from several tag functions at runtime and needed a "no-op" option.
But called normally, it's a useful helper, for instance in Isiah's escape
(where I assume the literal strings are already trusted, the substitutions
are not):
const escapeHTML = val => String(val).replace(/&/g, "&").replace(/</g,
"<");
const escape = (strings, ...subs) => {
return String.cook(strings, ...subs.map(escapeHTML));
};
const foo = "<script>alert('maliciousness!');<\/script>";
console.log(escape`<p>${foo}</p>`);
// => <p>&lt;script>alert('maliciousness!');&lt;/script></p>
I'm not that bothered either way, but I'd say it's a utility for tag functions to use, not a tag function itself.
-- T.J. Crowder
On Thu, Dec 13, 2018 at 6:37 PM T.J. Crowder <tj.crowder at farsightsoftware.com> wrote:
But called normally, it's a useful helper, for instance in Isiah's
escape...
Going through the process of the example for my message just now made me think more about this function. Suppose it:
-
Accepted an array of substitutions rather than a rest parameter, and
-
Accepted an optional mapping function
Then, what I wrote on my last message as:
const escape = (strings, ...subs) => {
return String.cook(strings, ...subs.map(escapeHTML));
};
would be
const escape = (strings, ...subs) => String.cook(strings, subs, escapeHTML);
...while still supporting the earlier usage (just without spread) if desired.
-- T.J. Crowder
On Dec 13, 2018, at 10:37 AM, T.J. Crowder <tj.crowder at farsightsoftware.com> wrote:
Fair enough, but the primary use of this function is not using it as a tag function, but rather as a normal function. It doesn't make sense to use it as a tag function, the result is the same as an untagged template. The only use case I could see for using it as a tag function is if you were selecting from several tag functions at runtime and needed a "no-op" option.
But called normally, it's a useful helper, for instance in Isiah's
escape(where I assume the literal strings are already trusted, the substitutions are not):
+1
The primary use case is not as a tag function but as a helper and it should be named accordingly.
My suggestion: String.interpolate
I think this is the right question. I agree that String.cook or whatever it is called with typically be called explicitly rather than used syntactically as a tag. However, putting the optional mapping function aside for a moment, if the job it is doing is equivalent to that done by a tag function, and if there are similar existing tags that can be called as a function to do a similar job, I think it would be better for them to have a similar signature and be used in an API compatible way.
In this case, if we choose a name like "cook" or "cooked" in order to make the analogy with String.raw, then it should have the same API surface as String.raw. Otherwise, there's too much to remember.
As for the optional mapping function, there's nothing about that which is more relevant to cooked base strings than to raw base strings. We should be able to apply mapping functions to either, as well as to other base tags, in a similar way. This suggests tag combinators:
const mapTag = (baseTag, mapFn) => (template, ...aubs) => baseTag(template,
...subs.map(mapFn));
mapTag(String.cooked, escapeHTML)`...`
As a completely separate point, this way of escaping html is not context sensitive, and likely horribly unsafe. Much of the motivation for template literals in the first place is to support context sensitive escaping, where the escaping of the x data in
safeHTML`....${x}....`
depends on where in the html parsing of the literal parts it is encountered. See the work of Mike Samuel (cc'ed).
On Thu, Dec 13, 2018 at 7:00 PM Mark Miller <erights at gmail.com> wrote:
As for the optional mapping function, there's nothing about that which is more relevant to cooked base strings than to raw base strings. We should be able to apply mapping functions to either, as well as to other base tags, in a similar way. This suggests tag combinators...
Probably as a separate thing? Since cook/cooked/interpolate isn't
primarily a tag function.
As a completely separate point, this way of escaping html is not context sensitive, and likely horribly unsafe.
It was just a throwaway for the example, for the specific context of using the string within the body of a tag (not in attribute text, etc.). Correct me if I'm wrong, but for that context, I believe it's sufficient.
In this case, if we choose a name like "cook" or "cooked" in order to make the analogy with String.raw, then it should have the same API surface as String.raw.
Sadly, yes. I say "sadly" because I quite liked cook (or cooked), but
only for frivolous reasons. :-)
+1 for Allen's String.interpolate. Something along these lines, accepting
array-likes rather than iterables (though that's a discussion to have):
const toLength = n => {
n = Math.min(+n || 0, Number.MAX_SAFE_INTEGER);
return n < 0 ? Math.ceil(n) : Math.floor(n);
};
Object.defineProperty(String, "interpolate", {
value(strings, subs = [], mapFn) {
const strslen = toLength(strings.length);
if (strslen <= 0) {
return "";
}
const subslen = toLength(subs.length);
let s = "";
let index = 0;
while (true) {
const subIndex = index;
s += strings[index++];
if (index == strslen) {
return s;
}
if (subIndex < subslen) {
const sub = subs[subIndex];
s += mapFn ? mapFn(sub) : sub;
}
}
},
configurable: true,
writable: true
});
Example: jsfiddle.net/tk9vyrw3
Like String.raw, that ignores entries in subs at or after
strings.length - 1. (It's basically String.raw's algorithm, and like
raw's algorithm, it's un-optimized for clarity.) But that's also a
discussion to have, perhaps it should tack them on the end. It won't come
up in the simple use case of passing along the strings and subs in a tag
function (subs will always have one fewer entry than strings), but it's
slightly more general-purpose if it tacks them on the end.
String.interpolate would also fit nicely if at some stage there were an
Array.interpolate (e.g., "zip").
-- T.J. Crowder
I'll admit that HTML escaping tag was probably a bad example. It was just for show, nothing more, and obviously I wouldn't recommend it for production.
I'll point out Kai could be on to something, although I disagree zip
would be the right abstraction. Maybe Array.interleave(...arrays)? You
could do Array.interleave(template, args).map(String).join("") for
similar effect, and it'd be more generally useful.
The key here is that iteration would stop after the index hits any array's length, so it'd be polyfilled kinda like this:
Array.interpolate = (...args) => {
let ret = []
let lengths = []
let count = 0
for (let i = 0; i < args.length; i++) {
lengths[i] = args[i].count
}
for (let index = 0; ; index++) {
for (let i = 0; i < args.length; i++) {
if (index === lengths[i]) return ret
ret[count++] = args[i][index]
}
}
}
(This could be optimized, though.)
On Fri, Dec 14, 2018 at 12:51 PM Isiah Meadows <isiahmeadows at gmail.com>
wrote:
I'll point out Kai could be on to something, although I disagree
zipwould be the right abstraction. MaybeArray.interleave(...arrays)? You could doArray.interleave(template, args).map(String).join("")for similar effect, and it'd be more generally useful.The key here is that iteration would stop after the index hits any array's length, so it'd be polyfilled kinda like this:
Array.interpolate = (...args) => { let ret = [] let lengths = [] let count = 0 for (let i = 0; i < args.length; i++) { lengths[i] = args[i].count } for (let index = 0; ; index++) { for (let i = 0; i < args.length; i++) { if (index === lengths[i]) return ret ret[count++] = args[i][index] } } }(This could be optimized, though.)
As a data point, something like this loop appears in most of the template tags I've written but it's never had these precise semantics so I didn't bother putting it into template-tag-common www.npmjs.com/package/template-tag-common.
That library makes it easy to split the operation of a template tag into 3 stages:
- An optional configuration stage accessed by calling the template tag as
a regular function: mytag({ /* options */)
... - Static analysis over the strings. This is memoized.
- Computing a result from (options, strings, results of step 2, interoplated values)
The final loop (step 3) in the template tags I maintain tends to looks like
function computeResult(options, staticState /* from step 2 */, strings, ...values) { let n = values.length; // Could do Math.max(strings.length - 1, values.length); let result = strings[0]; // Usually strings.raw for (let i = 0; i < n;) { const interpolatedValue = f(options, staticState[i], values[i]); // Sometimes code here looks backwards at the result to see if it needs to avoid token-merging hazards. result += interpolatedValue; result += strings[++i]; } return wrapResult(result); // Produce a value of a type that encapsulates the tag's security guarantees. }
The main difference with that loop is that it's generalized to any number
of arrays, not just two with the second array having length one less than
the first. Otherwise, it'd look exactly the same. BTW, I like this route
(Array.interleave) better since it doesn't have to result in just a
single string result - it could just be an array of strings plugged into
some API instead, or it could be procedurally streamed out in chunks.
On Fri, Dec 14, 2018 at 2:26 PM Isiah Meadows <isiahmeadows at gmail.com>
wrote:
The main difference with that loop is that it's generalized to any number of arrays, not just two with the second array having length one less than the first. Otherwise, it'd look exactly the same. BTW, I like this route (
Array.interleave) better since it doesn't have to result in just a single string result - it could just be an array of strings plugged into some API instead, or it could be procedurally streamed out in chunks.
Fair enough. If you're not looking for something template tag specific then a simple zip over iterators should do it?
function *ziperator(iterators) { let progressed; do { progressed = false; for (let iterator of iterators) { for (let element of iterator) { yield element; progressed = true; break; } } } while (progressed); }
console.log(Array.from(ziperator([ ['a', 'b', 'c']Symbol.iterator, [1, 2]Symbol.iterator ])).join('')); // -> a1b2c
(but optimized :)
I could go with an iterator equivalent, but I'd like to defer that to the seemingly-planned "iterlib" thing that's been considered since before ES2015 was released. Something that works with arrays is good enough for now.
BTW, your ziperator isn't really the same as my Array.interpolate
(which is better named Array.interleave). It needs to be this:
function *ziperator(...iters) {
for (let i = 0; i < iters.length; i++) {
iters[i] = iters[i][Symbol.iterator]()
}
while (true) {
for (let i = 0; i < iters.length; i++) {
const {done, value} = iters[i].next()
if (done) return undefined
yield value
}
}
}
The optimized version is pretty straightforward (using private fields
- methods here):
function ziperator(...iters) { return new InterleavedIterator(iters) }
class InterleavedIterator {
#iters, #index
constructor(iters) { this.#iters = iters; this.#index = 0 }
[Symbol.iterator]() { return this }
next(value) { return this.#invoke("next", value) }
throw(value) { return this.#invoke("throw", value) }
return(value) { return this.#invoke("return", value) }
#invoke(method, value) {
if (this.#iters == null) return {done: true, value: undefined}
const index = this.#index
this.#index = (index + 1) % this.#iters.length
const {done, value} = this.#iters[index][method](value)
if (done) this.#iters = undefined
return {done, value}
}
}
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
Fair enough.
I could go with an iterator equivalent, but I'd like to defer that to the seemingly-planned "iterlib" thing that's been considered since before ES2015 was released.
i'm against iterator-based design-patterns and libraries, as they lack a clear fit in most javascript solutions. the vast majority of looping in UX-workflows (javascript’s primary problem-domain) are over [serializable] JSON lists/dicts/strings, where Array/Object/String looping-methods will suffice.
all other use-cases are uncommon/complicated enough that custom for/while loops are usually better suited than custom-iterators.
Could you bring that up in a different thread instead of driving this off-topic? Also, please don't forget that a very significant chunk of JS doesn't even run in a GUI environment (consider: servers, IoT, satellites, etc.).
Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
everything using javascript is ultimately meant to solve a UX-workflow problem (including js-code in servers, IoT, satellites, etc.). if you're so caught up in low-level js library-code that you can't see how the piece fits/integrates into that big-picture, then you should go back to general-purpose programming in java/c++/c#/etc.
but of course, much of industry these days prefer hiring business-oriented programmers focused on solving UX-workflow problems rather than general-purpose programming ones.
Kai, that there may be some tenuous connection to ux-workflow does not support your sweeping claims. Also please stop with the personal attacks.
It'd be way easier to construct simple template tags if there was a built-in identity tag, like this:
String.identity = (template, ...args) => { let str = "" for (let i = 0; i < args.length; i++) { str += template[i] + String(args[i]) } return str + template[template.length - 1] }This would also provide some language consistency in that tag-less templates are evaluated as if they were tagged with the internal
String.identity.The usefulness of this is for simple utility template tags, like:
debug`Value: ${value}`, wherevalueis auto-inspected.trust`html`, which returns a raw HTML virtual DOM node.escape`trusted ${untrusted}`, which escapes template variablesHere's how
debugandtrustabove would be implemented:// `debug` - logs a message with inspected values to the console const debug = (template, ...args) => String.identity(template, ...args.map(arg => util.inspect(arg))) // `trust` - returns a Mithril `m.trust` vnode // https://mithril.js.org/trust.html const trust = (...args) => m.trust(String.identity(...args)) // `escape` - logs a message with inspected values to the console const escape = (template, ...args) => String.identity(template, ...args.map(escapeString))Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com
It'd be *way* easier to construct simple template tags if there was a built-in identity tag, like this: ```js String.identity = (template, ...args) => { let str = "" for (let i = 0; i < args.length; i++) { str += template[i] + String(args[i]) } return str + template[template.length - 1] } ``` This would also provide some language consistency in that tag-less templates are evaluated as if they were tagged with the internal `String.identity`. The usefulness of this is for simple utility template tags, like: - `` debug`Value: ${value}` ``, where `value` is auto-inspected. - `` trust`html` ``, which returns a raw HTML virtual DOM node. - `` escape`trusted ${untrusted}` ``, which escapes template variables Here's how `debug` and `trust` above would be implemented: ```js // `debug` - logs a message with inspected values to the console const debug = (template, ...args) => String.identity(template, ...args.map(arg => util.inspect(arg))) // `trust` - returns a Mithril `m.trust` vnode // https://mithril.js.org/trust.html const trust = (...args) => m.trust(String.identity(...args)) // `escape` - logs a message with inspected values to the console const escape = (template, ...args) => String.identity(template, ...args.map(escapeString)) ``` ----- Isiah Meadows contact at isiahmeadows.com www.isiahmeadows.com