String identity template tag

# Isiah Meadows (6 months ago)

It'd be way easier to construct simple template tags if there was a built-in identity tag, like this:

String.identity = (template, ...args) => {
    let str = ""
    for (let i = 0; i < args.length; i++) {
        str += template[i] + String(args[i])
    }
    return str + template[template.length - 1]
}

This would also provide some language consistency in that tag-less templates are evaluated as if they were tagged with the internal String.identity.

The usefulness of this is for simple utility template tags, like:

  • debug`Value: ${value}`, where value is auto-inspected.
  • trust`html`, which returns a raw HTML virtual DOM node.
  • escape`trusted ${untrusted}`, which escapes template variables

Here's how debug and trust above would be implemented:

// `debug` - logs a message with inspected values to the console
const debug = (template, ...args) =>
    String.identity(template, ...args.map(arg => util.inspect(arg)))

// `trust` - returns a Mithril `m.trust` vnode
// https://mithril.js.org/trust.html
const trust = (...args) =>
    m.trust(String.identity(...args))

// `escape` - logs a message with inspected values to the console
const escape = (template, ...args) =>
    String.identity(template, ...args.map(escapeString))

Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# T.J. Crowder (6 months ago)

On Mon, Dec 10, 2018 at 7:08 PM Isiah Meadows <isiahmeadows at gmail.com> wrote:

It'd be way easier to construct simple template tags if there was a built-in identity tag

Wholeheartedly agree, a couple of months ago I considered posting something very similar, both for utility reasons and in hopes that it would be an optimization target (being a standard operation).

I find the name identity unilluminating, though, partially because it's not quite the same meaning as the usual "identity" function (function identity(x) { return x; }), though it's close. assemble?

-- T.J. Crowder

# Isiah Meadows (6 months ago)

I'm not married to identity, and I agree the name is probably not ideal. I'm more concerned about functionality, though.


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# Michael Luder-Rosefield (6 months ago)

Why not String.tag or .tagged?

While we're at it, is there any good reason not to have something like this:

String.template = (template : String, taggerFn=String.identity/tag/tagged :
Function) => (keys : Array | Object) => taggerFn(template, (keys is Array)
? ...keys : keys)
// apologies for pseudo-semi-functional code
// having keys be an object allows template to be filled by key name rather
than just index

This would make templates closer to the traditional usage, where the template comes first and is later passed values to be filled in with. Having the taggerFn as an argument allows for things like Isiah's escape-then-apply tagging examples.

# Isiah Meadows (6 months ago)

Those names a little too generic for my liking here. What about String.expand(template, ...params)?

And also, let's not try to bake a traditional template engine into the JS spec - syntactic template strings already work well enough.


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# T.J. Crowder (6 months ago)

On Wed, Dec 12, 2018 at 6:59 AM Isiah Meadows <isiahmeadows at gmail.com> wrote:

Those names a little too generic for my liking here. What about String.expand(template, ...params)?

I like it.

And also, let's not try to bake a traditional template engine into the JS spec - syntactic template strings already work well enough.

Agreed.

-- T.J. Crowder

# David Teller (6 months ago)

Yeah, String.expand is the nicest one I've seen so far.

# Mark Miller (6 months ago)

What is the intuition behind "expand"? What is being expanded, and what is it expanding into?

# Isiah Meadows (6 months ago)

The template is being expanded as if the template itself is untagged. The point of this is a template tag that just does the default untagged behavior of coercing all expressions to strings and joining the whole thing together.

# Mark Miller (6 months ago)

On Wed, Dec 12, 2018 at 5:24 PM Isiah Meadows <isiahmeadows at gmail.com>

wrote:

The template is being expanded as if the template itself is untagged.

Does this mean that you describe what tagged templates do, or what untagged templates do, as "expanding" something? If so, what is the intuition behind that prior usage of "expand"?

The point of this is a template tag that just does the default untagged behavior of coercing all expressions to strings and joining the whole thing together.

I certainly agree that the name should suggest equivalence to the default untagged behavior. I just never would have thought to describe that behavior as "expanding" something. What is it expanded into?

# Isiah Meadows (6 months ago)

I mean equivalence to untagged behavior, if that helps.

FWIW, as stated previously, I'm not married to the name.

# Andrea Giammarchi (6 months ago)

I agree with Mark, and I wonder why String.tag is not the obvious choice here, since every interpolation is also coerced as String

# Claude Pache (6 months ago)

Random suggestions:

  • String.cooked, which pairs well with already existing String.raw
  • String.vanilla
  • String.plain
  • null, i.e., using a null (or undefined) value as tag before a template literal is equivalent to using no tag. (Con: not polyfillable)
# Isiah Meadows (6 months ago)

To me, String.tag seems more descriptive of the syntactic location (the template tag) than the semantics it carries.


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# Isiah Meadows (6 months ago)

I like String.cooked, especially considering String.raw already basically does this, just using template.raw instead of template.


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# T.J. Crowder (6 months ago)

In general, I think method names should be verbs in the imperative tense (okay, mood if you like linguistic distinctions), which would argue for cook rather than cooked. (String.raw is an unfortunate exception to this rule, which has largely been used throughout the standard library. Another exception is Reflect.ownKeys. There are probably others as well, but they are exceptions, not the norm.)

I like cook. It's assemble, but with more flavor.

The good news here, though, is we're all talking about a name, which suggests that in general people like the taste of the idea. There don't seem to be concerns that it's half-baked.

(I'll stop now.)

-- T.J. Crowder

# Andrea Giammarchi (6 months ago)

FWIW, I've used same logic for something like a no-op for i18nstrings [1] so, considering it has easily use cases with mapped interpolations too, I think it's more than natural to have that in core.

It's also backward compatible/polyfillable so this is kinda a no-brainer, name a part, of course.

[1] WebReflection/i18n-dummy/blob/master/index.js

# kai zhu (6 months ago)

why not copy python's list-zip static-function ( www.programiz.com/python-programming/methods/built-in/zip)?

i'm also against the spread-operator signature.

// python-inspired list-zip static-function
// List.zip(list1, list2)
str = List.zip(
    templateList,
    argList
// slice-out argList-padded undefined
).slice(0, -1).join("\n");



// List.zip only zips up to length of list1
// padding with undefined

List.zip([1, 2, 3], ["one", "two"]);
// [1, "one", 2, "two", 3, undefined]

List.zip([1, 2], ["one", "two", "three"]);
// [1, "one", 2, "two"]
# T.J. Crowder (6 months ago)

On Thu, Dec 13, 2018 at 1:03 PM kai zhu <kaizhu256 at gmail.com> wrote:

why not copy python's list-zip static-function

The result isn't a string, it's an array you'd then have to join with .join(""). Not that zip isn't a useful function too... (At least, presumably it is, it shows up in libs a lot. I don't recall having had to do it outside a tag function.)

i'm also against the spread-operator signature.

Good point to raise. (FWIW: It's "rest," not "spread;" and it's not an operator.)

An argument in favor of using a rest parameter is it aligns with String.raw. (It also makes cook a valid tag function, though a pointless one to use in that way as the result is what you'd get from an untagged template.)

An argument against using a rest parameter (taking an array instead) is that, to my mind anyway, the primary use case for this function is as a tool within other general-purpose tag functions (like Isiah's debug and trust examples). In a general-purpose tag function, since you don't know how many substitution values you're going to get, you're likely to have used a rest parameter, meaning you already have an array. Passing the array directly is more efficient, surely, than spreading it and having cook gather it up into a rest parameter. (That said, if engines don't already aggressively optimize calls using spread with an array that has the default iterator to functions using a perfectly-matching rest parameter list, presumably they will at some point, or investigations have proved it's not worth the trouble.)

I'm not bothered either way.

-- T.J. Crowder

# Mark Miller (6 months ago)

I like String.cooked best. While I agree that method names should generally be verbs, I suggest this rule should not be used for template literal tag names. Rather, the tag name should generally be descriptive, naming the language being parsed or the way it is interpreted or what the value of the template literal expression will be. Most often, it should name the language being parsed. By contrast with "raw", "cooked" is the right name for this language --- the language of escaped characters within a normal string literal.

Historical note: Template literals derive from E's quasi-literals www.erights.org/elang/grammar/quasi-overview.html . Template literal tags are E's quasi-parsers. We usually named quasi-parsers according to the language they were quasi-parsing. This is natural for most JS uses. See erights/quasiParserGenerator

# T.J. Crowder (6 months ago)

On Thu, Dec 13, 2018 at 5:56 PM Mark Miller <erights at gmail.com> wrote:

I like String.cooked best. While I agree that method names should generally be verbs, I suggest this rule should not be used for template literal tag names.

Fair enough, but the primary use of this function is not using it as a tag function, but rather as a normal function. It doesn't make sense to use it as a tag function, the result is the same as an untagged template. The only use case I could see for using it as a tag function is if you were selecting from several tag functions at runtime and needed a "no-op" option.

But called normally, it's a useful helper, for instance in Isiah's escape (where I assume the literal strings are already trusted, the substitutions are not):

const escapeHTML = val => String(val).replace(/&/g, "&").replace(/</g,
"<");

const escape = (strings, ...subs) => {
    return String.cook(strings, ...subs.map(escapeHTML));
};

const foo = "<script>alert('maliciousness!');<\/script>";

console.log(escape`<p>${foo}</p>`);
// => <p>&amp;lt;script>alert('maliciousness!');&amp;lt;/script></p>

(jsfiddle.net/n6p7xcvm)

I'm not that bothered either way, but I'd say it's a utility for tag functions to use, not a tag function itself.

-- T.J. Crowder

# T.J. Crowder (6 months ago)

On Thu, Dec 13, 2018 at 6:37 PM T.J. Crowder <tj.crowder at farsightsoftware.com> wrote:

But called normally, it's a useful helper, for instance in Isiah's escape...

Going through the process of the example for my message just now made me think more about this function. Suppose it:

  1. Accepted an array of substitutions rather than a rest parameter, and

  2. Accepted an optional mapping function

Then, what I wrote on my last message as:

const escape = (strings, ...subs) => {
    return String.cook(strings, ...subs.map(escapeHTML));
};

would be

const escape = (strings, ...subs) => String.cook(strings, subs, escapeHTML);

(jsfiddle.net/n6p7xcvm/1)

...while still supporting the earlier usage (just without spread) if desired.

-- T.J. Crowder

# Allen Wirfs-Brock (6 months ago)

On Dec 13, 2018, at 10:37 AM, T.J. Crowder <tj.crowder at farsightsoftware.com> wrote:

Fair enough, but the primary use of this function is not using it as a tag function, but rather as a normal function. It doesn't make sense to use it as a tag function, the result is the same as an untagged template. The only use case I could see for using it as a tag function is if you were selecting from several tag functions at runtime and needed a "no-op" option.

But called normally, it's a useful helper, for instance in Isiah's escape (where I assume the literal strings are already trusted, the substitutions are not):

+1

The primary use case is not as a tag function but as a helper and it should be named accordingly.

My suggestion: String.interpolate

# Mark Miller (6 months ago)

I think this is the right question. I agree that String.cook or whatever it is called with typically be called explicitly rather than used syntactically as a tag. However, putting the optional mapping function aside for a moment, if the job it is doing is equivalent to that done by a tag function, and if there are similar existing tags that can be called as a function to do a similar job, I think it would be better for them to have a similar signature and be used in an API compatible way.

In this case, if we choose a name like "cook" or "cooked" in order to make the analogy with String.raw, then it should have the same API surface as String.raw. Otherwise, there's too much to remember.

As for the optional mapping function, there's nothing about that which is more relevant to cooked base strings than to raw base strings. We should be able to apply mapping functions to either, as well as to other base tags, in a similar way. This suggests tag combinators:

const mapTag = (baseTag, mapFn) => (template, ...aubs) => baseTag(template,
...subs.map(mapFn));

mapTag(String.cooked, escapeHTML)`...`

As a completely separate point, this way of escaping html is not context sensitive, and likely horribly unsafe. Much of the motivation for template literals in the first place is to support context sensitive escaping, where the escaping of the x data in

safeHTML`....${x}....`

depends on where in the html parsing of the literal parts it is encountered. See the work of Mike Samuel (cc'ed).

# T.J. Crowder (6 months ago)

On Thu, Dec 13, 2018 at 7:00 PM Mark Miller <erights at gmail.com> wrote:

As for the optional mapping function, there's nothing about that which is more relevant to cooked base strings than to raw base strings. We should be able to apply mapping functions to either, as well as to other base tags, in a similar way. This suggests tag combinators...

Probably as a separate thing? Since cook/cooked/interpolate isn't primarily a tag function.

As a completely separate point, this way of escaping html is not context sensitive, and likely horribly unsafe.

It was just a throwaway for the example, for the specific context of using the string within the body of a tag (not in attribute text, etc.). Correct me if I'm wrong, but for that context, I believe it's sufficient.

In this case, if we choose a name like "cook" or "cooked" in order to make the analogy with String.raw, then it should have the same API surface as String.raw.

Sadly, yes. I say "sadly" because I quite liked cook (or cooked), but only for frivolous reasons. :-)

+1 for Allen's String.interpolate. Something along these lines, accepting array-likes rather than iterables (though that's a discussion to have):

const toLength = n => {
    n = Math.min(+n || 0, Number.MAX_SAFE_INTEGER);
    return n < 0 ? Math.ceil(n) : Math.floor(n);
};
Object.defineProperty(String, "interpolate", {
    value(strings, subs = [], mapFn) {
        const strslen = toLength(strings.length);
        if (strslen <= 0) {
            return "";
        }
        const subslen = toLength(subs.length);
        let s = "";
        let index = 0;
        while (true) {
            const subIndex = index;
            s += strings[index++];
            if (index == strslen) {
                return s;
            }
            if (subIndex < subslen) {
                const sub = subs[subIndex];
                s += mapFn ? mapFn(sub) : sub;
            }
        }
    },
    configurable: true,
    writable: true
});

Example: jsfiddle.net/tk9vyrw3

Like String.raw, that ignores entries in subs at or after strings.length - 1. (It's basically String.raw's algorithm, and like raw's algorithm, it's un-optimized for clarity.) But that's also a discussion to have, perhaps it should tack them on the end. It won't come up in the simple use case of passing along the strings and subs in a tag function (subs will always have one fewer entry than strings), but it's slightly more general-purpose if it tacks them on the end.

String.interpolate would also fit nicely if at some stage there were an Array.interpolate (e.g., "zip").

-- T.J. Crowder

# Isiah Meadows (6 months ago)

I'll admit that HTML escaping tag was probably a bad example. It was just for show, nothing more, and obviously I wouldn't recommend it for production.

# Isiah Meadows (6 months ago)

I'll point out Kai could be on to something, although I disagree zip would be the right abstraction. Maybe Array.interleave(...arrays)? You could do Array.interleave(template, args).map(String).join("") for similar effect, and it'd be more generally useful.

The key here is that iteration would stop after the index hits any array's length, so it'd be polyfilled kinda like this:

Array.interpolate = (...args) => {
    let ret = []
    let lengths = []
    let count = 0
    for (let i = 0; i < args.length; i++) {
        lengths[i] = args[i].count
    }
    for (let index = 0; ; index++) {
        for (let i = 0; i < args.length; i++) {
            if (index === lengths[i]) return ret
            ret[count++] = args[i][index]
        }
    }
}

(This could be optimized, though.)

# Mike Samuel (6 months ago)

On Fri, Dec 14, 2018 at 12:51 PM Isiah Meadows <isiahmeadows at gmail.com>

wrote:

I'll point out Kai could be on to something, although I disagree zip would be the right abstraction. Maybe Array.interleave(...arrays)? You could do Array.interleave(template, args).map(String).join("") for similar effect, and it'd be more generally useful.

The key here is that iteration would stop after the index hits any array's length, so it'd be polyfilled kinda like this:

Array.interpolate = (...args) => {
    let ret = []
    let lengths = []
    let count = 0
    for (let i = 0; i < args.length; i++) {
        lengths[i] = args[i].count
    }
    for (let index = 0; ; index++) {
        for (let i = 0; i < args.length; i++) {
            if (index === lengths[i]) return ret
            ret[count++] = args[i][index]
        }
    }
}

(This could be optimized, though.)

As a data point, something like this loop appears in most of the template tags I've written but it's never had these precise semantics so I didn't bother putting it into template-tag-common www.npmjs.com/package/template-tag-common.

That library makes it easy to split the operation of a template tag into 3 stages:

  1. An optional configuration stage accessed by calling the template tag as a regular function: mytag({ /* options */)...
  2. Static analysis over the strings. This is memoized.
  3. Computing a result from (options, strings, results of step 2, interoplated values)

The final loop (step 3) in the template tags I maintain tends to looks like

function computeResult(options, staticState /* from step 2 */, strings, ...values) { let n = values.length; // Could do Math.max(strings.length - 1, values.length); let result = strings[0]; // Usually strings.raw for (let i = 0; i < n;) { const interpolatedValue = f(options, staticState[i], values[i]); // Sometimes code here looks backwards at the result to see if it needs to avoid token-merging hazards. result += interpolatedValue; result += strings[++i]; } return wrapResult(result); // Produce a value of a type that encapsulates the tag's security guarantees. }

# Isiah Meadows (6 months ago)

The main difference with that loop is that it's generalized to any number of arrays, not just two with the second array having length one less than the first. Otherwise, it'd look exactly the same. BTW, I like this route (Array.interleave) better since it doesn't have to result in just a single string result - it could just be an array of strings plugged into some API instead, or it could be procedurally streamed out in chunks.

# Mike Samuel (6 months ago)

On Fri, Dec 14, 2018 at 2:26 PM Isiah Meadows <isiahmeadows at gmail.com>

wrote:

The main difference with that loop is that it's generalized to any number of arrays, not just two with the second array having length one less than the first. Otherwise, it'd look exactly the same. BTW, I like this route (Array.interleave) better since it doesn't have to result in just a single string result - it could just be an array of strings plugged into some API instead, or it could be procedurally streamed out in chunks.

Fair enough. If you're not looking for something template tag specific then a simple zip over iterators should do it?

function *ziperator(iterators) { let progressed; do { progressed = false; for (let iterator of iterators) { for (let element of iterator) { yield element; progressed = true; break; } } } while (progressed); }

console.log(Array.from(ziperator([ ['a', 'b', 'c']Symbol.iterator, [1, 2]Symbol.iterator ])).join('')); // -> a1b2c

(but optimized :)

# Isiah Meadows (6 months ago)

I could go with an iterator equivalent, but I'd like to defer that to the seemingly-planned "iterlib" thing that's been considered since before ES2015 was released. Something that works with arrays is good enough for now.

BTW, your ziperator isn't really the same as my Array.interpolate (which is better named Array.interleave). It needs to be this:

function *ziperator(...iters) {
    for (let i = 0; i < iters.length; i++) {
        iters[i] = iters[i][Symbol.iterator]()
    }
    while (true) {
        for (let i = 0; i < iters.length; i++) {
            const {done, value} = iters[i].next()
            if (done) return undefined
            yield value
        }
    }
}

The optimized version is pretty straightforward (using private fields

  • methods here):
function ziperator(...iters) { return new InterleavedIterator(iters) }

class InterleavedIterator {
    #iters, #index
    constructor(iters) { this.#iters = iters; this.#index = 0 }
    [Symbol.iterator]() { return this }
    next(value) { return this.#invoke("next", value) }
    throw(value) { return this.#invoke("throw", value) }
    return(value) { return this.#invoke("return", value) }
    #invoke(method, value) {
        if (this.#iters == null) return {done: true, value: undefined}
        const index = this.#index
        this.#index = (index + 1) % this.#iters.length
        const {done, value} = this.#iters[index][method](value)
        if (done) this.#iters = undefined
        return {done, value}
    }
}

Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# Mike Samuel (6 months ago)

Fair enough.

# kai zhu (6 months ago)

I could go with an iterator equivalent, but I'd like to defer that to the seemingly-planned "iterlib" thing that's been considered since before ES2015 was released.

i'm against iterator-based design-patterns and libraries, as they lack a clear fit in most javascript solutions. the vast majority of looping in UX-workflows (javascript’s primary problem-domain) are over [serializable] JSON lists/dicts/strings, where Array/Object/String looping-methods will suffice.

all other use-cases are uncommon/complicated enough that custom for/while loops are usually better suited than custom-iterators.

# Isiah Meadows (6 months ago)

Could you bring that up in a different thread instead of driving this off-topic? Also, please don't forget that a very significant chunk of JS doesn't even run in a GUI environment (consider: servers, IoT, satellites, etc.).


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# kai zhu (6 months ago)

everything using javascript is ultimately meant to solve a UX-workflow problem (including js-code in servers, IoT, satellites, etc.). if you're so caught up in low-level js library-code that you can't see how the piece fits/integrates into that big-picture, then you should go back to general-purpose programming in java/c++/c#/etc.

but of course, much of industry these days prefer hiring business-oriented programmers focused on solving UX-workflow problems rather than general-purpose programming ones.

# Mike Samuel (6 months ago)

Kai, that there may be some tenuous connection to ux-workflow does not support your sweeping claims. Also please stop with the personal attacks.