Discuss
  • Guidelines and General Direction for Standard Library Proposals

Guidelines and General Direction for Standard Library Proposals

# T.J. Crowder (7 years ago)

I'd find it helpful to have:

  1. A general steer from TC39 as to whether the committee are committed to a robust, feature-rich standard library, or a minimal one (with the expectation of a rich ecosystem of libraries, presumably).

  2. Guidelines of what to consider when proposing standard library features.

The latter can come from the community; the former needs to come from TC39.

We see a fair number of simple lib function proposals on the list, it would be useful to have some guidelines for what's worth properly proposing and what's likely to be a non-starter.

Naveen Chawla's recent suggestion is a good case in point: A standard lib function for taking an array (ideally, an iterable) and creating an object with properties referencing the elements by one of their property's values. (One could easily argue for a Set version as well.) Ignoring the details of his suggestion, this is a simple operation that I've needed in every codebase I've ever worked on. I haven't proposed it (having considered doing so repeatedly) because my perception had been that TC39 wants to keep the standard lib small. But with the additions in 2015, 2016, and 2017, is that simply not the case (anymore)?

Apologies if either (1) or (2) already exist and I've overlooked them.

-- T.J. Crowder

# Michał Wadas (7 years ago)

My thoughts: big source of JavaScript criticism is its small standard library and community attempts to solve that issue - hundreds of small modules. Significant part of lodash should be present in standard library. My lodash selection - chunk, dropWhile, first, last, flatten, fromPairs, pull, without, takeWhile, zip, flatMap, keyBy, property, groupBy, partition, sample, shuffle, sortBy, ary, memoize, once, operators as functions, clamp, inRange, get, mapKeys, mapValues, pickBy, many strings operations, attempt, constant, flow, noop, range.

# kai zhu (7 years ago)

my thoughts are the opposite. javascript (similar to perl and php) already has a huge number of specialized builtin functions and methods accumulated over its 20+ year evolution that pretty much solves every practical problem encountered in frontend programming (and effectively backend as well). the problem is a failure to educate new programmers on how to use existing builtins to elegantly solve everyday problems, instead of having them ignorantly reinvent things on their own.

the mindset to writing javascript programs is more akin to writing perl programs, e.g. "how do i leverage builtins to implement the requested one-off UX feature using elegant perl-like one-liners", instead of "how do i create extra abstractions, classes, and modules to implement this one-off UX feature". My thoughts: big source of JavaScript criticism is its small standard library and community attempts to solve that issue - hundreds of small modules. Significant part of lodash should be present in standard library. My lodash selection - chunk, dropWhile, first, last, flatten, fromPairs, pull, without, takeWhile, zip, flatMap, keyBy, property, groupBy, partition, sample, shuffle, sortBy, ary, memoize, once, operators as functions, clamp, inRange, get, mapKeys, mapValues, pickBy, many strings operations, attempt, constant, flow, noop, range.

# Michał Wadas (7 years ago)

I disagree. PHP have WeakReferences, OGG manipulation, MPEG manipulation, builtin support for Kerberos, Bzip2 bindings, Zlib bindings, configurable random number generation, OpenSSL bindings, simple hashing, drivers for databases, ImageMagick bindings, arbitrary precision math, statistics support, GPG support, PostScript support, Judy Arrays, neural networks, Lua integration, FTP support, XML support, 9 data structures, recursive iterators with well defined interfaces etc.

All of them are missing in JavaScript. PHP have extremely bloated standard library - and JS have very minimal one.

# kai zhu (7 years ago)

inline

WeakReferences, -1 footgun most web-devs have no idea how to use correctly (including me)

OGG manipulation, -0.5 slightly prefer it as an npm module

MPEG manipulation, -0.5 slightly prefer it as an npm module

builtin support for Kerberos,

impartial

Bzip2 bindings, -1 use nodejs child_process system calls to bzip2. i don’t see any common use-case for bzip2 other than file-to-file operations which can be accomplished with shell-scripting.

Zlib bindings,

already a nodejs builtin

configurable random number generation, -1 Math.random is good-enough for most use-cases. for the normal-distribution use-case, its easy enough to write your own transform with sine/cosine.

OpenSSL bindings,

already a nodejs builtin

simple hashing, +1 would be nice to have

drivers for databases, +1 a STABLE nodejs sqlite3 builtin is sorely needed for embedded systems (instead of the flaky npm module), but need to nag nodejs folks, not tc39

ImageMagick bindings, -1 use nodejs child_process system calls to imagemagick. i'm skeptical builtin bindings would be significantly more efficient enough to be worth the effort.

arbitrary precision math, -0.5 generally against due to it confusing people with 2 different numeric types, though i slightly sympathize with accounting/finance use-cases for decimal arithmetic.

statistics support, +1 would be useful

GPG support,

impartial

PostScript support, -1 use nodejs child_process system calls, as bindings would not be significantly more efficient.

Judy Arrays, -1 too esoteric and confusing to understand what the use-cases are for most web-devs

neural networks, -1 slightly too big a scope, and more the domain of wasm

Lua integration,

Impartial

FTP support, -0.5 i don’t see much use-case for an ftp-client but i may be wrong

XML support,

impartial

9 data structures, -1 javascript is a high-level language with a builtin feature-set targeted for expressing high-level concepts like ajax, file-uploads, event-handling, dom-manipulations, programmable forward/reverse proxies, etc. its NOT a low-level language for creating esoteric data-structures and algorithms in academia that have little relevance to web-development.

recursive iterators

javascript already has recursive closures that i frequently use like recursive iterators

# Eli Perelman (7 years ago)

its NOT a low-level language for creating esoteric data-structures and

algorithms in academia that have little relevance to web-development.

Let's not get into the realm of opinion stated as fact. It also doesn't make sense to say that some things are useful in Node.js but then deny other things because they aren't useful for web development.

There's no reason JS can't be more low level, and I find that enabling some of these low level standard libs would be beneficial, even for web development. The reality is that JS is a language that exists in many environments including the web, servers, and hardware/bare metal, where package dependency considerations are real. Keep that in mind. :)

Thanks,

Eli Perelman

# kai zhu (7 years ago)

no, its far more practical and cost-efficient to leverage existing algo/neural libs through wasm than to try and reinvent them in javascript (and constantly change the language spec), if there is a need for them. let javascript do what it does best - as a high-level glue-language for web-development. and let the people who hate javascript happily write their algo libraries in c#/c++/java/etc and ship it as wasm. win-win.

# Eli Perelman (7 years ago)

let javascript do what it does best - as a high-level glue-language for

web-development. and let the people who hate javascript happily write their algo libraries in c#/c++/java/etc and ship it as wasm.

This is unfair and misleading. For those that have no intention of using another language and want to stick with JS, you're boxing them into an impossible situation. Also, you're generalizing all JS users as web developers.

Let's let JS do more than just web development, as Node.js has proven is desired.

Eli Perelman

# kai zhu (7 years ago)

This is unfair and misleading. For those that have no intention of using another language and want to stick with JS, you're boxing them into an impossible situation. Also, you're generalizing all JS users as web developers.

i'm an exclusive javascript dev for 5+ year. i have no intention of using another language and want to stick with js. i wanted to write a neural-net program in js, except i don’t know how to write a neural-net program in js. so i found a portable and standalone neural-net c library, learned just enough c to create a js interface for it and compiled it to asm.js (this was 2+ years ago before wasm was available, but you get the idea).

here's the old functional demo @ hrku01-fann-lite-beta.herokuapp.com, hrku01-fann-lite-beta.herokuapp.com

here's another example of something not too "web-devy" that i suspect is along the lines of what you want to do, but is better served using asm.js/wasm:

kripken.github.io/sql.js/GUI, kripken.github.io/sql.js/GUI

also, if i was a hiring manager, i would find it odd for someone to claim they are a heavy javascript user and NOT be a web-developer. even if you’re using nodejs in an arduino project, the fact you chose nodejs instead of say python, was likely because you intend to expose a server with a tcp/http interface and such.

# Michał Wadas (7 years ago)

I agree - neural networks should not belong to standard library. And that's why I said that PHP is overbloated.

But you claimed that JS standard library is similar in size to PHP - but it isn't, it's much, much smaller.

Deque isn't exotic data structure - but something needed. Basic trees are not exotic. Judy Arrays can be considered exotic, though.

Compare Python's library to JS. docs.python.org/3/library/index.html Big part of it isn't relevant to JavaScript (interactions with external world are left to vendor), but even then Python have everything that have JS and much more.

# kai zhu (7 years ago)

Deque isn't exotic data structure - but something needed.

javascript already has deque - Array.protoype.shift and Array.prototype.unshift

Basic trees are not exotic.

i've only once encountered a use-case that required tree-balancing algorithms and such (and it quickly turned out to be inappropriate, as the expected dataset was too small to warrant trees).
most in-memory datasets handled by javascript are less than 1gb (~100,000 records assuming 10kb per record), and a simple object/dictionary container is more appropriate than trees for such small number of records.

# Michael Kriegel (7 years ago)

Kai, you are not speaking for all Web-Developers here and Web-Developers are not the only users of JavaScript, please keep that both in mind. If you do not like the idea of e.g. weak references, do not use them. But for some caching tasks it is just necessary and I would love to see them.

You wrote "...is a failure to educate new programmers on how to use existing builtins to elegantly solve everyday problems..." - existing builtins do not provide anything which can be used to get weak references. And maybe you should educate yourself to find out, what a feature like "weak references" can be used for. Else you are just not in the position to say: "-1 footgun most web-devs have no idea how to use correctly (including me)" - it is not a lack of the feature, but a lack of yours. You shouldn't drive a train if you don't know how. But you also should not say that trains are useless, just because you do not know, how to operate them. I often saw, that you tend to argue that features, which you do not like or not know how to use are not needed by web developers, especially not in asia. Don't you think, that's a bit arrogate?

# kai zhu (7 years ago)

inline

And maybe you should educate yourself to find out, what a feature like "weak references" can be used for. Else you are just not in the position to say: "-1 footgun most web-devs have no idea how to use correctly (including me)" - it is not a lack of the feature, but a lack of yours. You shouldn't drive a train if you don't know how. But you also should not say that trains are useless, just because you do not know, how to operate them.

maybe you're right. retract my -1, though i don’t like the idea of everyday programmers playing with WeakMap. before javascript, i was a pythonista and worked with weakref's for caching. they never worked (or gc'd) quite the way I wanted, and i usually fallback to using vanilla python dicts for caching objects instead.

also, here's a recent tc39 discussion on unresolved WeakMap memory leaks: tc39/tc39-notes/blob/master/es8/2017-07/jul-27.md#840-template-objects-are-eternal-when-put-into-weakmap, tc39/tc39-notes/blob/master/es8/2017-07/jul-27.md#840-template-objects-are-eternal-when-put-into-weakmap

.. Don't you think, that's a bit arrogate?

yes it is arrogant of me. partly its because i care alot about javascript and am upset at how fast and loose people play with its language spec. you have to expect eventual pushback against things moving so fast (if not me, then from someone else eventually).