Additional Math functions
Aren't there limits to the number of arguments you can pass to a ES function before getting a stack overflow?
I've gotten in trouble trying to abuse the arguments array like this
before, for example Math.max.apply(Math, someVeryLargeArray).
Empirically, with iojs 1.8.1:
> Math.max.apply(Math, new Array(100000))
NaN
> Math.max.apply(Math, new Array(1000000))
RangeError: Maximum call stack size exceeded
I suppose it would be nice if JavaScript engines fell back to passing arguments on the heap to avoid this problem, but I don't think that's part of the ES6 spec. Am I mistaken?
If we're going to add math functions to the standard library, my vote would be for www.evanmiller.org/statistical-shortcomings-in-standard-math-libraries.html --scott
On Apr 29, 2015, at 9:04 AM, C. Scott Ananian <ecmascript at cscott.net <mailto:ecmascript at cscott.net>> wrote:
Aren't there limits to the number of arguments you can pass to a ES function before getting a stack overflow?
Yup. For example WebKit limits to around 10000. This protects our other stack overflow detection logic from overflow.
I've gotten in trouble trying to abuse the
argumentsarray like this before, for exampleMath.max.apply(Math, someVeryLargeArray).Empirically, with iojs 1.8.1:
> Math.max.apply(Math, new Array(100000)) NaN > Math.max.apply(Math, new Array(1000000)) RangeError: Maximum call stack size exceededI suppose it would be nice if JavaScript engines fell back to passing arguments on the heap to avoid this problem, but I don't think that's part of the ES6 spec. Am I mistaken?
That would be a great idea! Filed
On Apr 29, 2015, at 9:04 AM, C. Scott Ananian wrote:
... I suppose it would be nice if JavaScript engines fell back to passing arguments on the heap to avoid this problem, but I don't think that's part of the ES6 spec. Am I mistaken?
that's an implementation detail. The ES spec. doesn't care whether or not you employee multiple argument passing strategies in your implementation. Personally, I wouldn't bother unless for some reason the implementation had a very small (<10?) stack based argument limit.
Similar, anyone who who wants to spread a large array into an argument list is probably misguided.
@Allen: I guess that cloning the array into the arguments list is indeed heavier than needed. So when large arrays are used, a method that works on arrays will be better. But for small arrays of unknown size, spreading them looks like a nice syntactical help.
@Scott: those functions are indeed interesting too, but their implementation is a lot more complex AFAICS. If you need to code that natively, you'll be busy for a while.
2015-04-29 18:56 GMT+02:00 Allen Wirfs-Brock <allen at wirfs-brock.com>:
On Wed, Apr 29, 2015 at 12:56 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>
wrote:
On Apr 29, 2015, at 9:04 AM, C. Scott Ananian wrote:
I suppose it would be nice if JavaScript engines fell back to passing arguments on the heap to avoid this problem, but I don't think that's part of the ES6 spec. Am I mistaken?
that's an implementation detail. The ES spec. doesn't care whether or not you employee multiple argument passing strategies in your implementation. Personally, I wouldn't bother unless for some reason the implementation had a very small (<10?) stack based argument limit.
Similar, anyone who who wants to spread a large array into an argument list is probably misguided.
I'm just saying: it's not safe to spread an arbitrary array into
arguments unless the spec explicitly says that the number of arguments is
limited only by heap size (not stack size) or something like that. The ES6
spec does not contain any such language.
For tail calls, the spec says:
A tail position call must either release any transient internal resources associated with the currently executing function execution context before invoking the target function or reuse those resources in support of the target function. NOTE For example, a tail position call should only grow an implementation’s activation record stack by the amount that the size of the target function’s activation record exceeds the size of the calling function’s activation record. If the target function’s activation record is smaller, then the total size of the stack should decrease.
I'm not sure exactly how you'd write equivalent language for "arbitrary arguments size", something like a note on section 9.2.12 "FunctionDeclarationInstantiation" stating, "An execution context should use resources only for named arguments of the function. Unnamed formals should not increase the size of the Environment Record."
C. Scott Ananian wrote:
I'm just saying: it's not safe to spread an arbitrary array into
argumentsunless the spec explicitly says that the number of arguments is limited only by heap size (not stack size) or something like that. The ES6 spec does not contain any such language.
We've talked about this in past meetings. We want implementations and developers to cooperate and compete, to find a normative spec we can back. It would be silly for us to "pick a number".
Just a note about the whole upper bounds on arguments thing
In V8’s case (per the comment regarding io.js), the maximum arguments count for Function.prototype.apply/Reflect.apply/Reflect.construct is 0x800000 (v8/v8-git-mirror/blob/81345f1a2cdceaee8c891fc7512ae671f171308e/src/macros.py#L75, v8/v8-git-mirror/blob/81345f1a2cdceaee8c891fc7512ae671f171308e/src/macros.py#L75) — On a 64mb system, this ends up being close to 70mb of data which the engine pushes to the stack, in addition to being on the heap in the array itself — which is not super tiny, but a desktop machine can probably handle a lot more. This isn’t a “real” stack overflow, because the RangeError is thrown before any arguments are actually pushed to the stack — it’s just an upper bound on the size of the array which can be pushed. SpiderMonkey has a similar arbitrary upper bounds on the argument length, ARGS_LENGTH_MAX (mozilla/gecko-dev/blob/d03ee825c74355f070b14cf4325897c813373902/js/src/vm/ArgumentsObject.h#L77, mozilla/gecko-dev/blob/d03ee825c74355f070b14cf4325897c813373902/js/src/vm/ArgumentsObject.h#L77) or 0x7A120, which is a much more conservative limit. Apparently nobody has complained about the smaller limit, so that’s something =)
I guess what this means is, people aren’t generally depending on being able to push zillions of arguments to the stack via fn.apply — it’s almost always one of two things: either a small-ish array literal, or an actual Arguments object, which most likely doesn’t contain zillions of arguments either. These limits don’t seem unreasonable, and it’s probably okay to figure out an appropriate limit (especially for mobile devices) that still makes developers happy.
I've actually found this limit in the past [1] mostly for string operations and I remember it was already discussed in here.
2048 was a safe bet for all browsers I could test those days, but it'b be
very nice if any generic function, accordingly with the environment
capabilities, could expose somehow a Function.MAX_SAFE_ARGUMENTS_LENGTH
like property, similar to what Number.MAX_SAFE_INTEGER does already.
[1] 2011 post: webreflection.blogspot.co.uk/2011/07/about-javascript-apply-arguments-limit.html
Reviving this thread, doing any type of simple statistics is more verbose
than it probably needs to be as calculating sums, averages, etc. makes most
resort to Array reduction. I understand the need for methods such as
Math.hypot(...values), but for ECMAScript to evolve to be useful in
statistics programming without needing to resort to R would definitely also
be a nice-to-have. I'm sure there are many statistical functions you
could add, but at a bare minimum, it would be useful:
Math.sum(...values) or Math.summation(...values) Math.mean(...values) // Spelling out mean here is succinct while still intentional, instead of .avg Math.variance(...values) Math.stddev(...values)
Obviously some of these can be computed from the results of others, but I think the convenience and intent of the methods, not to mention their wide usefulness outweigh the concern of Math bloat. Thoughts?
P.S. Definitely not against even more core stats methods, but have to start somewhere. :)
Eli Perelman Mozilla
In case it didn't come across, this is the thread I'm reviving:
Eli Perelman Mozilla
While Math.sum() and Math.mean() currently don't exist, they can easily be polyfilled: See developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce#Sum_all_the_values_of_an_array for summarizing the values of an array and the following code for building the average of the array values:
let arr = [0, 1, 2, 3]; let total = arr.reduce(function(a, b) { return a + b; }); let mean = total / arr.length;
Calculating the variance and standard deviation would require a bit more code, though are also easy to polyfill. Non-the-less I can see the need for having standard functions for this.
Sebastian
On 10/01/2015 23:10, Sebastian Zartner wrote:
While Math.sum() and Math.mean() currently don't exist, they can easily be polyfilled: See developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce#Sum_all_the_values_of_an_array for summarizing the values of an array and the following code for building the average of the array values:
let arr = [0, 1, 2, 3]; let total = arr.reduce(function(a, b) { return a + b; }); let mean = total / arr.length;
Calculating the variance and standard deviation would require a bit more code, though are also easy to polyfill. Non-the-less I can see the need for having standard functions for this.
Sebastian
Yes, Math.sum can be polyfilled, but doing so if you want accurate answers takes a fair amount of care. The code above is pretty bad as a polyfill because it sometimes produces highly inaccurate answers. For example, if arr = [1, 1e18, -1e18], then this polyfill will return the incorrect value 0 for total, while a more careful implementation would return 1.
WaldemarIn addition, using reduce just to sum array elements forces a function execution for every element. May not be bad for a minimal count of elements, but doing statistical work on larger collections would benefit from having an optimized summation.
Eli Perelman
I really don't think I'd want a basic Math.sum(a, b, c) meaning anything
other than a + b + c, i.e. (a + b) + c. We should all just come to
terms with the fact that floating point addition is not associative.
Or is there really some simple, O(n) algorithm to do a better (more "careful") job?
On 10/02/2015 13:30, Alexander Jones wrote:
I really don't think I'd want a basic
Math.sum(a, b, c)meaning anything other thana + b + c, i.e.(a + b) + c. We should all just come to terms with the fact that floating point addition is not associative.Or is there really some simple, O(n) algorithm to do a better (more "careful") job?
Kahan summation is simple and O(n).
There exist efficient algorithms to get the exact sum as well. See, for example, www.ti3.tuhh.de/paper/rump/RuOgOi07I.pdf
WaldemarInteresting. I still feel that these algorithms should be given their
proper names in a math library, because I would feel quite troubled if
Math.sum(a, b, c) !== a + b + c. Maybe I'm alone in this view, though.
What do other languages do?
On 10/02/2015 16:37, Alexander Jones wrote:
Interesting. I still feel that these algorithms should be given their proper names in a math library, because I would feel quite troubled if
Math.sum(a, b, c) !== a + b + c. Maybe I'm alone in this view, though. What do other languages do?On Friday, 2 October 2015, Waldemar Horwat <waldemar at google.com <mailto:waldemar at google.com>> wrote:
On 10/02/2015 13:30, Alexander Jones wrote: I really don't think I'd want a basic `Math.sum(a, b, c)` meaning anything other than `a + b + c`, i.e. `(a + b) + c`. We should all just come to terms with the fact that floating point addition is not associative. Or is there really some simple, O(n) algorithm to do a better (more "careful") job? Kahan summation is simple and O(n). There exist efficient algorithms to get the exact sum as well. See, for example, http://www.ti3.tuhh.de/paper/rump/RuOgOi07I.pdf Waldemar
In the cases where the algorithms produce something other than the best practical results, giving them descriptive names would be useful. However, compound naming can then get out of hand if you also include functions such as average, standard deviation, etc., which include computing the sum as a subroutine.
WaldemarNote that the theoretical O(n) isn't always a good unit of measurement for practical speed. Having to keep extra precision variables, or having extra operations inside the loop will seriously slow down the computation, while keeping it O(n).
I'd prefer that Math.sum(a, b, c) == a + b + c. In practical summations, one will rarely sum numbers like 1, 1e18 and -1e18. When people really work with numbers so precise that 1 on a scale between -1e18 and 1e18 isn't equal to 0, then they probably shouldn't use native JS numbers and their functions with that data.
A simple function like Math.sum needs to be simple to standardise, simple to implement, and simple to understand for the users.
Just my opinion, Sander
2015-10-03 2:18 GMT+02:00 Waldemar Horwat <waldemar at google.com>:
On Thu, Oct 1, 2015 at 6:52 PM Eli Perelman <eli at eliperelman.com> wrote:
Reviving this thread, doing any type of simple statistics is more verbose than it probably needs to be as calculating sums, averages, etc. makes most resort to Array reduction. I understand the need for methods such as
Math.hypot(...values), but for ECMAScript to evolve to be useful in statistics programming without needing to resort to R would definitely also be a nice-to-have. I'm sure there are many statistical functions you could add, but at a bare minimum, it would be useful:Math.sum(...values) or Math.summation(...values) Math.mean(...values) // Spelling out mean here is succinct while still intentional, instead of .avg Math.variance(...values) Math.stddev(...values)
Obviously some of these can be computed from the results of others, but I think the convenience and intent of the methods, not to mention their wide usefulness outweigh the concern of Math bloat. Thoughts?
P.S. Definitely not against even more core stats methods, but have to start somewhere. :)
Can you write up some preliminary proposal specifications? Here are some useful resource/references to get started with proposal/spec authoring:
Am I the only one here wondering if at least most of this belongs in a separate library, rather than JS core?
- Most everyday programmers would be okay with this. 100% accuracy isn't a requirement, but speed often is.
Math.sum = (...xs) => xs.reduce((a, b) => a + b, 0);
Math.mean = (...xs) => Math.sum(...xs) / xs.length;
// etc...
- Most scientific programmers need accuracy to the finest point. Speed isn't usually of much priority beyond just the big-O complexity, but accuracy is.
I see two extremely different use cases here. Even a C implementation may not be able to satisfy both worlds. In tight loops, I will still see this, even in C++:
template<typename T>
static inline T sum(T* entries, int len) {
T sum = 0;
while (len) sum += entries[--len];
return sum;
}
We all know that it's subject to inaccuracy, but it's still useful in certain circumstances.
Does this belong in a library, not the spec itself?
We aren't really arguing whether this can be in a separate library; in fact, it is probably implemented many times over in separate libraries. My argument is more that there is a deficiency if this must be reimplemented many times across those libraries. Another deficiency lies in the fact that Math contains many complex functions but does not contain simple ones like summation and mean. I believe if ES can be the building block for a good math library of userland functions, there are probably several core functions that should be specified at the language level, because efficiency, etc.
Eli Perelman
Wouldn't it make sense to wait for the bind syntax [1] before introducing new methods that work on arrays? That way it could be added to a standard or non-standard module without altering either the Math object or the Array/Iterator prototype. Something like this:
import {sum, mean} from "iterator-math";
[1,2,3,4]::sum(); //10
[1,2,3,4]::mean(); //2.5
[{a:1}, {a:2}, {a:3}, {a:4}].map(x => x.a)::sum(); //10
[1] zenparsing/es-function-bind
Marius Gundersen
On Mon, Oct 5, 2015 at 2:58 PM Marius Gundersen <gundersen at gmail.com> wrote:
Wouldn't it make sense to wait for the bind syntax [1] before introducing new methods that work on arrays?
The functions don't accept arrays or operate on arrays—they accept any number of arguments (eg. Math.hypot, Math.min, Math.max) and operate on those arguments as a list.
That way it could be added to a standard or non-standard module without altering either the Math object or the Array/Iterator prototype. Something like this:
import {sum, mean} from "iterator-math"; [1,2,3,4]::sum(); //10 [1,2,3,4]::mean(); //2.5 [{a:1}, {a:2}, {a:3}, {a:4}].map(x => x.a)::sum(); //10
These aren't particularly compelling considering the following can written without waiting for a bind operator:
let { sum, mean } = Math;
sum(...[1, 2, 3, 4]) // 10 mean(...[1, 2, 3, 4]) // 2.5 sum(...[{a:1}, {a:2}, {a:3}, {a:4}].map(x => x.a)) //10
(And remain consistent with existing Math.*(...) operations)
+1 for operating on arrays, though. That's too common of a case for most of this. Either that or a common helper function:
function apply(xs, f) {
return f(...xs)
}
[1, 2, 3, 4]->apply(Math.sum)
(There is talk of changing the syntax and splitting up the proposal in zenparsing/es-function-bind#26. It could use a signal boost about now.)
On Fri, Oct 2, 2015 at 6:37 PM, Alexander Jones <alex at weej.com> wrote:
What do other languages do?
Well, Python's standard library has both a sum() builtin and a
statistics module. They handle floating-point weirdness differently:
>>> sum([1.0, 1e18, -1e18])
0.0
>>> import statistics
>>> statistics.mean([1.0, 1e18, -1e18])
0.3333333333333333
See www.python.org/dev/peps/pep-0450 for statistics module
rationale. I didn't find a PEP for sum().
...But for JS, the question is: why should we think TC39 and JS engines will be as good at satisfying this need as third-party libraries?
I hope that users who require numerical accuracy can just crack open a book. They should have, in JS, all the floating-point-munging tools they need to implement a book algorithm. If they don't, let's focus on providing those!
Maybe JS engine implementors can provide as-good accuracy with better speed. But not necessarily. A standard Math.sum() may even be slower, because TC39 is likely to specify nice, consistent, but inconvenient behavior, such as supporting the iterator protocol or specially handling array holes or infinities or NaN or -0. Library authors don't have to care about this stuff. TC39 absolutely has to care---about the risks of underspecification, at least---even at the expense of speed.
Jason, I agree. Although I didn't exactly make it explicit enough, that was my basic reasoning.
Why include numbers at all? Some people only manipulate strings so why not leave numbers to a third-party library that might be better? Or maybe we should use a string type that randomly drops lowercase characters because uppercase is good enough for many people. People that want to keep those characters can just use a third-party library.
On Fri, Oct 9, 2015 at 10:26 AM, Michael McGlothlin <mike.mcglothlin at gmail.com> wrote:
Why include numbers at all? Some people only manipulate strings so why not leave numbers to a third-party library that might be better?
You can try, but I don't think sarcasm alone is going to get you there.
If you want to do something productive, write the polyfill for the API you would like to see standardized. Make it available as a library. Show that there is a user base for that exact thing. If it gets to be widely used, and a native implementation really would be faster than the pure JS version, then you can point to JSON as precedent for standardization.
IMHO, there are still some rather basic Math functions missing in ES.
Math.sum([value1[, value2[, value3[, ...]]]])A Math.sum() function would mostly be handy thanks to the spread operator. So you can sum a complete array. Being able to do
would be so much cleaner than
Even though arrow functions make it a lot more concise than before, the code with Math.sum is still a lot easier to read IMO.
Reference implementation:
Math.avg(value1[, value2[, value3[, ...]]])Calculating the average is just the sum divided by the number of values. This might be less used, and when Math.sum is implemented, it might even be clean enough to type something like
Maybe it's also not used as often as summing an array. But I still expect it would enhance the vanilla ES experience.
Reference implementation:
Math.clamp(value, min, max)This would basically be the same as
Perhaps with extra consistency checks, throwing a rangeError when min>max
for example. The problem with combining Math.min and Math.max to achieve a clamp function is that it's way too easy to mix them up, and get bugs in your code.
There are of course other math functions that could be added (like the list here: moutjs.com/docs/latest/math.html ), but I think that these will be the more useful ones (also used more often than Math.hypot I assume).
I do understand that a bloated language is in nobody's interest, but when using reduce to sum an array, I always wondered if there's no other way to do it. Without hacking on the language, and by typing what you mean. Thanks to the spread operator, there now is a way to define such a function, but sadly, there's no default function like that yet.
Thanks for your consideration, Sander
IMHO, there are still some rather basic Math functions missing in ES. 1. `Math.sum([value1[, value2[, value3[, ...]]]])` A Math.sum() function would mostly be handy thanks to the spread operator. So you can sum a complete array. Being able to do ``` var sum = Math.sum(...arr); ``` would be so much cleaner than ``` var sum = arr.reduce((a,b) => a + b); ``` Even though arrow functions make it a lot more concise than before, the code with Math.sum is still a lot easier to read IMO. Reference implementation: ``` Math.sum = function() { var sum = 0; var length = arguments.length; for (var i = 0; i < length; i++) { sum += +arguments[i]; // note the conversion to number, to avoid string concatenation } return sum; }; ``` 2. `Math.avg(value1[, value2[, value3[, ...]]])` Calculating the average is just the sum divided by the number of values. This might be less used, and when Math.sum is implemented, it might even be clean enough to type something like ``` var avg = Math.sum(...arr) / arr.length; ``` Maybe it's also not used as often as summing an array. But I still expect it would enhance the vanilla ES experience. Reference implementation: ``` Math.avg = function() { var sum = 0; var length = arguments.length; for (var i = 0; i < length; i++) { sum += +arguments[i]; // note the conversion to number, to avoid string concatenation } return sum / length; } ``` 3. `Math.clamp(value, min, max)` This would basically be the same as ``` Math.max(Math.min(value, max), min); ``` Perhaps with extra consistency checks, throwing a rangeError when min>max for example. The problem with combining Math.min and Math.max to achieve a clamp function is that it's way too easy to mix them up, and get bugs in your code. There are of course other math functions that could be added (like the list here: http://moutjs.com/docs/latest/math.html ), but I think that these will be the more useful ones (also used more often than Math.hypot I assume). I do understand that a bloated language is in nobody's interest, but when using reduce to sum an array, I always wondered if there's no other way to do it. Without hacking on the language, and by typing what you mean. Thanks to the spread operator, there now is a way to define such a function, but sadly, there's no default function like that yet. Thanks for your consideration, Sander -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20150429/f26aeafb/attachment-0001.html>