Bringing setTimeout to ECMAScript
+1 to standardizing the timer family.
I always thought this wasn't in because the specification didn't have any asynchronism and specifying timers would open Pandora's box.
Right: the barrier to setTimeout functionality is that ECMAscript does not define a concurrency model.
If we can define a concurrency model, then we can build setTimeout.
Two things off the top of my head to consider:
- timer granularity
- forbidding eval-like syntax
OTOH, if we have an event-loop system with conditionals, we might not need setTimeout because it would be trivial to build from primitives. But it might be a handy interface if ES starts to go toward the "batteries included" model.
I think this is an excellent idea. I will add it to the concurrency strawman.
However, since the concurrency strawman is on he agenda for May and I'm currently overloaded preparing for next week's March meeting, I won't get to this till April. Once I have a draft, further feedback would be great.
On 18 March 2011 17:25, Wes Garland <wes at page.ca> wrote:
Right: the barrier to setTimeout functionality is that ECMAscript does not define a concurrency model.
If we can define a concurrency model, then we can build setTimeout.
Two things off the top of my head to consider: - timer granularity - forbidding eval-like syntax
OTOH, if we have an event-loop system with conditionals, we might not need setTimeout because it would be trivial to build from primitives. But it might be a handy interface if ES starts to go toward the "batteries included" model.
As I understand it, this type of thing was kept out of the language proper intentionally, because of its strong dependency on host environment. Some host environments may require tight and overriding control of any event handling system, and exactly which types of events (such as timeouts) are suitable to an environment may vary. A server side host might not want to have to deal with asynchronous activity at all, for instance.
As I understand it, this type of thing was kept out of the language proper intentionally, because of its strong dependency on host environment. Some host environments may require tight and overriding control of any event handling system, and exactly which types of events (such as timeouts) are suitable to an environment may vary. A server side host might not want to have to deal with asynchronous activity at all, for instance.
Speaking as someone who has written and currently maintains a synchronous
server-side JavaScript environment (based on V8), I attest to the statement
that I would not like it if V8 had setTimeout() (...etc) in it, because
unless V8 were going to take care of that completely black-box for me, then
I'd have to either disable such interfaces, or figure out some more
complicated functionality in my environment to handle the "concurrency".
I prefer they stay out of the engine, unless the engine is going to completely take care of it. The most important part of the engine "taking care of it" would be blocking the end of the program to wait for any outstanding event loop "turns" that had not yet fired, etc. Seems like that could get really messy.
On Mar 18, 2011, at 9:25 AM, Wes Garland wrote:
Right: the barrier to setTimeout functionality is that ECMAscript does not define a concurrency model.
If we can define a concurrency model, then we can build setTimeout.
See my previous reply. JS with setTimeout has only pseudo-concurrency via time-deferred script executions or function calls. There's nothing concurrent in a racy sense there.
Two things off the top of my head to consider:
- timer granularity
This is an interop issue, mostly due to stupid benchmarks that degenerate to measuring whether the browser clamps at 10ms or 4ms. It is also now a standard: 4ms.
Various implementors have found that setTimeout(f, 0) that calls f immediately "breaks the web" (or at least the New York Times site). Similarly, waiting an event loop turn seems to break the web. By testing, 4ms has been arrived at. I welcome fresher data.
- forbidding eval-like syntax
What problem are you solving?
OTOH, if we have an event-loop system with conditionals, we might not need setTimeout because it would be trivial to build from primitives. But it might be a handy interface if ES starts to go toward the "batteries included" model.
What do you mean by conditionals? If you mean condition variables, JS will never grow to include shared-mutable-state threads. That would happen not only over my dead body, but the rest of TC39's. We'd fight back, so watch out.
On Mar 18, 2011, at 9:51 AM, Kyle Simpson wrote:
As I understand it, this type of thing was kept out of the language proper intentionally, because of its strong dependency on host environment. Some host environments may require tight and overriding control of any event handling system, and exactly which types of events (such as timeouts) are suitable to an environment may vary. A server side host might not want to have to deal with asynchronous activity at all, for instance.
Speaking as someone who has written and currently maintains a synchronous server-side JavaScript environment (based on V8), I attest to the statement that I would not like it if V8 had
setTimeout()(...etc) in it, because unless V8 were going to take care of that completely black-box for me, then I'd have to either disable such interfaces, or figure out some more complicated functionality in my environment to handle the "concurrency".
First, as a matter of principle, if it's in ES6 then V8 will implement. So I'm told by people I trust.
Second, what about your embedding is hostile to setTimeout, which is more accurately weakly-isochronous than "asynchronous"?
I prefer they stay out of the engine, unless the engine is going to completely take care of it. The most important part of the engine "taking care of it" would be blocking the end of the program to wait for any outstanding event loop "turns" that had not yet fired, etc. Seems like that could get really messy.
Read one way (the worst-case interpretation), this shows great confusion about "threads suck", i.e., setTimeout requires no multi-threading. In no scenario would there ever be multi-threaded blocking with races over shared-mutalble state. What gave you this idea?
Read another way, if you mean pseudo-threads implemented with setTimeout never preempt one another but must all end before some larger notion of "the program" ends, then what is the problem, exactly?
On Mar 18, 2011, at 5:54 AM, Peter van der Zee wrote:
+1 to standardizing the timer family.
I always thought this wasn't in because the specification didn't have any asynchronism and specifying timers would open Pandora's box.
How so? I created JS and its run-to-completion execution model and setTimeout all at once in 1995. It has not changed in execution-model semantics since.
There's no contradiction: programs including those eval'ed by setTimeout (or functions passed instead of a string to eval and called later) must and in fact do not nest or preempt any other executing JS in correct (to this approximation) implementations.
The issues with setTimeout have more been about "core langage" vs. "DOM level 0", and the lack of a "DOM level 0" standard prior to HTML5.
An HTML attachment was scrubbed... URL: esdiscuss/attachments/20110318/f0d3d445/attachment-0001
On 3/18/11 7:26 PM, Brendan Eich wrote:
Two things off the top of my head to consider:
- timer granularity
This is an interop issue, mostly due to stupid benchmarks that degenerate to measuring whether the browser clamps at 10ms or 4ms. It is also now a standard: 4ms.
More precisely, the standard is "at least 4ms".
Browsers must delay nested timeouts by at least 4ms. They can delay them by more if desired (and various browsers do or will in various circumstances: IE9 when on battery, Chrome and Firefox in background tabs, possibly other cases).
Le 19/03/2011 00:26, Brendan Eich a écrit :
On Mar 18, 2011, at 9:25 AM, Wes Garland wrote:
Two things off the top of my head to consider:
- timer granularity This is an interop issue, mostly due to stupid benchmarks that degenerate to measuring whether the browser clamps at 10ms or 4ms. It is also now a standard: 4ms.
Various implementors have found that setTimeout(f, 0) that calls f immediately "breaks the web" (or at least the New York Times site). Similarly, waiting an event loop turn seems to break the web. By testing, 4ms has been arrived at. I welcome fresher data.
As I said, one way to bring setTimeout would be to standardized a lower-level mechanism based on which setTimeout could be implemented. Maybe that this timer granularity could be reconsidered for the lower-level mechanism. Maybe that this lower-level mecanism could count nanoseconds (I'm purposefully exagerating). Moreover, maybe that a standardized setTimeout could leave some implementation-defined room. For web browsers, 4ms as a minimum may have become a standard, but maybe that node.js or other ES-based languages do not care about this de-facto standard since they don't have to be interoperable with web browser code.
- forbidding eval-like syntax What problem are you solving?
I would agree to forbid eval-like syntax. Of course, it would be stupid to do forbid such code in general since it's already in use. However, it could be forbidden in strict mode (but as said in my first e-mail, with FF4 shipping in 3 days with strict mode, without forbidding, it may be too late to consider such an option. If it is not, please tell so). If people trigger Harmony code by an opt-in, maybe we can catch that train.
The problem I see and that I would like to solve would not be technical at all, it would educational. Currently, people can do eval, they can pass strings as handlers to HTML attributes (with the famous "return false;") or through JS by assigning the myDiv.onclick property. They can use strings in setTimeout. They can decide to never use anonymous function and be in complete denial of ECMAScript being a functional language. However, if we bring setTimeout to ECMAScript Harmony, we can have the ability to say to people "please use functions instead of strings for setTimeout in Harmony code" (the last part is important to not break existing code). This could be a first step in explaining that functions can be something else. And let's face it, it's not a big step, it's just de-quoting and wrapping with function(){ ... } (or even #(){...} ?). With good error messages it will help people transitionning quickly and learning something on the way. The best argument I can give on functions being better than strings is code anyoneI have written at the very beginning. I used a string in setTimeout and was creating it dynamically with the value of some variable. this is hard to maintain, probably hard to optimize for the engine. With a function, you just use capture the variable value withing the function used as a closure. No maintainability issue, no problem to optimize.
That's my only opinion and it's fine if nobody agrees but at least it is said.
On Mar 18, 2011, at 7:18 PM, David Bruant wrote:
As I said, one way to bring setTimeout would be to standardized a lower-level mechanism based on which setTimeout could be implemented.
Now you've got two problems...
It is not precisely understood why setTimeout(..., 0) breaks the web if run nested or on next event turn. Arv may be able to supply details from one or a few sites. I suspect the whole problem cannot be understood in any bounded interval, since we're talking not only about the Google-indexable web but also paywalled content, etc.
I'll go with browser-market-share-tested phenomenology (which means "not just Chrome, yet").
Maybe that this timer granularity could be reconsidered for the lower-level mechanism. Maybe that this lower-level mecanism could count nanoseconds (I'm purposefully exagerating).
Let's go from where we are, based on field testing. Theory is not really helpful.
- forbidding eval-like syntax What problem are you solving? I would agree to forbid eval-like syntax.
Why? Given the no-nesting, future-event-turn execution, what invariants are upset by the eval flavor of setTimeout?
Speaking as someone who has written and currently maintains a synchronous server-side JavaScript environment (based on V8), I attest to the statement that I would not like it if V8 had
setTimeout()(...etc) in it, because unless V8 were going to take care of that completely black-box for me, then I'd have to either disable such interfaces, or figure out some more complicated functionality in my environment to handle the "concurrency".First, as a matter of principle, if it's in ES6 then V8 will implement. So I'm told by people I trust.
Second, what about your embedding is hostile to setTimeout, which is more accurately weakly-isochronous than "asynchronous"?
I prefer they stay out of the engine, unless the engine is going to completely take care of it. The most important part of the engine "taking care of it" would be blocking the end of the program to wait for any outstanding event loop "turns" that had not yet fired, etc. Seems like that could get really messy.
Read one way (the worst-case interpretation), this shows great confusion about "threads suck", i.e., setTimeout requires no multi-threading. In no scenario would there ever be multi-threaded blocking with races over shared-mutalble state. What gave you this idea?
Read another way, if you mean pseudo-threads implemented with setTimeout never preempt one another but must all end before some larger notion of "the program" ends, then what is the problem, exactly?
I understand that JavaScript doesn't have threads. I also understand that
JavaScript doesn't have true concurrency. I furtermore understand that when
I call setTimeout(fn,1000), it queues up fn to run after at least
1000ms, or later, at the next earliest "break" where there's a free "turn"
for it to run.
What I was saying is, if I run this program through V8 right now (with its theoretical future support of setTimeout() included), then what will happen:
function fn() { print("hello"); } for (var i=0; i<10; i++) { setTimeout(fn,i*1000); }
That for-loop will finish very quickly (probably <1 ms). Would V8 (or any other JS engine) "finish" in the sense that the calling embedding code thinks this program is completely finished, and it returns back control to the C/C++ embedding layer when:
a) right after the for-loop finishes; OR
b) after only the first call to fn, since it's timeout was effectively 0,
and so would have been immediately after the main program finished; OR
c) after all of the queued up calls to fn have finished, about 9 seconds
later?
I have a C/C++ program that embeds the V8 API, and it loads up a bit of
JavaScript from a file, and it tells V8 to execute that bit of code, then it
loads up another file, and tells it to run that code, etc. What's
problematic in my mind is if my hosting environment would be signaled that
the first "program" finished, if there were still fns that were queued up
to be called. I would need the V8 execution API from my C/C++ code to be
"blocked" and to wait for all of the "turns" of that code to be exhausted,
and all the "isochronous" queued fn calls to finish, before letting me go
on to run my next program.
In other words, to put it simply, if program A can call setTimeout(), and I want to run program A and then program B, I have to be able to make sure that I don't try to run program B until everything is fully finished in A. As V8 stands now, there's no way to do anything non-synchronous, so when A finishes, I know it's totally finished. I'm concerned that there'd be some new way with setTimeout()'s that this wouldn't be true.
On Fri, Mar 18, 2011 at 4:26 PM, Brendan Eich <brendan at mozilla.com> wrote:
On Mar 18, 2011, at 9:25 AM, Wes Garland wrote:
Right: the barrier to setTimeout functionality is that ECMAscript does not define a concurrency model.
If we can define a concurrency model, then we can build setTimeout.
See my previous reply. JS with setTimeout has only pseudo-concurrency via time-deferred script executions or function calls. There's nothing concurrent in a racy sense there.
Two things off the top of my head to consider:
- timer granularity
This is an interop issue, mostly due to stupid benchmarks that degenerate to measuring whether the browser clamps at 10ms or 4ms. It is also now a standard: 4ms.
Various implementors have found that setTimeout(f, 0) that calls f immediately "breaks the web" (or at least the New York Times site). Similarly, waiting an event loop turn seems to break the web. By testing, 4ms has been arrived at. I welcome fresher data.
Is this 4ms codified in spec language somewhere? I couldn't find it by casual searching.
What about setTimeout(thunk, 0.00001) ? In other words, is the rule
actualTimeout = max(requestedTimeout, 4)
or
actualTimeout = requestedTimeout || 4
?
- forbidding eval-like syntax
What problem are you solving?
OTOH, if we have an event-loop system with conditionals, we might not need setTimeout because it would be trivial to build from primitives. But it might be a handy interface if ES starts to go toward the "batteries included" model.
What do you mean by conditionals? If you mean condition variables, JS will never grow to include shared-mutable-state threads. That would happen not only over my dead body, but the rest of TC39's. We'd fight back, so watch out.
+1. Add my body to that list ;).
On 3/19/11 12:24 AM, Mark S. Miller wrote:
Is this 4ms codified in spec language somewhere? I couldn't find it by casual searching.
www.whatwg.org/specs/web-apps/current-work/multipage/timers.html#dom-windowtimers-settimeout step 5 (though note also step 9, which is the part that makes 4ms a minimum delay, not a hard line).
Also note that the 4ms thing only applies to "nested" timeouts.
actualTimeout = max(requestedTimeout, 4)
This is the rule for nested timeouts, yes.
-Boris
P.S. If one draft spec is preferred over another one, the same text is at dev.w3.org/html5/spec/Overview.html#dom-windowtimers-settimeout and at www.w3.org/TR/html5/timers.html#dom
On Fri, Mar 18, 2011 at 7:26 PM, Brendan Eich <brendan at mozilla.com> wrote:
If we can define a concurrency model, then we can build setTimeout.
See my previous reply. JS with setTimeout has only pseudo-concurrency via time-deferred script executions or function calls. There's nothing concurrent in a racy sense there.
pseudo-concurrency is a type of concurrency. I expect that any concurrency model standardized into ES will be event-loop concurrency, as implemented today in every browser and most server-side platforms that have any kind of concurrency at all.
Various implementors have found that setTimeout(f, 0) that calls f
immediately "breaks the web" (or at least the New York Times site). Similarly, waiting an event loop turn seems to break the web. By testing, 4ms has been arrived at. I welcome fresher data.
I don't have "breaks the web" data, but executing f before setTimeout() returns (rather than wait for us to get back in the event loop) changes the semantics of the program. This might seem obvious, but I have seen the analogue in suggestions in under the CommonJS umbrella for both module loaders and promises, where the authors were trying to unify the "async" (wince) and "sync" (more wincing) environments of systems with and without event loops. I have pointed out in those contexts that to insure deterministic program results that some kind of event loop needs to be used even platforms like the one Kyle was describing.
You introduced the term isosynchronous in this thread. Could you define it in this context? I am frustrated by the vocabulary I normally see used when considering systems with and without event loops. "async" and "sync" hurt me. For the time being, I'm assuming by isosynchronous you mean the same concurrency model as a C program on UNIX with neither threads nor fork.
- forbidding eval-like syntax What problem are you solving?
I suggested that for consideration, and after considering it, withdraw that suggestion, substituting instead the suggestion that setTimeout() forms which accept string arguments should behave as though they were lazily evaluated with indirect-eval.
My off-the-cuff suggestion attempted to solve the programmer-confusion problem I see surrounding setTimeout. The main problem with my suggestion, of course, is that you can't put the cat back in the bag.
What do you mean by conditionals? If you mean condition variables, JS will
never grow to include shared-mutable-state threads. That would happen not only over my dead body, but the rest of TC39's. We'd fight back, so watch out.
Good Lord, no. I was actually thinking of gdb watchpoint conditions when I wrote that. SetTimeout is, after all, just a way to place a hunk of code on the event loop that conditionally runs based on the clock.
Kyle;
On Fri, Mar 18, 2011 at 11:53 PM, Kyle Simpson <getify at gmail.com> wrote:
Speaking as someone who has written and currently maintains a synchronous
server-side JavaScript environment (based on V8),
I resemble this description, except that my platform is built on SpiderMonkey rather than V8. That said, I do not share your concerns with respect to the introduction of setTimeout.
You've laid out a number of implementation scenarios, but I believe they are properly addressed by the embedding API for your JS engine. Basically, the JS engine can do one of two things:
- not return until there are no more pending events
- return immediately, and let the embedder know that there is still code waiting to be serviced, and to provide some kind of servicing API.
.#1 is an unlikely implementation, as it would not play nicely with the browser, and, frankly, makes for a lousy API. In the strange case where this happens and the program needs to terminate immediately without concern for pending events, you would have to add some kind of mechanism in your platform that causes you to bail immediately.
One approach I have implemented in my own work is to throw a Number rather than an Error; e.g. throw 3 behaves a lot like exit(3) in C.
.#2 is far more likely and gives you total control. Either you can ignore pending events, or you can wait to service them. #2 not only plays well with the browser, but would be very easy to integrate with something like libev if you wanted to implement a platform like Node.js. I would be tickled pink if something like this appeared in SpiderMonkey.
What I was saying is, if I run this program through V8 right now (with its
theoretical future support of setTimeout() included), then what will happen:
function fn() { print("hello"); } for (var i=0; i<10; i++) { setTimeout(fn,i*1000); }
That for-loop will finish very quickly (probably <1 ms). Would V8 (or any other JS engine) "finish" in the sense that the calling embedding code thinks this program is completely finished, and it returns back control to the C/C++ embedding layer when:
a) right after the for-loop finishes; OR b) after only the first call to
fn, since it's timeout was effectively 0, and so would have been immediately after the main program finished; OR c) after all of the queued up calls tofnhave finished, about 9 seconds later?
a) is my #2, and I believe the right solution -- but I stress that this is an engine question and not a spec question. b) should never happen, and anyhow, it's a special case of either a or c. c) is my #1, and I believe is not a solution any browser-vendor's engine would implement....unless the JS engine's event loop also ran the browser, which strikes me as a backwards design.
In other words, to put it simply, if program A can call setTimeout(), and I
want to run program A and then program B, I have to be able to make sure that I don't try to run program B until everything is fully finished in A. As V8 stands now, there's no way to do anything non-synchronous, so when A finishes, I know it's totally finished. I'm concerned that there'd be some new way with setTimeout()'s that this wouldn't be true.
If V8 implements something like #2, they will need to include some kind of an API letting the embedder (you) know about pending events so that you can run them.
Like I said, I don't see this being an issue for our use-cases, provided the engine implementors continue to make good decisions.
On Mar 18, 2011, at 8:53 PM, Kyle Simpson wrote:
What I was saying is, if I run this program through V8 right now (with its theoretical future support of setTimeout() included), then what will happen:
function fn() { print("hello"); } for (var i=0; i<10; i++) { setTimeout(fn,i*1000); }
That for-loop will finish very quickly (probably <1 ms). Would V8 (or any other JS engine) "finish" in the sense that the calling embedding code thinks this program is completely finished, and it returns back control to the C/C++ embedding layer when:
a) right after the for-loop finishes; OR b) after only the first call to
fn, since it's timeout was effectively 0, and so would have been immediately after the main program finished; OR c) after all of the queued up calls tofnhave finished, about 9 seconds later?
(a) is the right answer because the event loop is run by the embedding, not v8. IMnsHO.
In other words, to put it simply, if program A can call setTimeout(), and I want to run program A and then program B, I have to be able to make sure that I don't try to run program B until everything is fully finished in A. As V8 stands now, there's no way to do anything non-synchronous, so when A finishes, I know it's totally finished. I'm concerned that there'd be some new way with setTimeout()'s that this wouldn't be true.
This is not true today with V8 in Chrome, precisely due to setTimeout!
It's not true in Node either.
I'm not sure where you think it's true. V8 embedding in something without setTimeout or anything like it, sure. But then you can't write simple time-based programs, set alarms, etc., so back in the door comes setTimeout or a workalike...
On Mar 18, 2011, at 9:43 PM, Wes Garland wrote:
I don't have "breaks the web" data, but executing f before setTimeout() returns (rather than wait for us to get back in the event loop) changes the semantics of the program. This might seem obvious, but I have seen the analogue in suggestions in under the CommonJS umbrella for both module loaders and promises, where the authors were trying to unify the "async" (wince) and "sync" (more wincing) environments of systems with and without event loops. I have pointed out in those contexts that to insure deterministic program results that some kind of event loop needs to be used even platforms like the one Kyle was describing.
Good point.
You introduced the term isosynchronous in this thread. Could you define it in this context?
Something occurring at a regular interval.
Obvious setInterval is more precisely isochronous but my point was that, ignoring clamping and latency in the scheduler and due to (virtualized-by-JS and virtualized-by-the-OS) CPU hogging, timeouts run at some number of msec in the future. Not like i/o which may happen indefinitely later (but within the TCP connection timer ;-).
Read more: www.answers.com/topic/isochronous#ixzz1H1O5GiCJ
I am frustrated by the vocabulary I normally see used when considering systems with and without event loops. "async" and "sync" hurt me. For the time being, I'm assuming by isosynchronous you mean the same concurrency model as a C program on UNIX with neither threads nor fork.
Not what I meant, although you could imagine pause between script, event handler, and timeout runs (event loop turns), and whatever signal has good enough resolution for alarming the pausing process.
You're right that "async" is abused and confusing enough that we wince. "Event loop concurrency" is a mouthful but hits the general execution model target. "No preemption" is important. And for setTimeout and setInterval, talking about time-based turns, reliable within some constraints modulo quality of implementation and availability denial issues, must matter.
It seems to me there are a couple pieces to Mark's concurrency proposal. One part is formalizing the event queue that already exists. Another part is introducing new concepts and features to the language (like promises and vats).
I want to hear what Mark has to say at the TC39 meeting, but my gut tells me it's too early to introduce new concurrency constructs for ES.next. There's actually a lot you can do with closures and objects (such as the promises libraries people have been experimenting with lately). And with generators, you can implement some really convenient constructs. I've recently been building a concurrency library called jsTask, that makes it possible to write blocking computations in a really natural style:
http://blog.mozilla.com/dherman/2011/03/11/who-says-javascript-io-has-to-be-ugly/
http://dherman.github.com/jstask/
As for event queue semantics, I think in principle that Ecma-262 is a reasonable place to specify it. It doesn't rely on specifics of the browser, and Node.js is a testament to that. (Besides, a non-interactive setting could probably be modeled as a trivial event queue, i.e. one with just a single "run the program" event.) I'm not sure whether this is worth trying to accomplish in time for ES.next, though.
On Sat, Mar 19, 2011 at 1:13 AM, Brendan Eich <brendan at mozilla.com> wrote:
On Mar 18, 2011, at 5:54 AM, Peter van der Zee wrote:
+1 to standardizing the timer family.
I always thought this wasn't in because the specification didn't have any asynchronism and specifying timers would open Pandora's box.
How so? I created JS and its run-to-completion execution model and setTimeout all at once in 1995. It has not changed in execution-model semantics since.
I can't think of a single way to simulate setTimeout in ES5. Correct me if I'm wrong, but I don't think ES5 exposes a single way of defining a mechanism like:
-- var x = 4; function f(){ x = 5; print(x); } timer(f, 1); print(f);
Such that it would output 4 before 5. That's what I meant with "didn't have any asynchronism", fwiw.
There's no contradiction: programs including those eval'ed by setTimeout (or
functions passed instead of a string to eval and called later) must and in fact do not nest or preempt any other executing JS in correct (to this approximation) implementations.
Ok, I never meant contradiction. Maybe "Pandora's box" should have been "Let the cat out of the bag" ;) I just never bothered to mention it (introducing timers) on esdiscuss. Seemed to be such an obvious miss to me that there had to be an obvious reason.
The issues with setTimeout have more been about "core langage" vs. "DOM level 0", and the lack of a "DOM level 0" standard prior to HTML5.
What do you mean by that? Why should that influence the choice of including it in the ECMA spec? What about embedded environments or ssjs?
Le 19/03/2011 07:02, David Herman a écrit :
It seems to me there are a couple pieces to Mark's concurrency proposal. One part is formalizing the event queue that already exists.
Is this already done in the current proposal? Because I haven't found it. Regardless, this work of formalizing the already existing event queue has already been done by the WHATWG: www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loops I think it would be a mistake to start the formalization work from scratch since a decent part of the work has already been done.
Another part is introducing new concepts and features to the language (like promises and vats).
I want to hear what Mark has to say at the TC39 meeting, but my gut tells me it's too early to introduce new concurrency constructs for ES.next. There's actually a lot you can do with closures and objects (such as the promises libraries people have been experimenting with lately). And with generators, you can implement some really convenient constructs. I've recently been building a concurrency library called jsTask, that makes it possible to write blocking computations in a really natural style:
blog.mozilla.com/dherman/2011/03/11/who-says-javascript-io-has-to-be-ugly, dherman.github.com/jstask
Once, I read the concurrency proposal with in mind "can setTimeout be implemented with what is in this proposal?". I have found the timeout section (strawman:concurrency#timeouts) and I thought "awesome!"... until I realized that this example is itself using setTimeout (which is consistently implemented in most ECMAScript-based environments but isn't itself part of any ECMAScript version). ( By the way, there is a typo in this example. I think that "setTimeout(const(){...}, 0)" should be "setTimeout(const(){...}, millis)", where "millis" is the second argument. )
I fully agree that the current proposal and all related work (yours included) offer very powerful mecanisms and solutions to common problems (I had already read your article on "Who says JavaScript I/O has to be ugly?" and really enjoyed what I saw). However, the tiny brick/element/component/part that is missing to implement "time event" ("clock event"?) is to take time into account in the proposal. No matter how powerful the current proposal is, it is lacking this elementary piece in order to implement/standardize setTimeout. And I intuit it wouldn't be that hard to add it. I am not very familiar with promises but from what I understand, having the ability to create a "timed promise" which resolution would be handled by the engine could be a solution. Once again, I'm not familiar with promises, so this idea may sound stupid, but people who are familiar with promises and the Q API will certainly have better ideas on integrating the notion of time to the concept in a way that could make setTimeout and friends implementable on top of this integration.
As for event queue semantics, I think in principle that Ecma-262 is a reasonable place to specify it. It doesn't rely on specifics of the browser, and Node.js is a testament to that. (Besides, a non-interactive setting could probably be modeled as a trivial event queue, i.e. one with just a single "run the program" event.) I'm not sure whether this is worth trying to accomplish in time for ES.next, though.
Does the fact that some event loop formalization work has already been done by the WHATWG within Standard HTML change your opinion on the matter?
I can't think of a single way to simulate setTimeout in ES5. Correct me if I'm wrong, but I don't think ES5 exposes a single way of defining a mechanism like:
var x = 4; function f(){ x = 5; print(x); } timer(f, 1); print(f);
Such that it would output 4 before 5. That's what I meant with "didn't have any asynchronism", fwiw.
while (!programHasQuit()) {
timers= (function () { var timers = []; var id=0; timer=function (f,t) { timers.push({func:f, interval:t, id:id++}); return id; } return timers; })
runScript("myscript.js"); handleEvents(timers,otherevents);
}
and there you go, in pure JS. If this is a gui program, you may expose a queue of GUI events to this "master script", but I believe that the event loop is best left to the embedding. If setTimeout etc are implemented in the core JS engine, the JS engine can simply expose the pool of timers as some data structure to the embedded C/C++ program to do with what you wish- The standard would presumably specify some guidelines for this.
never write code on no sleep.
that code sample should be :
timers= (function () { var timers = []; var id=0; timer=function (f,t) { timers.push({func:f, interval:t, id:id++}); return id; } return timers; })
runScript("env.js"); runScript("program.js"); ev=getEvents(); while (!programHasQuit()) {
handleEvents(timers,ev);
}
On 19 Mar 2011 14:43, "Breton Slivka" <zen at zenpsycho.com> wrote:
I can't think of a single way to simulate setTimeout in ES5. Correct me
if
I'm wrong, but I don't think ES5 exposes a single way of defining a mechanism like:
var x = 4; function f(){ x = 5; print(x); } timer(f, 1); print(f);
Such that it would output 4 before 5. That's what I meant with "didn't
have
any asynchronism", fwiw.
while (!programHasQuit()) {
timers= (function () { var timers = []; var id=0; timer=function (f,t) { timers.push({func:f, interval:t, id:id++}); return id; } return timers; })
runScript("myscript.js"); handleEvents(timers,otherevents);
}
and there you go, in pure JS. If this is a gui program, you may expose a queue of GUI events to this "master script", but I believe that the event loop is best left to the embedding. If setTimeout etc are implemented in the core JS engine, the JS engine can simply expose the pool of timers as some data structure to the embedded C/C++ program to do with what you wish- The standard would presumably specify some guidelines for this.
That only works if the host environment isn't waiting for you to finish / wait / sleep / yield / etc. But yeah, ok :)
What I was saying is, if I run this program through V8 right now (with its theoretical future support of setTimeout() included), then what will happen:
function fn() { print("hello"); } for (var i=0; i<10; i++) { setTimeout(fn,i*1000); }
That for-loop will finish very quickly (probably <1 ms). Would V8 (or any other JS engine) "finish" in the sense that the calling embedding code thinks this program is completely finished, and it returns back control to the C/C++ embedding layer when:
a) right after the for-loop finishes; OR b) after only the first call to
fn, since it's timeout was effectively 0, and so would have been immediately after the main program finished; OR c) after all of the queued up calls tofnhave finished, about 9 seconds later?(a) is the right answer because the event loop is run by the embedding, not v8. IMnsHO.
OK, so what I was asserting in my earlier email is, I'd prefer that if setTimeout() were going to be added to the official language spec, and in fact directly implemented in the engines, that instead of making the embedder's job harder (ie, me), that the engine entirely encapsulate and manage the event loop.
I realize I'm certainly in the vast minority with that opinion, but I'm simply saying that not everyone who embeds JavaScript deals with life in an asynchronous/isosynchronous mindset. We're not all clones of Node.js. My embedding is entirely synchronous, and always will be, for its use case. Which means that if we introduce asynchronicity/isosynchronicity at some level, like in the actual JavaScript layer (with setTimeout(), etc), then at some higher level, like the engine (my preference), or my embedding (not my preference), "blocking" behavior has to be written so that I can maintain the external synchronicity that my embedding provides. Namely, I have to be able to ensure that program A is totally done before program B runs.
Could I figure out event-loop semantics in my C/C++ layer, assuming the V8 API exposed some API way to determine if events were still unfulfilled, and basically provide the event-loop functionality per-program in my embedding? Sure, I guess I could always learn how to code that stuff. Or, I could simply go to extra work to disable any such native functions so that code running through my embedding cannot create isosynchronous conditions.
Neither of those two are very preferable scenarios to me. What I like about my embedding of V8 right now is, I don't have to worry about those details, so my embedding is straight-forward and simple.
In other words, to put it simply, if program A can call setTimeout(), and I want to run program A and then program B, I have to be able to make sure that I don't try to run program B until everything is fully finished in A. As V8 stands now, there's no way to do anything non-synchronous, so when A finishes, I know it's totally finished. I'm concerned that there'd be some new way with setTimeout()'s that this wouldn't be true.
This is not true today with V8 in Chrome, precisely due to setTimeout!
It's not true in Node either.
I'm not sure where you think it's true. V8 embedding in something without setTimeout or anything like it, sure. But then you can't write simple time-based programs, set alarms, etc., so back in the door comes setTimeout or a workalike...
If I embed V8 in a simple C/C++ program, and I try to run a snippet of
JavaScript that calls setTimeout(...), the V8 engine complains and says
that setTimeout is undefined. Ergo, my assertion that setTimeout() is not
in core V8, but must be added to V8 by the Chrome/Chromium embedding.
As I said above, I don't currently go to that extra trouble, because for my embedding use-case, such functionality is more trouble than it's worth.
What I'd simply prefer not to be forced into is some day the core engine (by virtue of the core JS spec requiring it) of something like V8 having setTimeout() defined, and forcing the embedding to have to deal with the event-loop. Not everyone who does JS embedding needs such constructs.
Perhaps my complaint is more an engine/embedding complaint than a JS spec complaint. Perhaps what I'm saying is, I'd want for there to be an easy way for embedding with V8 (or any other engine) to be able to turn off/disable isosynchronous APIs so that the embedding wasn't forced to deal with the event-loop.
It seems to me there are a couple pieces to Mark's concurrency proposal. One part is formalizing the event queue that already exists. Is this already done in the current proposal? Because I haven't found it.
Sorry, I guess I should say, we can't add concurrency without having it be compatible with existing event queue semantics, which likely requires formalizing that in Ecma-262.
Regardless, this work of formalizing the already existing event queue has already been done by the WHATWG: www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loops
Yeah, I know.
I think it would be a mistake to start the formalization work from scratch since a decent part of the work has already been done.
Duplicating specs certainly sucks. But it seems nonsensical to me to specify things that rely on the event queue without having the event queue in the semantics. Maybe the spec would have to cross-reference WHATWG? Ugh.
Actually, wait a minute -- I think I disagree with you here. WHATWG specifies the specific event queue of the browser. Node.js has its own event queue. Others may as well. The unofficial agreement of JS has always been, no matter where you embed it, it should never have pre-emption. So what we would be specifying is the rough concurrency framework required of any JS embedding. In other words, it would be a more abstract specification of event queues, of which WHATWG event queues are a valid implementation.
I don't think there's any way to specify "run-to-completion" (which is really just a hand-wavy way of saying "no pre-emption," which is itself really just a way of saying "sequentialized event queues") without specifying event queues.
But I still don't see why standardizing timers, and only timers, is so important. Node implements them, and they work. What problem are we fixing?
Once, I read the concurrency proposal with in mind "can setTimeout be implemented with what is in this proposal?". I have found the timeout section (strawman:concurrency#timeouts) and I thought "awesome!"... until I realized that this example is itself using setTimeout (which is consistently implemented in most ECMAScript-based environments but isn't itself part of any ECMAScript version).
Why do you think setTimeout is so special? It seems like just another host-environment API like, say, onClick. It provides access to one of the host environment's system service.
I still just don't get this whole thread.
However, the tiny brick/element/component/part that is missing to implement "time event" ("clock event"?) is to take time into account in the proposal.
How is this element's absence any more important than the absence of, say, onMouseover? Granted, it has more applicability than just the browser (since node.js uses it), but as Kyle's already pointed out, some embeddings may not want to expose the system clock.
And I intuit it wouldn't be that hard to add it.
It's not exactly a walk in the park. Have you ever tried to formalize real time? Einstein had a few words to say about this subject.
As for event queue semantics, I think in principle that Ecma-262 is a reasonable place to specify it. It doesn't rely on specifics of the browser, and Node.js is a testament to that. (Besides, a non-interactive setting could probably be modeled as a trivial event queue, i.e. one with just a single "run the program" event.) I'm not sure whether this is worth trying to accomplish in time for ES.next, though. Does the fact that some event loop formalization work has already been done by the WHATWG within Standard HTML change your opinion on the matter?
Not really, no.
On Sat, Mar 19, 2011 at 5:59 AM, David Bruant <david.bruant at labri.fr> wrote:
Le 19/03/2011 07:02, David Herman a écrit :
It seems to me there are a couple pieces to Mark's concurrency proposal. One part is formalizing the event queue that already exists.
Is this already done in the current proposal? Because I haven't found it. Regardless, this work of formalizing the already existing event queue has already been done by the WHATWG: www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loops I think it would be a mistake to start the formalization work from scratch since a decent part of the work has already been done.
Another part is introducing new concepts and features to the language (like promises and vats).
I want to hear what Mark has to say at the TC39 meeting, but my gut tells me it's too early to introduce new concurrency constructs for ES.next. There's actually a lot you can do with closures and objects (such as the promises libraries people have been experimenting with lately). And with generators, you can implement some really convenient constructs. I've recently been building a concurrency library called jsTask, that makes it possible to write blocking computations in a really natural style:
blog.mozilla.com/dherman/2011/03/11/who-says-javascript-io-has-to-be-ugly, dherman.github.com/jstask
Once, I read the concurrency proposal with in mind "can setTimeout be implemented with what is in this proposal?". I have found the timeout section ( strawman:concurrency#timeouts) and I thought "awesome!"... until I realized that this example is itself using setTimeout (which is consistently implemented in most ECMAScript-based environments but isn't itself part of any ECMAScript version). ( By the way, there is a typo in this example. I think that "setTimeout(const(){...}, 0)" should be "setTimeout(const(){...}, millis)", where "millis" is the second argument. )
Good catch. Fixed. Thanks.
I fully agree that the current proposal and all related work (yours included) offer very powerful mecanisms and solutions to common problems (I had already read your article on "Who says JavaScript I/O has to be ugly?" and really enjoyed what I saw). However, the tiny brick/element/component/part that is missing to implement "time event" ("clock event"?) is to take time into account in the proposal. No matter how powerful the current proposal is, it is lacking this elementary piece in order to implement/standardize setTimeout. And I intuit it wouldn't be that hard to add it. I am not very familiar with promises but from what I understand, having the ability to create a "timed promise" which resolution would be handled by the engine could be a solution. Once again, I'm not familiar with promises, so this idea may sound stupid, but people who are familiar with promises and the Q API will certainly have better ideas on integrating the notion of time to the concept in a way that could make setTimeout and friends implementable on top of this integration.
The idea is not stupid at all. It's perfectly sound. For example, the delay example you noticed could be made primitive and setTimeout built out of that. Either can be built from the other. The advantage of layering setTimeout on a new primitive, whether delay or < nodejs.org/docs/v0.4.3/api/all.html#process.nextTick> or something
else, is that the new primitive can avoid spec weirdnesses already entrenched for setTimeout, like this 4ms clamping. Instead, this 4ms weirdness could just be specifically part of setTimeout's behavior.
Layering the other way, with setTimeout at the bottom, would mean the event loop could never run faster than 4ms per turn. This is unacceptable, and probably has been since the mid '70s. Should we clamp our fast our CPUs can execute instructions as well? That said, I think something like setTimeout with better semantics is the right lower layer abstraction. The two things I'd fix in this lower layer abstraction:
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
Then I'd layer promise-based conveniences, like delay, on that.
Le 19/03/2011 16:39, David Herman a écrit :
It seems to me there are a couple pieces to Mark's concurrency proposal. One part is formalizing the event queue that already exists. Is this already done in the current proposal? Because I haven't found it.
Sorry, I guess I should say, we can't add concurrency without having it be compatible with existing event queue semantics, which likely requires formalizing that in Ecma-262.
Regardless, this work of formalizing the already existing event queue has already been done by the WHATWG: www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loops
Yeah, I know.
I think it would be a mistake to start the formalization work from scratch since a decent part of the work has already been done.
Duplicating specs certainly sucks. But it seems nonsensical to me to specify things that rely on the event queue without having the event queue in the semantics. Maybe the spec would have to cross-reference WHATWG? Ugh.
Actually, wait a minute -- I think I disagree with you here. WHATWG specifies the specific event queue of the browser. Node.js has its own event queue. (...)
Actually, we agree. I never said that TC39 should copy/paste the event-loop section written by the WHATWG. I said that WHATWG has done some on work specifying what currently happens in browser (joining with your idea of "be[ing] compatible with existing event queue semantics"). One idea would be to see what they've done, get inspired and specify the event queue in ECMAScript. Node.js implementation would be another source of inspiration. But the WHATWG has written a spec, not an implementation which is subject to change at anytime. That's the value I was seeing in particularly studying WHATWG event queue before working on adding event queue to ECMAScript (which, as you say in your next point is compulsory in a way or another to define pre-emption-less concurrency).
I don't think there's any way to specify "run-to-completion" (which is really just a hand-wavy way of saying "no pre-emption," which is itself really just a way of saying "sequentialized event queues") without specifying event queues.
But I still don't see why standardizing timers, and only timers, is so important. Node implements them, and they work. What problem are we fixing?
Once, I read the concurrency proposal with in mind "can setTimeout be implemented with what is in this proposal?". I have found the timeout section (strawman:concurrency#timeouts) and I thought "awesome!"... until I realized that this example is itself using setTimeout (which is consistently implemented in most ECMAScript-based environments but isn't itself part of any ECMAScript version).
Why do you think setTimeout is so special? It seems like just another host-environment API like, say, onClick. It provides access to one of the host environment's system service.
I agree that it can be considered as an host-environment API. However, setTimeout seems to be widely implemented. I think it makes more sense to specify it within ECMAScript. It could be an occasion to specify what happens when you're in strict mode and you do setTimeout("alert(this)", 0). Is this string interpreted as strict eval code? There is a spec hole here. Bringing setTimeout to ECMAScript would be an occasion to answer the question and not let implementors do whatever they prefer or even to forget the question. This would avoid yet another de facto standard.
However, the tiny brick/element/component/part that is missing to implement "time event" ("clock event"?) is to take time into account in the proposal.
How is this element's absence any more important than the absence of, say, onMouseover? Granted, it has more applicability than just the browser (since node.js uses it), but as Kyle's already pointed out, some embeddings may not want to expose the system clock.
Actually, a conforming ES5 implementation implements Date.now(). In that condition, it's hard to not expose the system clock, or am I missing something?
This is giving me a (terrible) idea to implement setTimeout. We could have two vats. One asks the second to resolve some promise after a certain amout of time. The second loops and resolve the promise when the delay is passed (measured as a delta between Date.now() at request reception and in the loop test. I warned on the terrible aspect of the idea). On resolution, the function passed to Q.when in the first vat is called. Despite the inelegancy, could it work?
And I intuit it wouldn't be that hard to add it.
It's not exactly a walk in the park. Have you ever tried to formalize real time? Einstein had a few words to say about this subject. :-) I am aware of the theorical problem. However, it has never prevented people from including "time" in different languages specifications, or implementing libraries using the concept of "time". There are also people out there writing "performance benchmarks" where they "measure time"! Joke aside, there is no need to formalize real time. For Date.now(), ES5 says: "The now function return a Number value that is the time value designating the UTC date and time of the occurrence of the call to now." About setTimeout and setInterval, WHATWG says: "This API does not guarantee that timers will fire exactly on schedule. Delays due to CPU load, other tasks, etc, are to be expected."
In my opinion, these are sufficiently accurate approximations of real time so that implementors can know what to do while admitting they've given up on formalizing real time and don't want to dig any further the topic.
I still just don't get this whole thread. But I still don't see why standardizing timers, and only timers, is so important. Node implements them, and they work. What problem are we fixing?
So, to summurize a couple of things that has been said:
- With the work on strawman:concurrency, the event loop concept will be added to ECMAScript.next (regardless what is decided for setTimeout).
- Standardizing setTimeout based on the event loop is basically just adding a timing component to the event loop.
- "Real time is already in ECMAScript 5 through Date.now()"
- Standardizing setTimeout is the occasion to fully specify the string-as-a-first argument case to avoid de facto standards before strict mode or Harmony opt-in is widely deployed.
I was about to say something about security, but since Mark replied, it will be more acurate to reply to him on the topic.
Le 19/03/2011 17:58, Mark S. Miller a écrit :
On Sat, Mar 19, 2011 at 5:59 AM, David Bruant <david.bruant at labri.fr <mailto:david.bruant at labri.fr>> wrote:
I fully agree that the current proposal and all related work (yours included) offer very powerful mecanisms and solutions to common problems (I had already read your article on "Who says JavaScript I/O has to be ugly?" and really enjoyed what I saw). However, the tiny brick/element/component/part that is missing to implement "time event" ("clock event"?) is to take time into account in the proposal. No matter how powerful the current proposal is, it is lacking this elementary piece in order to implement/standardize setTimeout. And I intuit it wouldn't be that hard to add it. I am not very familiar with promises but from what I understand, having the ability to create a "timed promise" which resolution would be handled by the engine could be a solution. Once again, I'm not familiar with promises, so this idea may sound stupid, but people who are familiar with promises and the Q API will certainly have better ideas on integrating the notion of time to the concept in a way that could make setTimeout and friends implementable on top of this integration.The idea is not stupid at all. It's perfectly sound. For example, the delay example you noticed could be made primitive and setTimeout built out of that. Either can be built from the other. The advantage of layering setTimeout on a new primitive, whether delay or nodejs.org/docs/v0.4.3/api/all.html#process.nextTick or something else, is that the new primitive can avoid spec weirdnesses already entrenched for setTimeout, like this 4ms clamping. Instead, this 4ms weirdness could just be specifically part of setTimeout's behavior.
Layering the other way, with setTimeout at the bottom, would mean the event loop could never run faster than 4ms per turn. This is unacceptable, and probably has been since the mid '70s. Should we clamp our fast our CPUs can execute instructions as well? That said, I think something like setTimeout with better semantics is the right lower layer abstraction. The two things I'd fix in this lower layer abstraction:
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
This last point is something I was about to raise when starting to think about extending the Q API. setTimeout returns a number, so a malicious could loop through numbers and call clearTimeout on them. An unforgeable object sounds like the correct response to this issue. Have you considered wrapping setTimeout&friends in SES with this solution? It wouldn't break code which do not rely on the returned value being a number (but would break code which does, of course). I think there would be a need to wrapped the passed callback in order to achieve garbage collection.
I agree on the rest you've said except one thing: I'm not really sure ES should standardized the 4ms clamping. It is a reality in web browsers and maybe fair for the WHATWG to standardize it as such for web backward compatibility, but maybe that other non-web browser ES-based implementations do not follow this 4ms restriction. Any idea if there is a 4ms clamping in node.js setTimeout? Or in other ES-based implementations? If there is none, the ECMAScript spec could just leave some room to implementation-dependent behaviors.
After giving it further thought, here are the ideas I've had on adding time to the Q API: Q.when(timedPromise, success); would be a nice syntax to call a 'success' function when the promise is resolved.
To reject a timedDefered, something like: timedDeferred.resolve(Q.reject('')) could do it. But it requires providing the resolver to the timedDeferred creator. I do not know if it's a good idea to provide the resolver to user script since, in the way I see it, the resolver should exclusively be given to the engine which has the responsibility to manage time. I may be wrong. It however could be an occasion to trigger a promise resolution in advance. In Kris Kowal Q implementation (my (untouched) fork DavidBruant/q/blob/master/lib/q.js#L135), 'reject' is provided as part of the promise if I understand well. Providing the rejecter but not the resolver could be a way to solve the second use case without providing the resolver to the user script. Do people have opinions on the resolver being provided or not to the user in case of a timed defered?
Something that could be convenient in the timedPromise/Deferred construtor would be to be allowed to pass a Date object. If I want a promise to be resolved when my next birthday happens, I would prefer to write Q.timedDefer( Date(2012, 3, 8, 12) ) instead of computing milliseconds since 1/1/70. A decision would have to be made for dates in the past (actually, for negative number of milliseconds too)
On the constructor itself, maybe an optional argument could be given to Q.defer. Maybe it would be better to have a Q.timedDefer function. These are just ideas. I have no strong opinion.
These are all good points-- I may not have time to reply this weekend but I will.
I just want quickly to point out that despite the wording on strawman:strawman ("TC39's response to..."), the vats/promises strawman has by no means been agreed upon yet. As I say, I believe it's not mature enough to be likely to make the May cutoff. I don't even understand what vats are well enough to have an opinion yet.
I'll reply in more detail when I get a chance.
Thanks,
On Sat, Mar 19, 2011 at 12:17 PM, Dave Herman <dherman at mozilla.com> wrote:
These are all good points-- I may not have time to reply this weekend but I will.
I just want quickly to point out that despite the wording on strawman:strawman ("TC39's response to..."), the vats/promises strawman has by no means been agreed upon yet.
Good point. FWIW, that line was meant as informal humor, but I see how it can be misread. Fixed.
As I say, I believe it's not mature enough to be likely to make the May cutoff. I don't even understand what vats are well enough to have an opinion yet.
As a probability estimate, I agree with this as well. I consider a long shot, but a shot worth taking. In any case, being on the agenda for May, I haven't yet invested the time to clarify the text. I hope to do so in April.
On Sat, Mar 19, 2011 at 12:58 PM, Mark S. Miller <erights at google.com> wrote:
The idea is not stupid at all. It's perfectly sound. For example, the delay example you noticed could be made primitive and setTimeout built out of that. Either can be built from the other. The advantage of layering setTimeout on a new primitive, whether delay or < nodejs.org/docs/v0.4.3/api/all.html#process.nextTick> or something else, is that the new primitive can avoid spec weirdnesses already entrenched for setTimeout, like this 4ms clamping. Instead, this 4ms weirdness could just be specifically part of setTimeout's behavior.
I agree 100% - there is no requirement to make setTimeout a fundamental building blocking; it can easily be treated as a high-level API. There is also no need to make the base time measurement unit milliseconds -- it is very easy to see a future where microsecond- or even nanosecond-resolution is desirable. (Assuming, of course, we manage to get over the current GHz barrier to make our individual cores faster).
A base specification built on nanosecond-resolution timers gives us a maximal timeout of 104 days with 53-bit "integers".
That said, I have to ask: is there a reason the base specification needs to discuss time at all? Why not just a function that returns true or false? This still allows us to implement setTimeout in terms of Date, probably requires less specification, and adds more power ("is a mouse button down?" vs. "is it 4 o'clock?")
In this untested example which implements setTimeout, I present a function called enqueue, which takes as it's argument a function that returns true if the function is to be re-interested at the end of the event queue once it was evaluated:
function setTimeout(f, delay) { var when = +(new Date()) + delay; enqueue(function() { if (+(new Date()) < when) return true; if (typeof f === "string")) indirectEval(f); else f(); }); }
Of course, we could make this a little cheaper by codifying moz's Date.now(), or even having a property of the event system which is the value of the timestamp when the current iteration of the event loop began.
In the past, a timer-specific low-level interface may have been desirable from an implementation point of view. I don't believe this the case any more, given two factors -- modern JS engines are really fast at this stuff, and we are not going to be doing any kind of pre-emptive timer (so alarm()-, setitimer()-, etc-backed implementations are unnecessary).
I really think if we're looking at setTimeout() we should be looking at arbitrary condition evaluation in the event loop. The embedders will be able to fill in the blanks, and I bet most browser vendors are pretty much already there.
Sounds good -- not trying to torpedo, just clarifying. Looking forward to hearing more.
On Sat, Mar 19, 2011 at 10:57 AM, David Bruant <david.bruant at labri.fr>wrote:
Le 19/03/2011 17:58, Mark S. Miller a écrit :
[...] The two things I'd fix in this lower layer abstraction:
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
This last point is something I was about to raise when starting to think about extending the Q API. setTimeout returns a number, so a malicious could loop through numbers and call clearTimeout on them. An unforgeable object sounds like the correct response to this issue. Have you considered wrapping setTimeout&friends in SES with this solution? It wouldn't break code which do not rely on the returned value being a number (but would break code which does, of course).
Caja does exactly this. So far we haven't found any code that this actually breaks. All uses we've encountered[1] treat the return value of setTimeout/setInterval as simply something they can remember and later pass to clearTimeout/clearInterval.
[1] That I'm aware of. Caja users should speak up if they've hit a counter-example. Or anyone else that has seen code in the wild that actually counts on these values being numbers.
I actually haven't thought about this in the context of SES, and I must. Thanks for raising this.
I think there would be a need to wrapped the passed callback in order to achieve garbage collection.
I didn't understand that. Could you expand? Thanks.
I agree on the rest you've said except one thing: I'm not really sure ES should standardized the 4ms clamping. It is a reality in web browsers and maybe fair for the WHATWG to standardize it as such for web backward compatibility, but maybe that other non-web browser ES-based implementations do not follow this 4ms restriction. Any idea if there is a 4ms clamping in node.js setTimeout? Or in other ES-based implementations? If there is none, the ECMAScript spec could just leave some room to implementation-dependent behaviors.
I like this approach. Does anyone know of any problems leaving the 4ms issue as dependent on the hosting environment, and not to be standardized as part of an ES setTimeout/setInterval standard?
After giving it further thought, here are the ideas I've had on adding time to the Q API: Q.when(timedPromise, success); would be a nice syntax to call a 'success' function when the promise is resolved.
I don't understand. Given that timedPromise is the kind of thing that delay() returns, just doing a Q.when() on a timePromise as above will already do the right thing. That's why the delay() and race() abstractions compose so nicely -- because race() does a Q.when on the promise returned by delay().
I suspect I'm misunderstanding you. I'll wait for clarification on this point before proceeding with the rest.
On Sat, Mar 19, 2011 at 2:49 PM, Breton Slivka <zen at zenpsycho.com> wrote:
never write code on no sleep.
that code sample should be :
timers= (function () { var timers = []; var id=0; timer=function (f,t) { timers.push({func:f, interval:t, id:id++}); return id; } return timers; })
runScript("env.js"); runScript("program.js"); ev=getEvents(); while (!programHasQuit()) {
handleEvents(timers,ev);
}
Actually, you're right. The timer family can be abstracted like this. I don't think it would be as convenient as the current setTimeout and setInterval because it kind of requires your code to use an architecture that can work with an (explicit) event loop. But that's not necessarily a problem.
So the only thing missing from this picture is a way to yield to the host environment. In a single thread environment (like node or the browser), you can't indefinitely loop because you'll lock up the host environment. So if there was some way of yielding to the host environment, I think the problem is solved. Not saying that such a mechanism is that trivial, but I do believe that's a missing key to building timers in the language since you currently can't yield, without timers :)
And re: return value. Some scripts might rely on the fact that the timers
return a number (an int, even). So if backwards compat is important (and it
always is), you can't really break with typeof token == 'number'. So if
there was a new native type, typeof would still have to return 'number',
which is not very desirable (null anyone?...). Also, given that setTimeout
and setInterval are already implemented in browsers for a while with
returning numbers, is it really still a problem we need to solve? Will
browsers really start implementing a setTimeout that returns a special timer
object? I think any implementation that uses "setTimeout" and "setInterval"
would have to mimic the current behavior in the browser...
(Even if we implement a host-environment yield kind of thing, I'd still prefer a built-in setTimeout abstraction built on it for convience. But I don't know about the security or whatever involved.)
Kyle: If there was a way to determine which timers are currently queued, would that solve your problem? That's probably the only thing I'm missing right now from the timer api: Some array with all queued timeouts/intervals. Maybe that's to prevent the clear "attack" mentioned before, looping all numbers and clearing them. I don't know. But if you had such an array (either with just ints, or even an object with {int,time/interval,callback}) would that suffice for you? You could check the list and block while there are timers left.
On Sat, Mar 19, 2011 at 1:05 PM, Mark S. Miller <erights at google.com> wrote:
On Sat, Mar 19, 2011 at 10:57 AM, David Bruant <david.bruant at labri.fr>wrote:
Le 19/03/2011 17:58, Mark S. Miller a écrit :
[...] The two things I'd fix in this lower layer abstraction:
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
This last point is something I was about to raise when starting to think about extending the Q API. setTimeout returns a number, so a malicious could loop through numbers and call clearTimeout on them. An unforgeable object sounds like the correct response to this issue. Have you considered wrapping setTimeout&friends in SES with this solution? It wouldn't break code which do not rely on the returned value being a number (but would break code which does, of course).
Caja does exactly this. So far we haven't found any code that this actually breaks. All uses we've encountered[1] treat the return value of setTimeout/setInterval as simply something they can remember and later pass to clearTimeout/clearInterval.
[1] That I'm aware of. Caja users should speak up if they've hit a counter-example. Or anyone else that has seen code in the wild that actually counts on these values being numbers.
I actually haven't thought about this in the context of SES, and I must. Thanks for raising this.
Yes, of course SES must do so, as Caja does. I'm not even sure why I needed a moment to think about this ;).
The raises the issue of how ES-future should specify setTimeout/setInterval, such that SES remains a compatible fail-stop subset of ES. Should ES leave unspecified what the return value is, so long as it is acceptable as input to clearTimeout/clearInterval? That way, SES would be a compatible subset of ES as spec'ed, even though it would not be a subset of ES as implemented. Or could we actually get away with changing the setTimeout spec to specify an unforgeable opaque object? If possible, cleaning this up would be awesome.
On Sat, Mar 19, 2011 at 1:34 PM, Peter van der Zee <ecma at qfox.nl> wrote:
[...] And re: return value. Some scripts might rely on the fact that the timers return a number (an int, even). So if backwards compat is important (and it always is), you can't really break with
typeof token == 'number'. So if there was a new native type, typeof would still have to return 'number', which is not very desirable (null anyone?...).
I agree that if compat requires typeof token === 'number', then the token
itself must remain a number. Has anyone seen seen any such dependency in the
wild?
Also, given that setTimeout and setInterval are already implemented in browsers for a while with returning numbers, is it really still a problem we need to solve? Will browsers really start implementing a setTimeout that returns a special timer object? I think any implementation that uses "setTimeout" and "setInterval" would have to mimic the current behavior in the browser...
In Caja, setTimeout/setInterval does indeed return an object, where typeof token === 'object' and that object is accepted by Caja's
clearTimeout/clearInterval. AFAIK, we have not encountered a single
incompatibility as a result of this change. If anyone knows a
counter-example in the wild, please speak up.
And yes, this is a problem that SES needs to solve.
On 19/03/2011, at 17:58, Mark S. Miller wrote:
(...) Layering the other way, with setTimeout at the bottom, would mean the event loop could never run faster than 4ms per turn. This is unacceptable, and probably has been since the mid '70s. Should we clamp our fast our CPUs can execute instructions as well? That said, I think something like setTimeout with better semantics is the right lower layer abstraction. The two things I'd fix in this lower layer abstraction:
- No clamping. Time runs as fast as the platform lets it run.
Of course. There are some perfectly valid reasons for wanting to spin the event loop as fast as possible... and the clamp is the only thing that forbids it. But if there were no clamp, wouldn't a setTimeout( f, 0 ) be === nextTick( f ) ?
I have always wondered about the reason to clamp (nested) setTImeout()s, why, what for ?
Is it to protect the users' mobiles/laptops batteries from event loops spinning unnecessarily fast in some badly written web pages or perhaps in some popular (and possibly botched) library ?
And if that's the reason, and if the clamp is needed/wanted/kept for that reason, what are we going to do to protect against functions that nextTick() themselves ?
And if we can't protect against nested nextTick()s, then, does it make sense to keep the protection against setTimeout()s of 0 ?
Le 19/03/2011 21:05, Mark S. Miller a écrit :
On Sat, Mar 19, 2011 at 10:57 AM, David Bruant <david.bruant at labri.fr <mailto:david.bruant at labri.fr>> wrote:
I think there would be a need to wrapped the passed callback in order to achieve garbage collection.I didn't understand that. Could you expand? Thanks.
It was just an implementation detail. Actually it is easier to explain by coding it. So here it is: gist.github.com/877797 What I was talking about is at line 11: To perform garbage collection, there is a need to explicitely remove the reference we have of the identity object. Otherwise, we take the risk to keep the reference during the entire program lifetime. I would have prefered to use a WeakMap for the "timeoutIdentityObjects" variable but it would be compatible with no current ES implementation, so a plain old array will do the job. Anyway, this gist could be a start (a couple of cases aren't covered, see comments) for your SES implementation. If you want it, it's all yours :-)
After giving it further thought, here are the ideas I've had on adding time to the Q API: Q.when(timedPromise, success); would be a nice syntax to call a 'success' function when the promise is resolved.I don't understand. Given that timedPromise is the kind of thing that delay() returns
Oooh. That wasn't the assumption I was making; I hadn't fully understood the delay function. But after re-reading the delay function, then yes, that's an excellent idea.
The only thing that I said afterward and that matter now is this (corrected): Something that could be convenient as the first argument of delay (Q.delay?) would be to be allow a Date object. If I want a delayed promise to be resolved when my next birthday happens, I would prefer to write Q.delay( new Date(2012, 3, 8, 12) ) instead of computing milliseconds since 1/1/70. A decision would have to be made for dates in the past (actually, for negative number of milliseconds (or the chosen time unit) too). Solving them right away? Rejecting them right away?
, just doing a Q.when() on a timePromise as above will already do the right thing. That's why the delay() and race() abstractions compose so nicely -- because race() does a Q.when on the promise returned by delay().
Things are getting more and more clear on promises. Now that I understand the line: var answer = race(bob ! foo(carol), delay(5000, Q.reject(new Error("timeout")))); I find it extremely elegant, indeed.
On Mar 19, 2011, at 3:16 AM, Peter van der Zee wrote:
On Sat, Mar 19, 2011 at 1:13 AM, Brendan Eich <brendan at mozilla.com> wrote: On Mar 18, 2011, at 5:54 AM, Peter van der Zee wrote:
+1 to standardizing the timer family.
I always thought this wasn't in because the specification didn't have any asynchronism and specifying timers would open Pandora's box.
How so? I created JS and its run-to-completion execution model and setTimeout all at once in 1995. It has not changed in execution-model semantics since.
I can't think of a single way to simulate setTimeout in ES5. Correct me if I'm wrong, but I don't think ES5 exposes a single way of defining a mechanism like:
-- var x = 4; function f(){ x = 5; print(x); } timer(f, 1); print(f);
Such that it would output 4 before 5. That's what I meant with "didn't have any asynchronism", fwiw.
That's not the same as "open[ing] Pandora's box".
Let's not use asynchronism loosely. "Asynchronous I/O" can mean threads and completion ports. It means shared-memory threads and non-determinism in general.
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already).
The issues with setTimeout have more been about "core langage" vs. "DOM level 0", and the lack of a "DOM level 0" standard prior to HTML5.
What do you mean by that? Why should that influence the choice of including it in the ECMA spec? What about embedded environments or ssjs?
Exactly my point: the historical accidents of what got spec'ed in which standards body, or not spec'ed at all for years, should not overrule other criteria now. If we want to spec setTimeout and event loopage in Ecma TC39, we can consider it on its merits.
On Mar 19, 2011, at 7:57 AM, Kyle Simpson wrote:
What I was saying is, if I run this program through V8 right now (with its theoretical future support of setTimeout() included), then what will happen:
function fn() { print("hello"); } for (var i=0; i<10; i++) { setTimeout(fn,i*1000); }
That for-loop will finish very quickly (probably <1 ms). Would V8 (or any other JS engine) "finish" in the sense that the calling embedding code thinks this program is completely finished, and it returns back control to the C/C++ embedding layer when:
a) right after the for-loop finishes; OR b) after only the first call to
fn, since it's timeout was effectively 0, and so would have been immediately after the main program finished; OR c) after all of the queued up calls tofnhave finished, about 9 seconds later?(a) is the right answer because the event loop is run by the embedding, not v8. IMnsHO.
OK, so what I was asserting in my earlier email is, I'd prefer that if setTimeout() were going to be added to the official language spec, and in fact directly implemented in the engines, that instead of making the embedder's job harder (ie, me), that the engine entirely encapsulate and manage the event loop.
You can encapsulate an event loop within a JS engine, but the API will need to allow for native GUI events, poll on sockets and other files, etc. to be embeddable in a browser or something like nodejs.
This is partly an API discussion, probably off topic for es-discuss. It's just a matter of how big a lump you want to make "on the inside", and how much API that lump needs to meet the demands of its consumers. But you might have to build it yourself.
IOW, this is not the place to demand that a "JS engine" encapsulate an event loop. It's not obviously better for all engines to have OS-specific event loop code, and all the API required (e.g., for X on Linux), than it is to leave that to what ECMA-262 calls "the host environment". But it doesn't really matter here, except that it changes the requirements on embedders.
You're right that standardizing setTimeout would change requirements for embedders of current engines. That's part of the trade-off. No free lunch, but possibly worth it for the core language to grow to include setTimeout.
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already).
`setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If compute' takes a long time, thenf1' runs before f2', otherwisef2' runs first.
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
This last point is something I was about to raise when starting to think about extending the Q API. setTimeout returns a number, so a malicious could loop through numbers and call clearTimeout on them. An unforgeable object sounds like the correct response to this issue. Have you considered wrapping setTimeout&friends in SES with this solution? It wouldn't break code which do not rely on the returned value being a number (but would break code which does, of course).
Caja does exactly this. So far we haven't found any code that this actually breaks. All uses we've encountered[1] treat the return value of setTimeout/setInterval as simply something they can remember and later pass to clearTimeout/clearInterval.
[1] That I'm aware of. Caja users should speak up if they've hit a counter-example. Or anyone else that has seen code in the wild that actually counts on these values being numbers.
I have a number of different projects where I've used timers (mostly intervals rather than timeouts), where the logic that I used relied on being able to tell if a value was falsy or not, to know if there's a timer attached to some variable (so that you only set the interval once, and not multiple times).
So, for instance, a variable like foo = false, when an interval is set,foo = setInterval(...), and when that interval is cleared, I not only callclearInterval(foo)but I also callfoo = falseagain to signal that the interval has been cleared (there is nocheckInterval(foo)API to check this). Then, the next time I want to set the interval, I first check to see iffoo` is truthy or falsy, and only set if it's indeed falsy. So, all this
is to say, I rely in those tests on whether the value is truthy or falsy.
Not exactly relying on its data type, but it's important not to assume that
noone ever checks those values -- in many cases, I do.
Kyle: If there was a way to determine which timers are currently queued, would that solve your problem? That's probably the only thing I'm missing right now from the timer api: Some array with all queued timeouts/intervals. Maybe that's to prevent the clear "attack" mentioned before, looping all numbers and clearing them. I don't know. But if you had such an array (either with just ints, or even an object with {int,time/interval,callback}) would that suffice for you? You could check the list and block while there are timers left.
Well, my point was, I'd like the embedding layer not to have to implement that kind of logic if there's a way to avoid it. But yes, in terms of least-effort, that sounds like the simplest approach to essentially blocking on the event loop queue to wait for it to complete. However, we'd have to consider all such possible "events", not just timers... XHR comes to mind. "server-sent events" comes to mind (aka, server-to-server events). etc.
On Sat, Mar 19, 2011 at 5:21 PM, Kyle Simpson <getify at gmail.com> wrote:
- No clamping. Time runs as fast as the platform lets it run.
- The return value is not an integer but a unique unforgeable object for canceling the event. No one without that object can cancel that event.
This last point is something I was about to raise when starting to think about extending the Q API. setTimeout returns a number, so a malicious could loop through numbers and call clearTimeout on them. An unforgeable object sounds like the correct response to this issue. Have you considered wrapping setTimeout&friends in SES with this solution? It wouldn't break code which do not rely on the returned value being a number (but would break code which does, of course).
Caja does exactly this. So far we haven't found any code that this actually breaks. All uses we've encountered[1] treat the return value of setTimeout/setInterval as simply something they can remember and later pass to clearTimeout/clearInterval.
[1] That I'm aware of. Caja users should speak up if they've hit a counter-example. Or anyone else that has seen code in the wild that actually counts on these values being numbers.
I have a number of different projects where I've used timers (mostly intervals rather than timeouts), where the logic that I used relied on being able to tell if a value was falsy or not, to know if there's a timer attached to some variable (so that you only set the interval once, and not multiple times).
So, for instance, a variable like foo = false
, when an interval is set,foo = setInterval(...), and when that interval is cleared, I not only callclearInterval(foo)but I also callfoo = falseagain to signal that the interval has been cleared (there is nocheckInterval(foo)API to check this). Then, the next time I want to set the interval, I first check to see iffoo` is truthy or falsy, and only set if it's indeed falsy. So, all this is to say, I rely in those tests on whether the value is truthy or falsy. Not exactly relying on its data type, but it's important not to assume that noone ever checks those values -- in many cases, I do.
Hi Kyle, you're correct that this example is sufficiently close to be worrisome. It is also at least amusing to note that switching to opaque token objects would not break such code, since these tokens are truthy.
On Mar 19, 2011, at 4:20 PM, Sam Tobin-Hochstadt wrote:
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already).
`setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If
compute' takes a long time, thenf1' runs beforef2', otherwisef2' runs first.
Isn't the non-determinism entirely in compute(), so that if you replace the two setTimeout calls with Date.now calls, saving results, you can measure the ND that either does or does not reorder f1 with respect to f2?
My point was that setTimeout, unlike threads racing over shared mutable state, does not introduce new sources of ND not already in the current execution model.
We need to be careful about order, though. If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this.
On 20/03/2011, at 04:11, Brendan Eich wrote:
We need to be careful about order, though.
Of course, yes, please.
If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this.
The only place that I know where this
setTimeout( f, 1); setTimeout( g, 2);
will eventually fire g() before f() is nodejs: joyent/node#604
I've never seen that in any browser.
Le 20/03/2011 04:11, Brendan Eich a écrit :
On Mar 19, 2011, at 4:20 PM, Sam Tobin-Hochstadt wrote:
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already). `setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If
compute' takes a long time, thenf1' runs beforef2', otherwisef2' runs first. Isn't the non-determinism entirely in compute(), so that if you replace the two setTimeout calls with Date.now calls, saving results, you can measure the ND that either does or does not reorder f1 with respect to f2?My point was that setTimeout, unlike threads racing over shared mutable state, does not introduce new sources of ND not already in the current execution model.
We need to be careful about order, though. If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this.
I'm sorry, but even though I agree on the principle, I do not know why "f1 /should/ be called first". With names:
setTimeout(f1, delay1); computea(); setTimeout(f2, delay2); computeb(); // to enforce the idea that the program doesn't stop right away
At the end, f1 is scheduled at t1, f2 at t2. (t1<t and t2<t with t present time). We have two timeouts ready to be fired, there is a decision that has to be made on the firing order. I would personnally choose yours, which is to call the function that has been the most delayed (max(t - t_i)). However, other decision policies could be valid. For instance, choosing max( (t-t_i)/delay_i ) could be a valid policy too. The rationale would be to keep at a minimum not the absolute delay, but the relative delay. Another thing that could be taken into account is heuristics on the time that f1 and f2 take to execute. Mixing all these things together, there are certainly other valid policies that could be considered.
Once again, my personal preference, the policy I find the most fair, is reducing absolute delay as you suggest, but is there a reason why this /should/ be the behavior accross browsers? Should even the behavior be consistent across browser? Across hardware, ES engine implementation and external factors (OS scheduling with other programs), computea and computeb can make t1 and t2 relative order different from one run to another. So there is no way to test the consistency.
Is there a reason why you're expecting browsers 1) to be consistent in their policies 2) to choose a particular policy?
On 20/03/2011, at 12:08, David Bruant wrote:
Le 20/03/2011 04:11, Brendan Eich a écrit :
On Mar 19, 2011, at 4:20 PM, Sam Tobin-Hochstadt wrote:
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already). `setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If
compute' takes a long time, thenf1' runs beforef2', otherwisef2' runs first. Isn't the non-determinism entirely in compute(), so that if you replace the two setTimeout calls with Date.now calls, saving results, you can measure the ND that either does or does not reorder f1 with respect to f2?My point was that setTimeout, unlike threads racing over shared mutable state, does not introduce new sources of ND not already in the current execution model.
We need to be careful about order, though. If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this. I'm sorry, but even though I agree on the principle, I do not know why "f1 /should/ be called first". With names:
setTimeout(f1, delay1); computea(); setTimeout(f2, delay2); computeb(); // to enforce the idea that the program doesn't stop right away
At the end, f1 is scheduled at t1, f2 at t2. (t1<t and t2<t with t present time). We have two timeouts ready to be fired, there is a decision that has to be made on the firing order. I would personnally choose yours, which is to call the function that has been the most delayed (max(t - t_i)). However, other decision policies could be valid. For instance, choosing max( (t-t_i)/delay_i ) could be a valid policy too. The rationale would be to keep at a minimum not the absolute delay, but the relative delay. Another thing that could be taken into account is heuristics on the time that f1 and f2 take to execute. Mixing all these things together, there are certainly other valid policies that could be considered.
Once again, my personal preference, the policy I find the most fair, is reducing absolute delay as you suggest, but is there a reason why this /should/ be the behavior accross browsers? Should even the behavior be consistent across browser? Across hardware, ES engine implementation and external factors (OS scheduling with other programs), computea and computeb can make t1 and t2 relative order different from one run to another. So there is no way to test the consistency.
Is there a reason why you're expecting browsers 1) to be consistent in their policies 2) to choose a particular policy?
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
On Sun, Mar 20, 2011 at 6:03 AM, Jorge <jorge at jorgechamorro.com> wrote:
will eventually fire g() before f() is nodejs: < joyent/node#604> I've never seen that in any browser.
This sounds like a bug in Node's clamping algorithm.
On 20/03/2011, at 13:51, Jorge wrote:
On 20/03/2011, at 12:08, David Bruant wrote:
Le 20/03/2011 04:11, Brendan Eich a écrit :
On Mar 19, 2011, at 4:20 PM, Sam Tobin-Hochstadt wrote:
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already). `setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If
compute' takes a long time, thenf1' runs beforef2', otherwisef2' runs first. Isn't the non-determinism entirely in compute(), so that if you replace the two setTimeout calls with Date.now calls, saving results, you can measure the ND that either does or does not reorder f1 with respect to f2?My point was that setTimeout, unlike threads racing over shared mutable state, does not introduce new sources of ND not already in the current execution model.
We need to be careful about order, though. If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this. I'm sorry, but even though I agree on the principle, I do not know why "f1 /should/ be called first". With names:
setTimeout(f1, delay1); computea(); setTimeout(f2, delay2); computeb(); // to enforce the idea that the program doesn't stop right away
At the end, f1 is scheduled at t1, f2 at t2. (t1<t and t2<t with t present time). We have two timeouts ready to be fired, there is a decision that has to be made on the firing order. I would personnally choose yours, which is to call the function that has been the most delayed (max(t - t_i)). However, other decision policies could be valid. For instance, choosing max( (t-t_i)/delay_i ) could be a valid policy too. The rationale would be to keep at a minimum not the absolute delay, but the relative delay. Another thing that could be taken into account is heuristics on the time that f1 and f2 take to execute. Mixing all these things together, there are certainly other valid policies that could be considered.
Once again, my personal preference, the policy I find the most fair, is reducing absolute delay as you suggest, but is there a reason why this /should/ be the behavior accross browsers? Should even the behavior be consistent across browser? Across hardware, ES engine implementation and external factors (OS scheduling with other programs), computea and computeb can make t1 and t2 relative order different from one run to another. So there is no way to test the consistency.
Is there a reason why you're expecting browsers 1) to be consistent in their policies 2) to choose a particular policy?
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
->
f1() f2() f3() f4() f5()
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
On 20/03/2011, at 13:51, Jorge wrote:
On 20/03/2011, at 12:08, David Bruant wrote:
Le 20/03/2011 04:11, Brendan Eich a écrit :
On Mar 19, 2011, at 4:20 PM, Sam Tobin-Hochstadt wrote:
On Sat, Mar 19, 2011 at 6:50 PM, Brendan Eich <brendan at mozilla.com> wrote:
setTimeout does not introduce threads, shared memory among them, or even non-determinism if we spec carefully (browsers have to implement carefully already). `setTimeout' already introduces nondeterminism based on the speed of the implementation. Consider:
setTimeout(f1,1000) compute() setTimeout(f2,0)
If
compute' takes a long time, thenf1' runs beforef2', otherwisef2' runs first. Isn't the non-determinism entirely in compute(), so that if you replace the two setTimeout calls with Date.now calls, saving results, you can measure the ND that either does or does not reorder f1 with respect to f2?My point was that setTimeout, unlike threads racing over shared mutable state, does not introduce new sources of ND not already in the current execution model.
We need to be careful about order, though. If setTimeout(f1, 1000) schedules f1 to be called at t1, and setTimeout(f2, 0) schedules f2 at t2, then if t2 >= t1, f1 should be called first. I'm not sure all browsers implement this. I'm sorry, but even though I agree on the principle, I do not know why "f1 /should/ be called first". With names:
setTimeout(f1, delay1); computea(); setTimeout(f2, delay2); computeb(); // to enforce the idea that the program doesn't stop right away
At the end, f1 is scheduled at t1, f2 at t2. (t1<t and t2<t with t present time). We have two timeouts ready to be fired, there is a decision that has to be made on the firing order. I would personnally choose yours, which is to call the function that has been the most delayed (max(t - t_i)). However, other decision policies could be valid. For instance, choosing max( (t-t_i)/delay_i ) could be a valid policy too. The rationale would be to keep at a minimum not the absolute delay, but the relative delay. Another thing that could be taken into account is heuristics on the time that f1 and f2 take to execute. Mixing all these things together, there are certainly other valid policies that could be considered.
Once again, my personal preference, the policy I find the most fair, is reducing absolute delay as you suggest, but is there a reason why this /should/ be the behavior accross browsers? Should even the behavior be consistent across browser? Across hardware, ES engine implementation and external factors (OS scheduling with other programs), computea and computeb can make t1 and t2 relative order different from one run to another. So there is no way to test the consistency.
Is there a reason why you're expecting browsers 1) to be consistent in their policies 2) to choose a particular policy? A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything. Once again, I agree with the policy, but there is no way to enforce it within the spec. There is no way to write an ECMAScript test to be added to test262 to tell which implementation is valid and which is not. For anything tied to real time, it's impossible. The best test that could be written for Date.now() is to make sure that the returned value is bigger than the date when the test has been written. Or maybe, making sure that two sequential calls verify that the time goes forward. That's the best we can do. I do not really understand how you can "expect" things if you can't verify if your expectation is met or not.
In low-memory environment, if a program does an intensive use of timers, maybe that heuristics on the time the timer firing will take could be used to try to get rid of a maximum of timers as soon as possible to save up memory. I would consider this policy as valid due to environment constraints.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
-> f1() f2() f3() f4() f5()
I think it is very dangerous to use talk about things like t0 === t1 and Date.now() >= t0 (concerned about the "or equal"). If ECMAScript has
millisecond as granularity, most systems have microseconds if not nanoseconds. 1ms = 10^(6)ns. That's a lot! It leaves a lot of room for interpretation. And even the setTimeout calls aren't instantaneous.
Consider (milliseconds in comment): // 0 setTimeout( f1, 1 ); setTimeout( f2, 0 ); // 1 (millisecond change in the middle of the call) // 2 // -- decision to make on which timer to fire. Since the millisecond change happened during the second setTimeout, when is scheduled this second timeout? We can arbitrarly choose one, but I do not know why we would do so. Time is not discrete and especially not milliseconds. The most accurate thing we could discretize and base timing on is clock ticks. But this is not a "real time" unit and there is no absolute relationship between clock ticks and real time. How accurate was Intel when they sold me a "2GHz processor"? It certainly changes throughout time and with external conditions (temperature...) too.
As pointed out by Dave Herman, real time is difficult to take into account. Having strong expectations on real time ("I want this timer to fire before this one under such real-time-dependent conditions") within a programming language doesn't like a good idea.
On 20/03/2011, at 14:00, Wes Garland wrote:
On Sun, Mar 20, 2011 at 6:03 AM, Jorge <jorge at jorgechamorro.com> wrote: will eventually fire g() before f() is nodejs: joyent/node#604 I've never seen that in any browser.
This sounds like a bug in Node's clamping algorithm.
It's not, there's an -unwanted- 1ms clamp, but that's not the problem. It's a limitation of the optimizations in place in the implementation of setTimeout()/setInterval(). To guarantee the correct order would require a less performant implementation.
On 20/03/2011, at 15:18, David Bruant wrote:
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything.
Why ? When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms". (Let's leave aside for now the problems wrt Date.now() / wall clock time).
Once again, I agree with the policy, but there is no way to enforce it within the spec. There is no way to write an ECMAScript test to be added to test262 to tell which implementation is valid and which is not. For anything tied to real time, it's impossible. The best test that could be written for Date.now() is to make sure that the returned value is bigger than the date when the test has been written. Or maybe, making sure that two sequential calls verify that the time goes forward. That's the best we can do. I do not really understand how you can "expect" things if you can't verify if your expectation is met or not.
I don't see why "you can't verify your expectation".
In low-memory environment, if a program does an intensive use of timers, maybe that heuristics on the time the timer firing will take could be used to try to get rid of a maximum of timers as soon as possible to save up memory. I would consider this policy as valid due to environment constraints.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
-> f1() f2() f3() f4() f5()
I think it is very dangerous to use talk about things like t0 === t1 and Date.now() >= t0 (concerned about the "or equal"). If ECMAScript has millisecond as granularity, most systems have microseconds if not nanoseconds. 1ms = 10⁶ns. That's a lot! It leaves a lot of room for interpretation. And even the setTimeout calls aren't instantaneous.
Consider (milliseconds in comment): // 0 setTimeout( f1, 1 ); setTimeout( f2, 0 ); // 1 (millisecond change in the middle of the call) // 2 // -- decision to make on which timer to fire. Since the millisecond change happened during the second setTimeout, when is scheduled this second timeout?
A setTimeout( f, ms ) should be scheduled to fire at t = Date.now() + ms, so in the example above, both would be scheduled to fire at the same t, but, as f1 was setup before f2, f1 should be called before f2. You can achieve this easily with a stack per target time.
I said that WHATWG has done some on work specifying what currently happens in browser (joining with your idea of "be[ing] compatible with existing event queue semantics"). One idea would be to see what they've done, get inspired and specify the event queue in ECMAScript.
OK. I'm on board for that. I'm not committing to ES.next, though; IMO, new features are more important than specifying things that are only de facto standards but generally working pretty well.
Why do you think setTimeout is so special? It seems like just another host-environment API like, say, onClick. It provides access to one of the host environment's system service. I agree that it can be considered as an host-environment API. However, setTimeout seems to be widely implemented. I think it makes more sense to specify it within ECMAScript.
I suppose so.
It could be an occasion to specify what happens when you're in strict mode and you do setTimeout("alert(this)", 0). Is this string interpreted as strict eval code? There is a spec hole here. Bringing setTimeout to ECMAScript would be an occasion to answer the question and not let implementors do whatever they prefer or even to forget the question. This would avoid yet another de facto standard.
That question is orthogonal.
It's only one instance of a larger question, which is what happens when a host environment is asked to trigger JS code and it's not obvious which mode to use. For example, <a onclick="alert(this)">click me</a>. This does not need to be, and shouldn't be, a problem solved exclusively for setTimeout.
Instead, we will have to deal with this -- as always -- in a way that masks out specifics of the browser, but deals with the notion of "legacy mode" evaluation. This is at least in spirit what the purpose of the versioning strawman is for. There's lots of work left to do there.
How is this element's absence any more important than the absence of, say, onMouseover? Granted, it has more applicability than just the browser (since node.js uses it), but as Kyle's already pointed out, some embeddings may not want to expose the system clock. Actually, a conforming ES5 implementation implements Date.now(). In that condition, it's hard to not expose the system clock, or am I missing something?
Sorry, you're absolutely right there. I was completely overlooking Date.now().
Now, an implementor may not want to have to implement an event queue. I won't try to speak for implementors. But setTimeout introduces a different overall program control flow model than the old start-compute-exit model. Most JS embeddings use the interactive, event queue-based model, but it sounds like at least Kyle's doesn't.
But that alone should probably not stop us from moving ahead with concurrency. If an engine wants to provide a sequential JS, they can probably just do so and say they're conformant with everything except for the concurrency parts. Or maybe we can highlight the concurrency parts of the spec and say a sequential JS doesn't have to implement them. This probably isn't too important.
This is giving me a (terrible) idea to implement setTimeout. We could have two vats. One asks the second to resolve some promise after a certain amout of time. The second loops and resolve the promise when the delay is passed (measured as a delta between Date.now() at request reception and in the loop test. I warned on the terrible aspect of the idea). On resolution, the function passed to Q.when in the first vat is called.
You don't need vats for this, just a top-level driver loop.
Despite the inelegancy, could it work?
No way -- polling isn't just inelegant, it starves your processor. You'd at least have to expose some sort of sleep() command, no?
It's not exactly a walk in the park. Have you ever tried to formalize real time? Einstein had a few words to say about this subject. :-) I am aware of the theorical problem. However, it has never prevented people from including "time" in different languages specifications, or implementing libraries using the concept of "time". There are also people out there writing "performance benchmarks" where they "measure time"! Joke aside, there is no need to formalize real time.
Fair enough. Sorry if I was a little hyperbolic.
For Date.now(), ES5 says: "The now function return a Number value that is the time value designating the UTC date and time of the occurrence of the call to now." About setTimeout and setInterval, WHATWG says: "This API does not guarantee that timers will fire exactly on schedule. Delays due to CPU load, other tasks, etc, are to be expected."
In my opinion, these are sufficiently accurate approximations of real time so that implementors can know what to do while admitting they've given up on formalizing real time and don't want to dig any further the topic.
That sounds reasonable. I'll admit to not knowing enough of the complexities of this subject to really understand if there are any perils of underspecification.
So, to summurize a couple of things that has been said:
- With the work on strawman:concurrency, the event loop concept will be added to ECMAScript.next (regardless what is decided for setTimeout).
No. That's part of my point. We may not be able to standardize the event loop for ES.next. As I say, I'm open to doing it at some point, and if we add concurrency features we'll have to specify the event queue as well, but I think there's a high probability that we won't get to this in time for ES.next. We have a lot on our plate.
- Standardizing setTimeout based on the event loop is basically just adding a timing component to the event loop.
Sure.
- "Real time is already in ECMAScript 5 through Date.now()"
Right you are.
- Standardizing setTimeout is the occasion to fully specify the string-as-a-first argument case to avoid de facto standards before strict mode or Harmony opt-in is widely deployed.
I think that's not really relevant. That's an instance of a wider problem, as I said above.
Le 20/03/2011 16:04, Jorge a écrit :
On 20/03/2011, at 15:18, David Bruant wrote:
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything. Why ? When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms".(Let's leave aside for now the problems wrt Date.now() / wall clock time).
I agree that for one timer, there is no problem. Tricky case is if you want to enforce a policy on how to deal with several delayed timers (let's call it "scheduling policy"). That's where I say that it's not possible to pretend expecting anything.
In low-memory environment, if a program does an intensive use of timers, maybe that heuristics on the time the timer firing will take could be used to try to get rid of a maximum of timers as soon as possible to save up memory. I would consider this policy as valid due to environment constraints.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
-> f1() f2() f3() f4() f5()
I think it is very dangerous to use talk about things like t0 === t1 and Date.now() >= t0 (concerned about the "or equal"). If ECMAScript has millisecond as granularity, most systems have microseconds if not nanoseconds. 1ms = 10⁶ns. That's a lot! It leaves a lot of room for interpretation. And even the setTimeout calls aren't instantaneous.
Consider (milliseconds in comment): // 0 setTimeout( f1, 1 ); setTimeout( f2, 0 ); // 1 (millisecond change in the middle of the call) // 2 // -- decision to make on which timer to fire. Since the millisecond change happened during the second setTimeout, when is scheduled this second timeout? A setTimeout( f, ms ) should be scheduled to fire at t = Date.now() + ms,so in the example above, both would be scheduled to fire at the same t, "both would be scheduled to fire the same t". Who said that? The millisecond change is in the middle of the setTimeout call. From the external point of view, the second timer is scheduled to be called between two milliseconds and you (external point of view, not implementor) have no idea which one.
but, as f1 was setup before f2, f1 should be called before f2. You can achieve this easily with a stack per target time.
Your argument is based on the implementor point of view. I agree that any policy can be implemented. I do not doubt that. I am just saying that you cannot verify it from a script point of view, (and so there is no point specifying it).
I don't see why "you can't verify your expectation".
If you think you can verify your expectation, please write ECMAScript interoperable test cases that show how to test whether an ECMAScript engine is conform to your scheduling policy or not. It will be enough to convince me. Testing one timer ("When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms" ") will not be difficult.
Testing your scheduling policy is a different story.
A spec is a contract between an provider (implementor) and a client (ECMAScript script writer). If the client has no way to ensure that a spec detail has been met, there is no point adding this detail to the contract. With maybe the exception of Date.now() and Math.random() which are convinient but comes with no guarantee whatsoever. Their spec are empty (not literally, but close). While you seem to have "expectations" on a scheduling algorithm.
If I am missing something and you can write test cases to enforce expectation over a particular scheduling policy, please do so.
I don't see why "you can't verify your expectation". If you think you can verify your expectation, please write ECMAScript interoperable test cases that show how to test whether an ECMAScript engine is conform to your scheduling policy or not. It will be enough to convince me. Testing one timer ("When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms" ") will not be difficult. Testing your scheduling policy is a different story.
Forgive my naivety, but would it not be possible to test such a scheduling policy with something like this:
var test = 1;
function a() { test *= -1; } function b() { assertEquals(test,-1); }
setTimeout(a,0); for(i=0;i<1E10;i++){i=i;} // or any other sufficiently long running algorithm setTimeout(b,0);
Le 20/03/2011 16:56, David Herman a écrit :
This is giving me a (terrible) idea to implement setTimeout. We could have two vats. One asks the second to resolve some promise after a certain amout of time. The second loops and resolve the promise when the delay is passed (measured as a delta between Date.now() at request reception and in the loop test. I warned on the terrible aspect of the idea). On resolution, the function passed to Q.when in the first vat is called. You don't need vats for this, just a top-level driver loop.
Despite the inelegancy, could it work? No way -- polling isn't just inelegant, it starves your processor. You'd at least have to expose some sort of sleep() command, no?
Starving the processor is what I called "inelegant" :-) I was trying to give an example of a setTimeout implemented with Vats+pure ES5 (no sleep). Otherwise, be sure that I would do differently. Sleep is one idea. Having another passive-wait primitive like the delay() example of the concurrency strawman is another idea.
It's not exactly a walk in the park. Have you ever tried to formalize real time? Einstein had a few words to say about this subject. :-) I am aware of the theorical problem. However, it has never prevented people from including "time" in different languages specifications, or implementing libraries using the concept of "time". There are also people out there writing "performance benchmarks" where they "measure time"! Joke aside, there is no need to formalize real time. Fair enough. Sorry if I was a little hyperbolic.
No worries.
(moved) Now, an implementor may not want to have to implement an event queue. I won't try to speak for implementors. But setTimeout introduces a different overall program control flow model than the old start-compute-exit model. Most JS embeddings use the interactive, event queue-based model, but it sounds like at least Kyle's doesn't.
But that alone should probably not stop us from moving ahead with concurrency. If an engine wants to provide a sequential JS, they can probably just do so and say they're conformant with everything except for the concurrency parts. Or maybe we can highlight the concurrency parts of the spec and say a sequential JS doesn't have to implement them. This probably isn't too important.
So, to summurize a couple of things that has been said:
- With the work on strawman:concurrency, the event loop concept will be added to ECMAScript.next (regardless what is decided for setTimeout). No. That's part of my point. We may not be able to standardize the event loop for ES.next. As I say, I'm open to doing it at some point, and if we add concurrency features we'll have to specify the event queue as well, but I think there's a high probability that we won't get to this in time for ES.next. We have a lot on our plate.
Actually, this is the part I was missing. My initial e-mail was based on the assumption that an event loop will be part of ECMAScript.next through Promises. And my point was that if it's the case, adding time to it and implementing setTimeout on top of that will not be hard. But if this is not happening, then my idea comes too early, indeed.
On Sun, Mar 20, 2011 at 9:17 AM, David Bruant <david.bruant at labri.fr> wrote:
Le 20/03/2011 16:56, David Herman a écrit :
This is giving me a (terrible) idea to implement setTimeout. We could have two vats. One asks the second to resolve some promise after a certain amout of time. The second loops and resolve the promise when the delay is passed (measured as a delta between Date.now() at request reception and in the loop test. I warned on the terrible aspect of the idea). On resolution, the function passed to Q.when in the first vat is called. You don't need vats for this, just a top-level driver loop.
Despite the inelegancy, could it work? No way -- polling isn't just inelegant, it starves your processor. You'd at least have to expose some sort of sleep() command, no? Starving the processor is what I called "inelegant" :-) I was trying to give an example of a setTimeout implemented with Vats+pure ES5 (no sleep). Otherwise, be sure that I would do differently. Sleep is one idea. Having another passive-wait primitive like the delay() example of the concurrency strawman is another idea.
It's not exactly a walk in the park. Have you ever tried to formalize real time? Einstein had a few words to say about this subject. :-) I am aware of the theorical problem. However, it has never prevented people from including "time" in different languages specifications, or implementing libraries using the concept of "time". There are also people out there writing "performance benchmarks" where they "measure time"!
Joke aside, there is no need to formalize real time. Fair enough. Sorry if I was a little hyperbolic. No worries.
(moved) Now, an implementor may not want to have to implement an event queue. I won't try to speak for implementors. But setTimeout introduces a different overall program control flow model than the old start-compute-exit model. Most JS embeddings use the interactive, event queue-based model, but it sounds like at least Kyle's doesn't.
But that alone should probably not stop us from moving ahead with concurrency. If an engine wants to provide a sequential JS, they can probably just do so and say they're conformant with everything except for the concurrency parts. Or maybe we can highlight the concurrency parts of the spec and say a sequential JS doesn't have to implement them. This probably isn't too important.
So, to summurize a couple of things that has been said:
- With the work on strawman:concurrency, the event loop concept will be added to ECMAScript.next (regardless what is decided for setTimeout). No. That's part of my point. We may not be able to standardize the event loop for ES.next. As I say, I'm open to doing it at some point, and if we add concurrency features we'll have to specify the event queue as well, but I think there's a high probability that we won't get to this in time for ES.next. We have a lot on our plate.
Actually, this is the part I was missing. My initial e-mail was based on the assumption that an event loop will be part of ECMAScript.next through Promises. And my point was that if it's the case, adding time to it and implementing setTimeout on top of that will not be hard. But if this is not happening, then my idea comes too early, indeed.
I agree with Dave that it's unlikely, but I'm gonna give it a shot anyway. If I miss, as is likely, then I will make a serious try to get it into ES-after-next (this naming convention is starting to pinch ;)). So all this discussion remains relevant at least for the long term. Don't worry about "too early".
On 20/03/2011, at 16:57, David Bruant wrote:
Le 20/03/2011 16:04, Jorge a écrit :
On 20/03/2011, at 15:18, David Bruant wrote:
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything. Why ? When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms".(Let's leave aside for now the problems wrt Date.now() / wall clock time). I agree that for one timer, there is no problem. Tricky case is if you want to enforce a policy on how to deal with several delayed timers (let's call it "scheduling policy"). That's where I say that it's not possible to pretend expecting anything.
In low-memory environment, if a program does an intensive use of timers, maybe that heuristics on the time the timer firing will take could be used to try to get rid of a maximum of timers as soon as possible to save up memory. I would consider this policy as valid due to environment constraints.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
-> f1() f2() f3() f4() f5()
I think it is very dangerous to use talk about things like t0 === t1 and Date.now() >= t0 (concerned about the "or equal"). If ECMAScript has millisecond as granularity, most systems have microseconds if not nanoseconds. 1ms = 10⁶ns. That's a lot! It leaves a lot of room for interpretation. And even the setTimeout calls aren't instantaneous.
Consider (milliseconds in comment): // 0 setTimeout( f1, 1 ); setTimeout( f2, 0 ); // 1 (millisecond change in the middle of the call) // 2 // -- decision to make on which timer to fire. Since the millisecond change happened during the second setTimeout, when is scheduled this second timeout? A setTimeout( f, ms ) should be scheduled to fire at t = Date.now() + ms,so in the example above, both would be scheduled to fire at the same t, "both would be scheduled to fire the same t". Who said that? The millisecond change is in the middle of the setTimeout call. From the external point of view, the second timer is scheduled to be called between two milliseconds and you (external point of view, not implementor) have no idea which one.
but, as f1 was setup before f2, f1 should be called before f2. You can achieve this easily with a stack per target time. Your argument is based on the implementor point of view. I agree that any policy can be implemented. I do not doubt that. I am just saying that you cannot verify it from a script point of view, (and so there is no point specifying it).
I don't see why "you can't verify your expectation". If you think you can verify your expectation, please write ECMAScript interoperable test cases that show how to test whether an ECMAScript engine is conform to your scheduling policy or not. It will be enough to convince me. Testing one timer ("When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms" ") will not be difficult. Testing your scheduling policy is a different story.
A spec is a contract between an provider (implementor) and a client (ECMAScript script writer). If the client has no way to ensure that a spec detail has been met, there is no point adding this detail to the contract. With maybe the exception of Date.now() and Math.random() which are convinient but comes with no guarantee whatsoever. Their spec are empty (not literally, but close). While you seem to have "expectations" on a scheduling algorithm.
If I am missing something and you can write test cases to enforce expectation over a particular scheduling policy, please do so.
In order to know exactly the target time (Date.now() +ms) that a given setTimeout() has been scheduled for, setTimeout() would have to return it (it doesn't currently):
function newSetTimeout ( f, ms ) { var targetTime= Date.now()+ ms; scheduleTimeout(f, targetTime); return [ targetTime ]; }
The way it's now, most of the times you can, but sometimes you can't tell:
var ms= 50; var t= Date.now(); setTimeout(f, ms); if (t === Date.now()) { //We know: targetTime === t+ms } else { //Not sure, it can be t+ms or t+ms+1 }
But honestly, I don't see what your point is.
On 3/19/11 5:29 PM, Jorge wrote:
I have always wondered about the reason to clamp (nested) setTImeout()s, why, what for ?
Originally, implementations effectively clamped setTimeout because the timer primitives they were using only had about 10ms resolution (due to being based on the context-switching frequency of the OS, which was 100Hz for at least Windows and Linux at the time; not sure about others).
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it).
At least that's what I understand; you may want to ask the Chrome folks what issues they ran into when using a 1ms clamp.
Using no clamp at all will swamp the event loop in common cases used on the web and doesn't seem desirable.
Is it to protect the users' mobiles/laptops batteries from event loops spinning unnecessarily fast in some badly written web pages
Not just batteries. Also the UI in browsers that don't have separate threads for content and the UI.
Note that as far as batteries go at least IE9 seems to use a different clamp when running on battery. This probably has to do with the fact that getting timers with resolution better than 10ms on Windows involves changing the scheduler's timeslice in a way that consumes more power even if you aren't running timers all the time.
or perhaps in some popular (and possibly botched) library ?
0ms timeouts are all over the place on the web. They're in libraries, they're in websites. They're everywhere.
And if that's the reason, and if the clamp is needed/wanted/kept for that reason, what are we going to do to protect against functions that nextTick() themselves ?
Accidental misuse is a lot less likely if there is never a clamp in place. In particular, the "text disappears too fast" scenario can't arise in this situation. And I expect that web-facing UAs will introduce additional clamps as needed to deal with resource constraints (e.g. what IE9 is doing on battery).
And if we can't protect against nested nextTick()s, then, does it make sense to keep the protection against setTimeout()s of 0 ?
It does in web-facing UAs, due to the huge mass of existing poorly-authored content.
Le 20/03/2011 17:25, Jorge a écrit :
On 20/03/2011, at 16:57, David Bruant wrote:
Le 20/03/2011 16:04, Jorge a écrit :
On 20/03/2011, at 15:18, David Bruant wrote:
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
A timer that expires is an event, and I would expect events to be serviced in the order they happen. As when I click twice, I'd expect the first click to be serviced before the second click.
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes.
The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything. Why ? When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms".(Let's leave aside for now the problems wrt Date.now() / wall clock time). I agree that for one timer, there is no problem. Tricky case is if you want to enforce a policy on how to deal with several delayed timers (let's call it "scheduling policy"). That's where I say that it's not possible to pretend expecting anything.
In low-memory environment, if a program does an intensive use of timers, maybe that heuristics on the time the timer firing will take could be used to try to get rid of a maximum of timers as soon as possible to save up memory. I would consider this policy as valid due to environment constraints.
And if t0 were === t1, I would also expect them to be serviced in the same order in which they were setup:
setTimeout( f1, 0 ); setTimeout( f2, 0 ); setTimeout( f3, 0 ); setTimeout( f4, 0 ); setTimeout( f5, 0 );
-> f1() f2() f3() f4() f5()
I think it is very dangerous to use talk about things like t0 === t1 and Date.now() >= t0 (concerned about the "or equal"). If ECMAScript has millisecond as granularity, most systems have microseconds if not nanoseconds. 1ms = 10⁶ns. That's a lot! It leaves a lot of room for interpretation. And even the setTimeout calls aren't instantaneous.
Consider (milliseconds in comment): // 0 setTimeout( f1, 1 ); setTimeout( f2, 0 ); // 1 (millisecond change in the middle of the call) // 2 // -- decision to make on which timer to fire. Since the millisecond change happened during the second setTimeout, when is scheduled this second timeout? A setTimeout( f, ms ) should be scheduled to fire at t = Date.now() + ms,so in the example above, both would be scheduled to fire at the same t, "both would be scheduled to fire the same t". Who said that? The millisecond change is in the middle of the setTimeout call. From the external point of view, the second timer is scheduled to be called between two milliseconds and you (external point of view, not implementor) have no idea which one.
but, as f1 was setup before f2, f1 should be called before f2. You can achieve this easily with a stack per target time. Your argument is based on the implementor point of view. I agree that any policy can be implemented. I do not doubt that. I am just saying that you cannot verify it from a script point of view, (and so there is no point specifying it).
I don't see why "you can't verify your expectation". If you think you can verify your expectation, please write ECMAScript interoperable test cases that show how to test whether an ECMAScript engine is conform to your scheduling policy or not. It will be enough to convince me. Testing one timer ("When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms" ") will not be difficult. Testing your scheduling policy is a different story.
A spec is a contract between an provider (implementor) and a client (ECMAScript script writer). If the client has no way to ensure that a spec detail has been met, there is no point adding this detail to the contract. With maybe the exception of Date.now() and Math.random() which are convinient but comes with no guarantee whatsoever. Their spec are empty (not literally, but close). While you seem to have "expectations" on a scheduling algorithm.
If I am missing something and you can write test cases to enforce expectation over a particular scheduling policy, please do so. In order to know exactly the target time (Date.now() +ms) that a given setTimeout() has been scheduled for, setTimeout() would have to return it (it doesn't currently):
function newSetTimeout ( f, ms ) { var targetTime= Date.now()+ ms; scheduleTimeout(f, targetTime); return [ targetTime ]; }
The way it's now, most of the times you can, but sometimes you can't tell:
var ms= 50; var t= Date.now(); setTimeout(f, ms); if (t === Date.now()) { //We know: targetTime === t+ms
I am sorry, but we don't "know". Maybe that by the time we've "reached the bracket", the millisecond has changed. We can have been pre-empted by the OS scheduler, the system may be slow. There might have been garbage collection, maybe a repaint of the screen. There is no guarantee when it's about real time. The only guarantee is that the time moves forward.
} else { //Not sure, it can be t+ms or t+ms+1 }
But honestly, I don't see what your point is.
When I write setTimeout(f, 1000), I expect f to be called "after at least one second" (with the implicit "as soon as possible after the given time"). That's what programmers have expected since the beginning and implementors cannot really provide a better guarantee.
setTimeout(f1, 1); setTimeout(f2, 0); compute();
As a programmer, since you have no idea of how much time compute is going to take, you cannot have any expectation on f1 being called before f2 since at the end of compute one millisecond may have passed or not. And you organize your program to be independent of the calling order. My point is that since you can't rely on a scheduling policy and you already organize your program to be independent of the scheduling policy, there is no point standardizing the scheduling policy. No one will benefit from it. Neither the implementor, nor the programmer.
If you write:
setTimeout(f1, 0); setTimeout(f2, 0);
and expect f1 to be called before f2, then why don't you just do "setTimeout(function(){f1(); f2();}, 0);" ? the language already allows you to do what you want. There is no need to enforce the scheduling policy. If two timers happen to have been scheduled at the same target time by a more complicated manner, then it's you are already aware in both calls that the system doesn't guarantee anything about real time, so you're prepared for a race condition and as such do not have expectations on who will be fired first.
To answer your concern, my point is that imposing the scheduling policy won't solve a use case for users and won't help the work of the implementors (see my low-memory environment example).
The more I think about it and give counter-examples, the more I doubt what Brendan Eich said that no-determinism can be avoided if the spec is carefully written. The non-determinism due to code execution time seems to be unavoidable and leaks to any determinism we could try to reach with a scheduling policy as long as setTimeout respects "fire after at least x milliseconds".
Le 20/03/2011 17:14, Kyle Simpson a écrit :
I don't see why "you can't verify your expectation". If you think you can verify your expectation, please write ECMAScript interoperable test cases that show how to test whether an ECMAScript engine is conform to your scheduling policy or not. It will be enough to convince me. Testing one timer ("When you do a setTimeout( f, ms ) you know you are saying "fire this @ t >= Date.now() + ms" ") will not be difficult. Testing your scheduling policy is a different story.
Forgive my naivety, but would it not be possible to test such a scheduling policy with something like this:
var test = 1;
function a() { test *= -1; } function b() { assertEquals(test,-1); }
setTimeout(a,0); for(i=0;i<1E10;i++){i=i;} // or any other sufficiently long running algorithm
Actually, a dead code elimination optimization would make the loop disappear. Anyway, another "sufficiently long running algorithm" could loop and exit the loop with a condition on a time difference measured with Date.now()
setTimeout(b,0);
I agree that some properties of a scheduling policy can be tested (like you can test that times moves forward with two sequential calls to Date.now() or testing statistical properties of Math.rand()) but there is something in the time that code uses to execute that is inherently non-determinist (the dead code elimination optimization is actually an excellent proof of this). This cannot be avoided in my opinion, making scheduling policy non-testable, not verifyable and hence non-relyable.
On 3/20/11 1:19 PM, David Bruant wrote:
I agree that some properties of a scheduling policy can be tested (like you can test that times moves forward with two sequential calls to Date.now() or testing statistical properties of Math.rand()) but there is something in the time that code uses to execute that is inherently non-determinist (the dead code elimination optimization is actually an excellent proof of this). This cannot be avoided in my opinion, making scheduling policy non-testable, not verifyable and hence non-relyable.
It should be possible to statistically verify (which is the only way you can verify anything via a test suite, by the way) the following assertion: "Given two calls |setTimeout(f, n)| and |setTimeout(g, n)| using the same value of |n|, if the |setTimeout(f, n)| happens before the |setTimeout(g, n)| then f will be called before g."
Last I checked, the web depends on that behavior to some extent. Implementing it is not particularly burdensome.
Le 20/03/2011 18:23, Boris Zbarsky a écrit :
On 3/20/11 1:19 PM, David Bruant wrote:
I agree that some properties of a scheduling policy can be tested (like you can test that times moves forward with two sequential calls to Date.now() or testing statistical properties of Math.rand()) but there is something in the time that code uses to execute that is inherently non-determinist (the dead code elimination optimization is actually an excellent proof of this). This cannot be avoided in my opinion, making scheduling policy non-testable, not verifyable and hence non-relyable.
It should be possible to statistically verify (which is the only way you can verify anything via a test suite, by the way) the following assertion: "Given two calls |setTimeout(f, n)| and |setTimeout(g, n)| using the same value of |n|, if the |setTimeout(f, n)| happens before the |setTimeout(g, n)| then f will be called before g."
That's what I understand Kyle was testing too (the loop wasn't necessary) and I agree (as I said). Testing some properties and expecting some properties to be respected sound like a good idea.
On Mar 20, 2011, at 7:18 AM, David Bruant wrote:
Le 20/03/2011 14:33, Jorge Chamorro a écrit :
On 20/03/2011, at 13:51, Jorge wrote:
So given 2 timers, expiring at t0 and t1 with t0 < t1, if Date.now() is >= t0 and >= t1, I would expect t0 to be serviced first, yes. The difference is that the system can understand what is your expectation as a user when you've clicked twice. Both click are clearly sequenced. For timers, the non-determinism due to the computation time of computea and computeb prevents you from /expecting/ anything.
There's no non-determinism in the order in which f1 and f2 were set as timeouts. The setTimeout(f1, 1000) call happens-before setTimeout(f2, 0), deterministically.
The issue here is resolved by breaking ties by inserting after all timers with the same deadline.
On 20/03/2011, at 18:11, David Bruant wrote:
Le 20/03/2011 17:25, Jorge a écrit :
In order to know exactly the target time (Date.now() +ms) that a given setTimeout() has been scheduled for, setTimeout() would have to return it (it doesn't currently):
function newSetTimeout ( f, ms ) { var targetTime= Date.now()+ ms; scheduleTimeout(f, targetTime); return [ targetTime ]; }
The way it's now, most of the times you can, but sometimes you can't tell:
var ms= 50; var t= Date.now(); setTimeout(f, ms); if (t === Date.now()) { //We know: targetTime === t+ms I am sorry, but we don't "know". Maybe that by the time we've "reached the bracket", the millisecond has changed. We can have been pre-empted by the OS scheduler, the system may be slow. There might have been garbage collection, maybe a repaint of the screen. There is no guarantee when it's about real time. The only guarantee is that the time moves forward.
No, no, if Date.now() was === before and after the setTimeout(), it must have been scheduled to trigger at t+ms, for sure.
} else { //Not sure, it can be t+ms or t+ms+1 }
But honestly, I don't see what your point is. When I write setTimeout(f, 1000), I expect f to be called "after at least one second" (with the implicit "as soon as possible after the given time"). That's what programmers have expected since the beginning and implementors cannot really provide a better guarantee.
Of course. We all know that ;-)
setTimeout(f1, 1); setTimeout(f2, 0); compute();
As a programmer, since you have no idea of how much time compute is going to take, you cannot have any expectation on f1 being called before f2
It has nothing to do with compute(), it has to with the target times of the setTimeout()s, which were decided before compute().
since at the end of compute one millisecond may have passed or not. And you organize your program to be independent of the calling order. My point is that since you can't rely on a scheduling policy and you already organize your program to be independent of the scheduling policy, there is no point standardizing the scheduling policy. No one will benefit from it. Neither the implementor, nor the programmer.
I have relied on this behaviour many times. For example, when you wanted to do an animation in a browser (before we had CSS animations), you had to rely on this all the time.
If you write:
setTimeout(f1, 0); setTimeout(f2, 0);
and expect f1 to be called before f2, then why don't you just do "setTimeout(function(){f1(); f2();}, 0);" ?
Why would I need to ?
the language already allows you to do what you want. There is no need to enforce the scheduling policy.
There's "no need", but given:
setTimeout(f, 10); setTimeout(g, 20);
var t = Date.now() + 50; while ( Date.now() < t ) ;
it's not crazy to expect f() to be called before g()
If two timers happen to have been scheduled at the same target time by a more complicated manner, then it's you are already aware in both calls that the system doesn't guarantee anything about real time, so you're prepared for a race condition and as such do not have expectations on who will be fired first.
To answer your concern, my point is that imposing the scheduling policy won't solve a use case for users and won't help the work of the implementors (see my low-memory environment example).
The more I think about it and give counter-examples, the more I doubt what Brendan Eich said that no-determinism can be avoided if the spec is carefully written. The non-determinism due to code execution time seems to be unavoidable and leaks to any determinism we could try to reach with a scheduling policy as long as setTimeout respects "fire after at least x milliseconds".
You know the order in which they've been setup (because they've been pushed on stacks), and you know the exact target times of each and every one of these stacks, while it's true that you can't tell for sure the exact time when they're finally going to be serviced, once it arrives, whatever it is ( >= targetTime ), you can (and should, imo) at least dispatch them respecting both the insertion order (pulling from the stacks) and the target times too (servicing the stacks in ascending targetTime order).
I think that would be the right way to do it. And I'd bet that's ~ what Brendan was thinking about when he said "We need to be careful about order, though". But only Brendan knows.
On 11:59 AM, Boris Zbarsky wrote:
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it).
I'm not convinced that this is the meaning. I use |setTimeout(f, 0)| to mean "schedule f after this event completes" or "push |f| onto the event queue"; I think that is the common meaning. A delay value of zero is perfectly sensible, but we mean "zero seconds after other queued events", not "zero seconds after this event". We assume (without real evidence I suppose) that UI and IO events can be queued while our JS (the caller of |setTimeout(f, 0)|) runs.
The essential logic of |setTimeout(f, 0)| is cooperative multiprocessing yield: we believe that our current event processing could be long enough to interfere with user interaction and |f| can be delayed without serious impact on the UI. We have no reason to pick a value other than zero. If we want to give the user 3 seconds to read text we use 3000.
The "swamp the event loop" case is logically unrelated to this case, just like setInterval is unrelated. Multiple repeated calls to |setTimeout(f,0)| are bugs and setInterval of zero would be a bug. Of course the browser has to deal with theses cases, but the lower limit on setTimeout seems like crude solution.
jjb
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it). I'm not convinced that this is the meaning. I use |setTimeout(f, 0)| to mean "schedule f after this event completes" or "push |f| onto the event queue"; I think that is the common meaning. A delay value of zero is perfectly sensible, but we mean "zero seconds after other queued events", not "zero seconds after this event". We assume (without real evidence I suppose) that UI and IO events can be queued while our JS (the caller of |setTimeout(f, 0)|) runs.
I'd actually say that the most common meaning for setTimeout(f,0) is: "do
f as soon as possible after the current code finishes." There's a bunch of
places I do things like that. For instance, in IE (older versions), it's
prudent to delay any code with a setTimeout(..., 0); that is adding script
elements to the DOM, to avoid certain crashing race conditions. And there's
of course dozens of other cases where similar things are a reality on the
web.
If:
setTimeout(a,0); setTimeout(b,0); // some non-trivial computation
... is really to be interpreted that a and b have a non-deterministic
ordering, that's quite counter-intuitive and in fact will definitely have
potential to break some code across the web. On the contrary, I've never
seen that pattern be unreliable, so I would suspect all the browsers are
already guaranteeing that order, using some "queue" (not a "stack",
obviously, as Jorge has been saying) for each target time.
Furthermore, this pattern should also be true here, right?
setTimeout(a,5); var then = (new Date()).getTime(), now; do { now = (new Date()).getTime(); } while (now < (then + 5)); setTimeout(b,0);
If in this case b can go before a, that seems like that violates the
principle of least-surprise, because as a programmer I've basically tried to
ensure with that snippet that both timers are set to fire at essentially the
same time target, and if they are firing at ~ the same time, I can't see any
expectation that makes sense except "first-come-first-served" (in other
words, the time-target queueing).
There are cases where non-determinism are unfortunate reality, like the iteration order of objects :), but I don't think that timer/event ordering should be one of them, if it can be avoided.
Multiple repeated calls to |setTimeout(f,0)| are bugs
I don't agree with that assertion at all. Two different functions might "queue up" two different snippets to happen "as soon as possible, later", each of them using their own setTimeout(..., 0).
and setInterval of zero would be a bug.
setInterval(...,0) may be silly, but that doesn't mean it's a bug. It means "make this happen as fast as possible", just like above where setTimeout(...,0) means "make this happen as soon as possible".
The "swamping" would occur if setInterval(f,0) was actually going to spin in sub-millisecond speeds. It could also occur if you fake setInterval with:
function f() { // ... setTimeout(f,0); } f();
But in either of those cases, I don't see why there'd be any reason for "clamping" at anything higher than 1ms (being the smallest unit of time I can address with the API anyway)?
If it has been more than 20 milliseconds i expect both of them to be able to be fired, so sure. In some ways this is similar how i don't rely on the order of event listeners being fired.
On 3/20/2011 12:03 PM, Kyle Simpson wrote:
...
Multiple repeated calls to |setTimeout(f,0)| are bugs
I don't agree with that assertion at all. Two different functions might "queue up" two different snippets to happen "as soon as possible, later", each of them using their own setTimeout(..., 0).
But I didn't say two different snippets. I said Multiple repeat calls to the same snippet.
and setInterval of zero would be a bug.
setInterval(...,0) may be silly, but that doesn't mean it's a bug. It means "make this happen as fast as possible", just like above where setTimeout(...,0) means "make this happen as soon as possible".
Looping as fast as possible is likely to be a bug. It's not similar to queuing events.
The "swamping" would occur if setInterval(f,0) was actually going to spin in sub-millisecond speeds. It could also occur if you fake setInterval with:
function f() { // ... setTimeout(f,0); } f();
Yes, like I said, multiple repeat calls to the same snippet.
But in either of those cases, I don't see why there'd be any reason for "clamping" at anything higher than 1ms (being the smallest unit of time I can address with the API anyway)?
I can't see any reason to lump event queuing with looping when its looping that causes the issues.
jjb
On Sun, Mar 20, 2011 at 3:03 PM, Kyle Simpson <getify at gmail.com> wrote:
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they
really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it).
I'm not convinced that this is the meaning. I use |setTimeout(f, 0)| to mean "schedule f after this event completes" or "push |f| onto the event queue"; I think that is the common meaning. A delay value of zero is perfectly sensible, but we mean "zero seconds after other queued events", not "zero seconds after this event". We assume (without real evidence I suppose) that UI and IO events can be queued while our JS (the caller of |setTimeout(f, 0)|) runs.
I'd actually say that the most common meaning for
setTimeout(f,0)is: "dofas soon as possible after the current code finishes." There's a bunch of places I do things like that. For instance, in IE (older versions), it's prudent to delay any code with a setTimeout(..., 0); that is adding script elements to the DOM, to avoid certain crashing race conditions. And there's of course dozens of other cases where similar things are a reality on the web.If:
setTimeout(a,0); setTimeout(b,0); // some non-trivial computation
... is really to be interpreted that
aandbhave a non-deterministic ordering, that's quite counter-intuitive and in fact will definitely have potential to break some code across the web. On the contrary, I've never seen that pattern be unreliable, so I would suspect all the browsers are already guaranteeing that order, using some "queue" (not a "stack", obviously, as Jorge has been saying) for each target time.Furthermore, this pattern should also be true here, right?
setTimeout(a,5); var then = (new Date()).getTime(), now; do { now = (new Date()).getTime(); } while (now < (then + 5)); setTimeout(b,0);
If in this case
bcan go beforea, that seems like that violates the principle of least-surprise, because as a programmer I've basically tried to ensure with that snippet that both timers are set to fire at essentially the same time target, and if they are firing at ~ the same time, I can't see any expectation that makes sense except "first-come-first-served" (in other words, the time-target queueing).
This is a nice demonstration the non-determinism Brenden suggested. But you say "firing at ~ the same time" -- that should be suspect right there -- implementations would need to deign the developer's intent. AFAICT the only way to make this behave sanely is to take the event turn into consideration when scheduling -- b should deterministically happen ~ 5 ms before a because they were both scheduled in the same event turn and thus have the same beginning time. Maybe it's not perfect but it's also not all that surprising, and never ND.
There are cases where non-determinism are unfortunate reality, like the iteration order of objects :), but I don't think that timer/event ordering should be one of them, if it can be avoided.
What implementation has ND iteration order of objects?
Multiple repeated calls to |setTimeout(f,0)| are bugs
I don't agree with that assertion at all. Two different functions might "queue up" two different snippets to happen "as soon as possible, later", each of them using their own setTimeout(..., 0).
Also, if this were really the case, node wouldn't have process.nextTick. Again, it all comes down to fine-grained control over turns of the event loop.
and setInterval of zero would be a bug.
setInterval(...,0) may be silly, but that doesn't mean it's a bug. It means "make this happen as fast as possible", just like above where setTimeout(...,0) means "make this happen as soon as possible".
The "swamping" would occur if setInterval(f,0) was actually going to spin in sub-millisecond speeds. It could also occur if you fake setInterval with:
function f() { // ... setTimeout(f,0); } f();
But in either of those cases, I don't see why there'd be any reason for "clamping" at anything higher than 1ms (being the smallest unit of time I can address with the API anyway)?
There's no theoretical reason to clamp even at 1ms -- just historic *break the web *reasons. The non-determinism problem seems like a good justification for TC39 to considering formalizing the event loop. Still, if you all standardize anything it probably shouldn't be setTimeout with its clamping baggage -- let the web platform crowd deal with that mess :)
On 3/20/11 1:55 PM, John J. Barton wrote:
On 11:59 AM, Boris Zbarsky wrote:
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it). I'm not convinced that this is the meaning. I use |setTimeout(f, 0)| to mean "schedule f after this event completes"
You do. Some sites do that. A lot of other sites mean "oh, just do it the way it looks to me when I test it in this one browser I bothered to test with".
I think that is the common meaning.
Both meanings are very common. If you're claiming that one is more common than the other, I'd like to see some data.
In any case, both meanings are common enough that breaking either one significantly is not an option.
Again, Chrome tried to make setTimeout(f, 0) run after no more than 1ms, and discovered that sites "break" whatever that means.
We assume (without real evidence I suppose)
Yep, definitely without evidence. ;)
The essential logic of |setTimeout(f, 0)| is cooperative multiprocessing yield: we believe that our current event processing could be long enough to interfere with user interaction and |f| can be delayed without serious impact on the UI. We have no reason to pick a value other than zero. If we want to give the user 3 seconds to read text we use 3000.
- You would be surprised at what web pages do, re your 3000 thing.
- "yield" is not good enough if each execution of your timeout takes an appreciable amount of time. For example, if your execution is 10ms but you schedule the timer to run every 2ms, you will swamp the event loop. 84% of the time or so things will be nonresponsive.
The "swamp the event loop" case is logically unrelated to this case,
Not at all.
just like setInterval is unrelated. Multiple repeated calls to |setTimeout(f,0)| are bugs and setInterval of zero would be a bug.
Those are the cases that get clamped. If you just setTimeout(f, 0) when not inside a timeout, you don't get clamped.
Of course the browser has to deal with theses cases, but the lower limit on setTimeout seems like crude solution.
It's the one browser makers have all implemented.
On Sun, Mar 20, 2011 at 4:05 PM, Boris Zbarsky <bzbarsky at mit.edu> wrote:
On 3/20/11 1:55 PM, John J. Barton wrote:
On 11:59 AM, Boris Zbarsky wrote:
Nowadays the clamp is there because sites use |setTimeout(f, 0)| when they really mean "run this at 10Hz" and if you run it with no delay then they swamp your event loop and possible render "wrong" (e.g. the text disappears before the user has a chance to read it).
I'm not convinced that this is the meaning. I use |setTimeout(f, 0)| to mean "schedule f after this event completes"
You do. Some sites do that. A lot of other sites mean "oh, just do it the way it looks to me when I test it in this one browser I bothered to test with".
I think that is the common meaning.
Both meanings are very common. If you're claiming that one is more common than the other, I'd like to see some data.
In any case, both meanings are common enough that breaking either one significantly is not an option.
Again, Chrome tried to make setTimeout(f, 0) run after no more than 1ms, and discovered that sites "break" whatever that means.
We assume (without real evidence I suppose)
Yep, definitely without evidence. ;)
The essential logic of |setTimeout(f, 0)| is cooperative multiprocessing
yield: we believe that our current event processing could be long enough to interfere with user interaction and |f| can be delayed without serious impact on the UI. We have no reason to pick a value other than zero. If we want to give the user 3 seconds to read text we use 3000.
- You would be surprised at what web pages do, re your 3000 thing.
- "yield" is not good enough if each execution of your timeout takes an appreciable amount of time. For example, if your execution is 10ms but you schedule the timer to run every 2ms, you will swamp the event loop. 84% of the time or so things will be nonresponsive.
The "swamp the event loop" case is logically unrelated to this case,
Not at all.
just like setInterval is unrelated. Multiple repeated calls to
|setTimeout(f,0)| are bugs and setInterval of zero would be a bug.
Those are the cases that get clamped. If you just setTimeout(f, 0) when not inside a timeout, you don't get clamped.
Of course the browser has to deal with theses cases, but the lower limit
on setTimeout seems like crude solution.
It's the one browser makers have all implemented.
The key word here is "browser makers". Maybe node's process.nextTick is a footgun, but it's one you'll need if you want to roll your own scheduling, for instance. Apparently there is real evidence that clamping in setTimeout is necessary but that doesn't mean it makes sense as a language construct.
For an event with multiple listeners you can't rely on a certain order because it's spec'ed that you shouldn't.
But given two events of the same kind and a single listener, you'd expect them to be dispatched in order, won't you ?
A timer that times out is an event, and timers in the farther past should timeout before more recent ones. There's an order there that should be honored.
On 20/03/2011, at 21:30, Dean Landolt wrote:
On Sun, Mar 20, 2011 at 4:05 PM, Boris Zbarsky <bzbarsky at mit.edu> wrote:
It's the one browser makers have all implemented.
The key word here is "browser makers". Maybe node's process.nextTick is a footgun,
It isn't a footgun, it's a necessity for example for long-running tasks that shouldn't block the event loop. Either a nextTick() or a non-clamping setTimeout( f, 0 );
but it's one you'll need if you want to roll your own scheduling, for instance.
Exactly.
Apparently there is real evidence that clamping in setTimeout is necessary but that doesn't mean it makes sense as a language construct.
It doesn't, so we're going to need a non-clamping alias. Perhaps an [ugly] setTimeout ( ƒ, -1 ) ?
It doesn't, so we're going to need a non-clamping alias. Perhaps an [ugly] setTimeout ( ƒ, -1 ) ?
I posit that the clamping behaviour and timer resolution are domain-specific (embedding-specific) implementation details. Browser makers have been able to deal with run-away CPU scripts, even the ES5 theoretically allows programs which consume all available CPU. Similarly, I'm sure they can manage to figure out how to clamp setTimeout() even if it isn't specified to have an explicit clamping behaviour.
That said, I am personally more interested in a set of primitives which can be used to implement setTimeout, rather than setTimeout itself. Although, browser makers will probably have to clamp the primitives, too..
On Sun, Mar 20, 2011 at 6:19 PM, Wes Garland <wes at page.ca> wrote:
It doesn't, so we're going to need a non-clamping alias. Perhaps an [ugly] setTimeout ( ƒ, -1 ) ?
I posit that the clamping behaviour and timer resolution are domain-specific (embedding-specific) implementation details. Browser makers have been able to deal with run-away CPU scripts, even the ES5 theoretically allows programs which consume all available CPU. Similarly, I'm sure they can manage to figure out how to clamp setTimeout() even if it isn't specified to have an explicit clamping behaviour.
That said, I am personally more interested in a set of primitives which can be used to implement setTimeout, rather than setTimeout itself. Although, browser makers will probably have to clamp the primitives, too..
IIUC the only reason for the clamping requirement is due to code in the wild that relies on that behavior from setTimeout. If you build setTimeout on a lower-level primitive then clamping the primitive would be unnecessary, functionally equivalent to rate-limiting while(1){}.
It isn't in the past though, it is scheduled to be run as soon as
possible past a certain point. That being said as soon as possible
may not be in a particular order (os schedulers / threads / event
listeners in js for example). I view this close to even listeners in
the respect that schedule of events cannot be determined. While it may
appear to be logical to have events FIFO based upon maxage, most other
systems besides a simple manual iteration of a set do not follow this.
I don't schedule events in a particular order because I have never
encountered a situation where I would want 2 timers where I couldn't
schedule the 2nd one that depends upon the first, after the first. Is
there a situation where it makes sense to have
setTimeout(f,10) setTimeout(g,20)
where g depends f instead of:
setTimeout(function(){ f(); setTimeout(g,10); },10)
Which is more verbose, but far clearer on intention.
- This allows for the linear fashion that people may desire, without causing limiting behavior on the scheduler.
- This is much more clear that g should fire 10milliseconds after f rather than possibly immediately after f (which presumably is what the example wants).
- It encourages less
magicin how things are scheduled. Expectations of manipulation are lowered and code design is encouraged over engine implementation.
The fact that you say an order should be honored is a bit misleading for 2 facts.
- I would expect g to fire 10ms after f, but it may fire immediately.
- By design it separates the dependent functions f and g so I have no reason seeing that code to expect g to rely on f.
So while you may see this as being an expectation of FIFO based upon max age, I merely see it as after x milliseconds something should be fired, nothing more, no magic.
On Sun, Mar 20, 2011 at 12:24 PM, John J. Barton <johnjbarton at johnjbarton.com> wrote:
Looping as fast as possible is likely to be a bug. It's not similar to queuing events.
It's the behaviour intentionally (if unwisely) requested by a lot of animations and games, for what it's worth. There are better models becoming available (*RequestAnimationFrame, for example), but not standardized or universal.
Mike
On 21/03/2011, at 00:21, Bradley Meck wrote:
It isn't in the past though, it is scheduled to be run as soon as possible past a certain point. That being said
as soon as possiblemay not be in a particular order (os schedulers / threads / event listeners in js for example). I view this close to even listeners in the respect that schedule of events cannot be determined. While it may appear to be logical to have events FIFO based upon maxage, most other systems besides a simple manual iteration of a set do not follow this. I don't schedule events in a particular order because I have never encountered a situation where I would want 2 timers where I couldn't schedule the 2nd one that depends upon the first, after the first. Is there a situation where it makes sense to havesetTimeout(f,10) setTimeout(g,20)
where g depends f instead of:
setTimeout(function(){ f(); setTimeout(g,10); },10)
Which is more verbose, but far clearer on intention.
- This allows for the linear fashion that people may desire, without causing limiting behavior on the scheduler.
- This is much more clear that g should fire 10milliseconds after f rather than possibly immediately after f (which presumably is what the example wants).
- It encourages less
magicin how things are scheduled. Expectations of manipulation are lowered and code design is encouraged over engine implementation.The fact that you say an order should be honored is a bit misleading for 2 facts.
- I would expect g to fire 10ms after f, but it may fire immediately.
- By design it separates the dependent functions f and g so I have no reason seeing that code to expect g to rely on f.
So while you may see this as being an expectation of FIFO based upon max age, I merely see it as after x milliseconds something should be fired, nothing more, no magic.
I don't think there's any magic in this:
1.- If you could service every timer at exactly its target time, that would be the expected order. 2.- If you can't, because Date.now() is already > their target times, well, just service them in the expected order, as in (1).
Foreword Each time I write "setTimeout", I mean "setTimeout and setInterval (and there clear functions)" (please forgive the recursive flaw)
Introduction Before the concurrency proposal (strawman:concurrency), a pure ECMAScript program had a start and an end, that was it. No event loop, no asynchronous callback, etc. If ECMAScript standardizes this proposal or another, there will be a need to standardize an event loop in a way or another. Adding a timing aspect to this event loop and setTimeout can be standardized in ECMAScript.
Current_standardization_state setTimeout isn't part of ECMAScript. setTimeout is nonetheless standardized as part of "HTML Standard" (www.whatwg.org/specs/web-apps/current-work/multipage/timers.html#timers). Besides the "task" dependency (which is part of the standardized event-loop in the same document: www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#concept-task), this is more or less ECMAScript.
Current_use As anyone will have certainly noticed, setTimeout has no reason to be considered as client-side web specific. And, as a matter of fact, there is a setTimeout in node.js and in ActionScript apparently. I wouldn't be surprised if most (if not all) ECMAScript-based languages had a setTimeout function consistently.
For all these reasons, I am wondering if setTimeout wouldn't be had being standardized in ECMAScript itself.
How? I currently see two main tracks:
Advantages
I haven't found any trace of previous discussion of this topic. Has there been any?