__doc__ for functions, classes, objects etc.
On 21.08.2011 22:26, Peter van der Zee wrote:
On Sun, Aug 7, 2011 at 6:56 PM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com> wrote:
Hi,
What about to standardize (Python's) "doc"-umentation comments for JS? What's the difference between that and the /** ... */ way of "JSDoc"?
/**
- This function does stuff
- @constructor
- @param {Object} foo
- @returns {Array} */ var f = function(){ ... };
It doesn't matter for me how exactly (syntactically) it will look like. In this particular case the comment is outside a function's body -- you then should consider and allow or not newlines after the doc-comment, etc. Though, as well as in case of placing the comment inside the body, but in this case I think we should mention that the doc-comment is should be placed in the prologue.
What would a new language construct for this solve?
That's said, triple-strings by themselves (not only for doc-comments) are very powerful construct to have and standardize. Did you work with them in Python or CoffeeScript?
P.S.:
Another thing to notice. As was mentioned, it's good to have this ability also for simple objects. From this viewpoint doc-string should be set via property. But if I remember correctly, someone (all?) from the committee didn't like Python's names (despite the fact that JS actually use them for proto, noSuchMethod, count, parent, iterator etc. -- non is standardized, stratified meta-level won). But it would be great to have the ability to:
let foo = { doc: "My awesome object", x: 100, y: 200 };
Object.help(foo); // "My awesome object"
or simply help(foo)
.
Dmitry.
On 23.08.2011 20:54, Brendan Eich wrote:
A convenient notation for multiline documentation comments, with convenient reflection (not via toString() scraping!), would be a fine thing.
Yes, exactly this -- help(...) function, and also good auto-complete of object methods helped me some time ago formerly to learn Python very quickly just playing with it in console. By the way, ECMAScript (who will take responsibility to implement the ECMAScript -- ethanol implementation? :)) also needs some installation binaries with the console. And yes -- with this great features for learning
This is a great idea, but I wonder if the concept could be implemented using comments as we already know them? This would leave it up to the user how they want to parse the property's value. Just a thought - either way, I think this is a valuable idea.
On Sun, Aug 7, 2011 at 6:56 PM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com> wrote:
Hi,
What about to standardize (Python's) "doc"-umentation comments for JS?
What's the difference between that and the /** ... */ way of "JSDoc"?
/**
- This function does stuff
- @constructor
- @param {Object} foo
- @returns {Array} */ var f = function(){ ... };
What would a new language construct for this solve?
On 22.08.2011 12:47, Irakli Gozalishvili wrote:
I'm very much interested in getting something like this! In fact I have been useing similar documentation style for some time already:
function doc(lambda) { var doc = //*([\s\S]*)?*//.exec(String(lambda)) return doc ? doc[1].trim() : '' }
function sum(a, b) { /* Calculates sum of given
a
andb
arguments */ return a + b; }doc(sum) \ > "Calculates sum of given
a
andb
arguments"Unfortunately this does not works on spidermonkey based js engines :(
Right. That's why I raised this issue, because the functionality is very
powerful. Backing to my concrete example with redis, I just wanted to
understand why the heck client.get('x'), after setting client.set('x',
10) returns for me true
, but not 10. If I had this help(...)
functionality I'd be able to get the description right from the console,
saying that .get
function should accept callback which already returns
the value: client.get('x', function(err, data) {return data;}) -> 10.
I of course could go to the documentation website (what actually I did after my fail tries :)), but I think it goes without saying that it would be the best if I could do this right from the console! Besides, this [[Documentation]] property can be used in hints (with showing parameters and their types) when typing function name.
From this viewpoint Python just wins and IMO we should have this feature in the ES.
Also I found that best way to support this on spidermonkey is via E4X hacks
function sum(a, b) { <doc>Calculates sum of given
a
andb
arguments</doc> return a + b }Unfortunately I don't have any solution working across all engines that would not require multiline string escaping:
function sum(a, b) { var doc = "Calculates
sum of given
a
andb
argumnets"return a + b }
Yes, it's another pros to have [[Documentation]] standardized.
On Sunday, 2011-08-07 at 21:30 , Dmitry A. Soshnikov wrote:
On 21.08.2011 22:26, Peter van der Zee wrote:
On Sun, Aug 7, 2011 at 6:56 PM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote:
Hi,
What about to standardize (Python's) "doc"-umentation comments for JS? What's the difference between that and the /** ... */ way of "JSDoc"?
/**
- This function does stuff
- @constructor
- @param {Object} foo
- @returns {Array} */ var f = function(){ ... };
It doesn't matter for me how exactly (syntactically) it will look like. In this particular case the comment is outside a function's body -- you then should consider and allow or not newlines after the doc-comment, etc. Though, as well as in case of placing the comment inside the body, but in this case I think we should mention that the doc-comment is should be placed in the prologue.
What would a new language construct for this solve?
That's said, triple-strings by themselves (not only for doc-comments) are very powerful construct to have and standardize. Did you work with them in Python or CoffeeScript?
P.S.:
Another thing to notice. As was mentioned, it's good to have this ability also for simple objects. From this viewpoint doc-string should be set via property. But if I remember correctly, someone (all?) from the committee didn't like Python's names (despite the fact that JS actually use them for proto, noSuchMethod, count, parent, iterator etc. -- non is standardized, stratified meta-level won). But it would be great to have the ability to:
let foo = { doc: "My awesome object", x: 100, y: 200 };
Object.help(foo); // "My awesome object"
or simply
help(foo)
.I don't think we need special property for that, as anyone can just use
doc
or whatever they decide, bigger issue IMO is a property docs. How do you documentfoo.x
if x is not a function or object ?
It was just an example. Mostly this doc is needed for functions. Though, we can think how we can do this for every property.
Dmitry.
So, is that's all? Anyone else thinks it's a needed thing in the ES?
A convenient notation for multiline documentation comments, with convenient reflection (not via toString() scraping!), would be a fine thing.
Some of the design dimensions:
-
Comment vs. string / quasiliteral?
-
function-only, or object literal too -- or any declaration?
-
Before function, a la javadoc comments, or first thing in body, a la the prologue directive idea?
-
Reflected via function .doc property, a Function.extractDocComment static method, or something even more mirror-like/stratified?
I'm not sure what is best, I lack experience programming in languages with doc-comment or triple-quoted equivalents (Python attached tests, e.g.). Comments (heh) welcome.
Thinking about 1, I would start with function-only.
For 2 I'm inclined to say "in body" because it's too easy to lose the "before" context during the life of a function, compared to losing part of the body by accident.
Regarding 3, I bet Function.extractDocComment or a better name wins, especially if the whole solution allows monkey-patching a polyfill for downrev browsers that support source recovery (not SpiderMonkey's decompiler).
On 8/23/11 at 9:54, brendan at mozilla.com (Brendan Eich) wrote:
- Before function, a la javadoc comments, or first thing in body, a la the prologue directive idea?
I have found it difficult to read a Java program from top to bottom because the javadoc comments proceed the function/method header. I want to see what the compiler reads first and then read the human commentary about what it means. With the way Java is set up, I have to jump around in the source instead of reading from top to bottom.
Cheers - Bill
Bill Frantz | Snow shuts down federal | Periwinkle (408)356-8506 | government, life goes on. | 16345 Englewood Ave www.pwpconsult.com | - Associated press 2/9/2010 | Los Gatos, CA 95032
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
Some summarized features below:
- Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though)
- Auto-suggest and hints of parameters and their types when typing function name in console.
- Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc).
- Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments")
As an example I show a spec description for parseInt
function:
15.1.2.2 parseInt (string , radix)
Let [[Documentation]] property of the parseInt
function be the
following string:
"The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X."
...
x.x.xx help(F)
When the help
function is called on the F object, the following steps
are taken:
- If [[Class]] of the function is not "Function" throw TypeError
- Let doc be [[Documentation]] property of F
2.1 If doc is
undefined
return empty string 2.2. return String(doc)
It's a simplified version of course (moreover, Allen wanted to eliminate [[Class]], so -- it's just an example).
Do we need this?
Dmitry.
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov < dmitry.soshnikov at gmail.com> wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
Some summarized features below:
- Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though)
- Auto-suggest and hints of parameters and their types when typing function name in console.
- Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc).
- Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments")
As an example I show a spec description for
parseInt
function:15.1.2.2 parseInt (string , radix)
Let [[Documentation]] property of the
parseInt
function be the following string:"The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X."
...
x.x.xx help(F)
A built-in, global object function property named "help()" will no doubt conflict with userland code
On 30.08.2011 17:41, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. Some summarized features below: + Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though) + Auto-suggest and hints of parameters and their types when typing function name in console. + Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc). - Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments") As an example I show a spec description for `parseInt` function: 15.1.2.2 parseInt (string , radix) Let [[Documentation]] property of the `parseInt` function be the following string: "The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X." ... x.x.xx help(F)
A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though,
IMO help
name isn't used much). The main thing I want to clarify at
this step, whether we need at all this thing in ES?
Since committee keeps silence, I'd like to consider it better as "Yes, we need it and there is nothing to add", rather than "We don't need and don't want even to discuss" (since there is no big sense in later, because the functionality seems useful). However, we need to clarify how exactly it's useful. Maybe using JS in console is so rare case and it isn't required much. OTOH, era of client-side-only-JS is behind and JS is also on server now. And from this viewpoint, in the console, it's the best to get the help, hints for parameters and even type checking for functions.
So, what should I do to apply the first meaning?
Dmitry.
On Tue, Aug 30, 2011 at 10:41 AM, Dmitry A. Soshnikov < dmitry.soshnikov at gmail.com> wrote:
On 30.08.2011 17:41, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov < dmitry.soshnikov at gmail.com> wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
Some summarized features below:
- Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though)
- Auto-suggest and hints of parameters and their types when typing function name in console.
- Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc).
- Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments")
As an example I show a spec description for
parseInt
function:15.1.2.2 parseInt (string , radix)
Let [[Documentation]] property of the
parseInt
function be the following string:"The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X."
...
x.x.xx help(F)
A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though, IMO
help
name isn't used much).
That's tough to say for sure, and even if true still probably isn't good enough. Regardless, there's no excuse for this kind of global, especially in light of a module system. Sure, specifically in a repl you'd probably want to skip the require -- for this node offers up dot-prefixed repl keywords. So something like .help(F) would be a lot safer and nearly as convenience as a help global.
I think it would be useful for the committee to weigh in on the idea of repl plugins. This particular design is pretty nice but it conflicts for statements beginning with non-zero-prefixed decimals. In practice I don't see this causing any real problems, but it'd be great if the grammer explicitly forbid this, effectively reserving it for repl hooks.
The main thing I want to clarify at this step, whether we need at all this thing in ES?
Since committee keeps silence, I'd like to consider it better as "Yes, we need it and there is nothing to add", rather than "We don't need and don't want even to discuss" (since there is no big sense in later, because the functionality seems useful). However, we need to clarify how exactly it's useful. Maybe using JS in console is so rare case and it isn't required much. OTOH, era of client-side-only-JS is behind and JS is also on server now. And from this viewpoint, in the console, it's the best to get the help, hints for parameters and even type checking for functions.
Brenden weighed in and laid out some of the design space. I suspect this functionality can largely be pieced together with what harmony already gives us. As Brenden hinted, quasis give us multiline strings. The doc key is just screaming to be a private name instead (generally true of all wunderbars IMHO). This could be tacked onto functions and objects imperatively, but it would be pretty goofy to put the doc string after its function body. So for this to be serviceable we'd need a declarative way to do this inside the function body. ISTR some syntax being thrown around to set an own-prop on a function from inside its body but I can't recall specifics -- is there anything harmonious for this?
Put the above three bits together with a de facto repl plugin syntax and module mapping and you get a more sanitized (and extensible) version of python's repl with the same developer ergonomics.
On Tue, Aug 30, 2011 at 10:41 AM, Dmitry A. Soshnikov < dmitry.soshnikov at gmail.com> wrote:
On 30.08.2011 17:41, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov < dmitry.soshnikov at gmail.com> wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
Some summarized features below:
- Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though)
- Auto-suggest and hints of parameters and their types when typing function name in console.
- Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc).
- Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments")
As an example I show a spec description for
parseInt
function:15.1.2.2 parseInt (string , radix)
Let [[Documentation]] property of the
parseInt
function be the following string:"The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X."
...
x.x.xx help(F)
A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though, IMO
help
name isn't used much). The main thing I want to clarify at this step, whether we need at all this thing in ES?
I support this idea :)
BTW, here are some incomplete numbers on "help()" (sort of) www.google.com/codesearch#search/&q=lang:^javascript$ help\(&type=cs
This feature is particularly useful for framework authors and their users. Countless times I've found myself using a browser's console debugging tool to figure out why something's busted. In these cases, I nearly always resort to navigating to some JS framework's website for their documentation or scrutinizing their code directly to figure out what's going wrong. If I could just call 'printHelp(xyz....someMethod)', less of my time would be used. The other bit of utility here is that framework documentation can live side-by-side with the code and be automatically generated from it (e.g., pydoc.org/2.5.1/timeit.html) as opposed to maintaining it separately. A very useful feature to have from my Python-biased perspective:)
My best,
Dave
From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Dmitry A. Soshnikov Sent: Tuesday, August 30, 2011 7:42 AM To: Rick Waldron Cc: Brendan Eich; es-discuss Steen Subject: Re: doc for functions, classes, objects etc.
On 30.08.2011 17:41, Rick Waldron wrote: On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com<mailto:dmitry.soshnikov at gmail.com>> wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
Some summarized features below:
- Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though)
- Auto-suggest and hints of parameters and their types when typing function name in console.
- Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc).
- Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments")
As an example I show a spec description for parseInt
function:
15.1.2.2 parseInt (string , radix)
Let [[Documentation]] property of the parseInt
function be the following string:
"The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X."
...
x.x.xx help(F)
A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though, IMO help
name isn't used much). The main thing I want to clarify at this step, whether we need at all this thing in ES?
Since committee keeps silence, I'd like to consider it better as "Yes, we need it and there is nothing to add", rather than "We don't need and don't want even to discuss" (since there is no big sense in later, because the functionality seems useful). However, we need to clarify how exactly it's useful. Maybe using JS in console is so rare case and it isn't required much. OTOH, era of client-side-only-JS is behind and JS is also on server now. And from this viewpoint, in the console, it's the best to get the help, hints for parameters and even type checking for functions.
So, what should I do to apply the first meaning?
Dmitry.
When the help
function is called on the F object, the following steps are taken:
- If [[Class]] of the function is not "Function" throw TypeError
- Let doc be [[Documentation]] property of F
2.1 If doc is
undefined
return empty string 2.2. return String(doc)
It's a simplified version of course (moreover, Allen wanted to eliminate [[Class]], so -- it's just an example).
Do we need this?
Dmitry.
On 09.08.2011 23:59, Dmitry A. Soshnikov wrote: On 23.08.2011 20:54, Brendan Eich wrote: A convenient notation for multiline documentation comments, with convenient reflection (not via toString() scraping!), would be a fine thing.
Yes, exactly this -- help(...) function, and also good auto-complete of object methods helped me some time ago formerly to learn Python very quickly just playing with it in console. By the way, ECMAScript (who will take responsibility to implement the ECMAScript -- ethanol implementation? :)) also needs some installation binaries with the console. And yes -- with this great features for learning -- auto-complete of methods (the best thing to investigate objects just pressing tab
) and help(...) function. I remember ES4 had/has the REPL, why not ES5? Of course we have all those consoles directly from the browsers (and also Node.js REPL), but it could be useful. Anyway, it's another topic, just relatively touches help(...) functions.
Some of the design dimensions:
-
Comment vs. string / quasiliteral?
-
function-only, or object literal too -- or any declaration?
I think functions are the main case, yes. Not sure about literals, since there are open issues such as, "how to document a property" (via descriptor field?)? 2. Before function, a la javadoc comments, or first thing in body, a la the prologue directive idea?
That's said, if "before", then we should consider white-spaces and newlines after the comment and before the function header (though, as well as in the prologue). If "before", then tones of old code written in javadoc will just work with the new engine. OTOH, this means that the exact syntax of javadocs will be standardized at the level of the spec (and it's a sound thing -- people will have to write exactly e.g. @property that the "doc-er" catch it correctly. OTOH again -- why not? -- to standardize common syntax of documenting functions). However, the spec may not parse the exact content of the comment but just save it as a string, regardless of what is written insides.
A variant with "inside" is also good, underlines that the comment is sort of a function's "property" (I like yours simple "doc" name for that). Both variants are good, just "outside" has the advantage that the old code will just hook on it. 3. Reflected via function .doc property, a Function.extractDocComment static method, or something even more mirror-like/stratified?
Yes, both are fine, ".doc" is great. Perhaps, even global binding help(...). I'm not sure what is best, I lack experience programming in languages with doc-comment or triple-quoted equivalents (Python attached tests, e.g.). Comments (heh) welcome.
Thinking about 1, I would start with function-only.
Yes, this is the main case. For 2 I'm inclined to say "in body" because it's too easy to lose the "before" context during the life of a function, compared to losing part of the body by accident.
At first glance there should be no issues with parsing in both cases. We sure have to restrict that only the first multiline comment from above of a function is treated as doc-comment. It will not touch any other upper-comments
// this one isn't caught
/* and this one also */
/**
- but this is OK */ function foo() {}
console.log(foo.doc); // "but this is OK"
With the prologue -- there also directives' places should be considered. I think the best place for them is under the doc-comment.
function foo() {
"" My function """
"use strict"; } Regarding 3, I bet Function.extractDocComment or a better name wins, especially if the whole solution allows monkey-patching a polyfill for downrev browsers that support source recovery (not SpiderMonkey's decompiler).
Yes, maybe.
Dmitry.
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime. Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs.
Rather than trying to extend the core language, I suggest that this type of functionality is better viewed as simply another form of reflection that would be best modeled as a kind of Mirror. It should manifest it self as a module that provides access to such mirrors. For example, you might access it as
var docMirrorFn = DocMirror.for(fn); var helpStr = docMirrorFn.help(); var helpStrLocalized = docMirrorFn.helpLocale(); var helpStrI18N = docMirrorFn.helpLocale(someLocale); var competeDoc = docMirrorFn.full({format: 'HTML'}); var src = docMirrorFn.sourceCode; // etc.
If a REPL provider wants to offer a "help" command it would implement it by accessing an appropriate documentation mirror.
The definition of the DocMirror module would define how the appropriate meta data is obtained and even alternatives and fall backs for obtaining it that does not require direct embedding in the loaded JS code.
My recommendation would be rather than trying to extend to the core language at this time, experiment with defining such a module. At some point we may be ready to consider "standardizing" such modules but even before that there is nothing preventing community adoption of such a convention.
On 30.08.2011 19:33, Dean Landolt wrote:
On Tue, Aug 30, 2011 at 10:41 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote:
On 30.08.2011 17:41, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote: OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. Some summarized features below: + Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though) + Auto-suggest and hints of parameters and their types when typing function name in console. + Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc). - Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments") As an example I show a spec description for `parseInt` function: 15.1.2.2 parseInt (string , radix) Let [[Documentation]] property of the `parseInt` function be the following string: "The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X." ... x.x.xx help(F) A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though, IMO `help` name isn't used much).
That's tough to say for sure, and even if true still probably isn't good enough. Regardless, there's no excuse for this kind of global, especially in light of a module system. Sure, specifically in a repl you'd probably want to skip the require -- for this node offers up dot-prefixed repl keywords. So something like .help(F) would be a lot safer and nearly as convenience as a help global.
Yes, I guess. Anyway, all this about only "debug-in-console" mode, so .help(...) would be just fine.
I think it would be useful for the committee to weigh in on the idea of repl plugins. This particular design is pretty nice but it conflicts for statements beginning with non-zero-prefixed decimals. In practice I don't see this causing any real problems, but it'd be great if the grammer explicitly forbid this, effectively reserving it for repl hooks.
Maybe, though, in contrast with other langs, ES is different since because of browser vendors has different implementations. Actually, there is no such implementation as "ECMAScript". In contrast, languages such as e.g. Python or Erlang have the sample implementations (usually that which is placed directly on the website of the language), and there of course the attention to a good console and downloadable and installable "in one click" REPLs is paid much. And that's said, this increases and speeds up the learning of the language. I don't know what to propose in case of ECMAScript, since repeat, there is no such language as ECMAScript "in real life" and therefore, there is no "the sample REPL" for which it would be the best to have .help(...) function which shows the doc-comment of a function.
And in case of a browser -- I don't think it's needed much. OTOH, of course devs. also debug the browser code in tools such as Firebug, where they also can use this .help(...).
So I'm still thinking myself, is it worth to move it to the level of the ES spec (the [[Doc]] property and similar) or it's not needed much and is only for "playing in console". OTOH, this "playing in console" is what makes Python the best in this respect.
The main thing I want to clarify at this step, whether we need at all this thing in ES? Since committee keeps silence, I'd like to consider it better as "Yes, we need it and there is nothing to add", rather than "We don't need and don't want even to discuss" (since there is no big sense in later, because the functionality seems useful). However, we need to clarify how *exactly* it's useful. Maybe using JS in console is so rare case and it isn't required much. OTOH, era of client-side-only-JS is behind and JS is also on server now. And from this viewpoint, in the console, it's the best to get the help, hints for parameters and even type checking for functions.
Brenden weighed in and laid out some of the design space. I suspect this functionality can largely be pieced together with what harmony already gives us. As Brenden hinted, quasis give us multiline strings. The doc key is just screaming to be a private name instead (generally true of all wunderbars IMHO). This could be tacked onto functions and objects imperatively, but it would be pretty goofy to put the doc string /after/ its function body. So for this to be serviceable we'd need a declarative way to do this inside the function body. ISTR some syntax being thrown around to set an own-prop on a function from inside its body but I can't recall specifics -- is there anything harmonious for this?
Put the above three bits together with a de facto repl plugin syntax and module mapping and you get a more sanitized (and extensible) version of python's repl with the same developer ergonomics.
Yes, but all this needs to be considered as "needed" first. The lack which I mentioned myself of the feature is that the source code will be increased (lint tools though will provide "Remove doc-comments" for indemnified scripts) and the size of a byte-code to keep additional property of a function.
Dmitry.
On 30.08.2011 19:54, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 10:41 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote:
On 30.08.2011 17:41, Rick Waldron wrote:
On Tue, Aug 30, 2011 at 3:39 AM, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com <mailto:dmitry.soshnikov at gmail.com>> wrote: OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. Some summarized features below: + Ability to get help of any built-in and any user-defined function directly in runtime (in later case, authors should provide the documentations though) + Auto-suggest and hints of parameters and their types when typing function name in console. + Ability even to have sort of guards for types with issuing warnings in case if types mismatch (sort of recent contract.coffee projects and actually other langs, e.g. Haskell, Erlang, etc). - Code size will be increased to store the doc string. This can be optional in case of minimized scripts (lint tools can have option "Remove doc-comments") As an example I show a spec description for `parseInt` function: 15.1.2.2 parseInt (string , radix) Let [[Documentation]] property of the `parseInt` function be the following string: "The parseInt function produces an integer value dictated by interpretation of the contents of the string argument according to the specified radix. Leading white space in string is ignored. If radix is undefined or 0, it is assumed to be 10 except when the number begins with the character pairs 0x or 0X, in which case a radix of 16 is assumed. If radix is 16, number may also optionally begin with the character pairs 0x or 0X." ... x.x.xx help(F) A built-in, global object function property named "help()" will no doubt conflict with userland code
It's derived question, we may choose any name which fits well (though, IMO `help` name isn't used much). The main thing I want to clarify at this step, whether we need at all this thing in ES?
I support this idea :)
BTW, here are some incomplete numbers on "help()" (sort of) www.google.com/codesearch#search/&q=lang:^javascript$ help\(&type=cs www.google.com/codesearch#search/&q=lang:^javascript$ help\(&type=cs
OK, OK, you've convinced me ;) it's used much as I see. Though, would be the best name. Function.getDocumentation(...) is also nice but long. Funciton.getInfo(...) may return an object with meta-info: {documentation: "...", length: 2, arguments: {"a": int, "b": string}}. But don't wanna go down into the bikeshed.
Dmitry.
Right. And this is where Python just wins. In respect of bult-ins and methods provided by framework authors. That's said in the initial letter, we can always go to the language documentation website and read it, but to the help "in-place" seems much better.
Though, there is another issue here (beside the source- and byte- code size increasing). The "offline" documentation embedded inside the function objects may become obsolete, while the online is up-to-date. But, for the specified built-ins it's not the case, since the spec doesn't change too often. And regarding user-defined framework codes, anyway the code is parsed each time -- it is the property and the deal of the JS engine to correctly parse doc-comment and assign it as a property to a function. That is, the documentation may change in-time and if you use external script file, then you always have up-to-date documentation.
Dmitry.
On 30.08.2011 20:39, Allen Wirfs-Brock wrote:
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime. Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs.
Sure, though I don't think that application level code have to be
dependent in logic on documentation string. The doc-string is just the
addition for debug. No more, no less. It's when you use a new lib (my
example with Redis's get
method which should accept callback), you
just get this help directly "from here" without going to some websites
(which can even be down for this time) and solve the problems much
faster. And for minified scripts, I think the option "Remove
doc-comments" is the case.
Rather than trying to extend the core language, I suggest that this type of functionality is better viewed as simply another form of reflection that would be best modeled as a kind of Mirror. It should manifest it self as a module that provides access to such mirrors. For example, you might access it as
var docMirrorFn = DocMirror.for(fn); var helpStr = docMirrorFn.help(); var helpStrLocalized = docMirrorFn.helpLocale(); var helpStrI18N = docMirrorFn.helpLocale(someLocale); var competeDoc = docMirrorFn.full({format: 'HTML'}); var src = docMirrorFn.sourceCode; // etc.
Yeah, why not. Also is a variant. Though, for the end-user I'd like to see all this stuff encapsulated somewhere and to get the needed thing as just .help(...). But, yes, it can be done by a host-environment as a debugger which provides this .help(...) which inside does this long manipulations.
If a REPL provider wants to offer a "help" command it would implement it by accessing an appropriate documentation mirror.
Yes.
The definition of the DocMirror module would define how the appropriate meta data is obtained and even alternatives and fall backs for obtaining it that does not require direct embedding in the loaded JS code.
My recommendation would be rather than trying to extend to the core language at this time, experiment with defining such a module. At some point we may be ready to consider "standardizing" such modules but even before that there is nothing preventing community adoption of such a convention.
Yes, it's not a problem to write a pre-processor of doc-comments
(actually we have many auto-doc generation tools for JavaDocs), i.e.:
to turn:
/**
- connect
- @param {String} host
- @param {Number} port
- Connects to my awesome host */ function connect(host, port) { ... }
into the:
function connect(host, port) { ... } connect.doc = "connect\n at param {String} host\n at param {Number} port\nConnects to my awesome host"
or even into the:
function connect(host, port) {
// ! pre-processor header !!
if (typeof host != "string") console.warning("connect: host must be a string"); if (typeof port != "number") console.warning("connect: port must be a number");
// ! end of pre-processor header !!
// rest of user-defined implementation
}
it's just a variant of course (we need not to show this embedded checker
in the toString
decompilation of a function and to handle prologue
before for directives), but the general idea is this.
It's not the problem to write such a pre-processor, but pre-processor means to do something additional manually by hands before the execution and not all users use some building-tools for that. OTOH, we can write an extension for e.g. Narcissus to handle it on the engine's level and try will it go or not (but something suggests me that it will ;)).
Dmitry.
On Aug 31, 2011, at 1:57 AM, Dmitry A. Soshnikov wrote:
On 30.08.2011 20:39, Allen Wirfs-Brock wrote:
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime. Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs.
Sure, though I don't think that application level code have to be dependent in logic on documentation string. The doc-string is just the addition for debug. No more, no less. It's when you use a new lib (my example with Redis's
get
method which should accept callback), you just get this help directly "from here" without going to some websites (which can even be down for this time) and solve the problems much faster. And for minified scripts, I think the option "Remove doc-comments" is the case.
When designing language features, a designer always has some specific use cases in mind. But that doesn't restrict users of the language from discovering other use cases. Once a feature is incorporated into a language, implementation have to support all possible use cases, not just the original intended use case. That's why it is important for language designer to think broadly about all conceivable uses and feature interaction when extending a language. Sure, no application is required to have logic dependencies upon the content of a documentation string but once they exist any application could have such dependencies. As a language designers you have to think about the implications of such usages. It isn't good enough to just say that wasn't the intended use of the feature.
On 31.08.2011 18:38, Allen Wirfs-Brock wrote:
On Aug 31, 2011, at 1:57 AM, Dmitry A. Soshnikov wrote:
On 30.08.2011 20:39, Allen Wirfs-Brock wrote:
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime. Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs. Sure, though I don't think that application level code have to be dependent in logic on documentation string. The doc-string is just the addition for debug. No more, no less. It's when you use a new lib (my example with Redis's
get
method which should accept callback), you just get this help directly "from here" without going to some websites (which can even be down for this time) and solve the problems much faster. And for minified scripts, I think the option "Remove doc-comments" is the case. When designing language features, a designer always has some specific use cases in mind. But that doesn't restrict users of the language from discovering other use cases. Once a feature is incorporated into a language, implementation have to support all possible use cases, not just the original intended use case. That's why it is important for language designer to think broadly about all conceivable uses and feature interaction when extending a language. Sure, no application is required to have logic dependencies upon the content of a documentation string but once they exist any application could have such dependencies. As a language designers you have to think about the implications of such usages. It isn't good enough to just say that wasn't the intended use of the feature.
Yes, of course it's true, we have to consider the most hardcore use-cases of our programs. Well, then I have to think on it, but the first thing which comes in mind -- already mentioned "Remove doc-comments" option for minifiers, i.e. a user himself is responsible for this action. Moreover, minification is again mostly for browser-scripting, where this help(...) functionality isn't so required as it is for the server programming in the console.
And for the later case I wrote a simple version of such a pre-processor: gist.github.com/1186853
Simple example:
/**
- sum
- @param {Number} x
- @param {Number} y
- The function of summation. */ function sum(x, y) { return x + y; }
/**
- multSum
- @param {Number} x
- @param {Number} y
- @param {String} value
- The function of calculation. */ function calculate(x, y, value) { // comment print(value + ": " + sum(x, y)); }
// ---------- Test documentation ----------
// Result:
// Help on "sum" function: // // /** // * sum // * @param {Number} x // * @param {Number} y // * The function of summation. // */ help(sum);
// Help on "calculate" function: // // /** // * multSum // * @param {Number} x // * @param {Number} y // * @param {String} value // * The function of calculation. // */ help(calculate);
// ---------- Test guards ----------
sum(1, 2); // 3, OK
calculate(1, 2, "value"); // value: 3
sum(1, "2"); // TypeError: "y" must be a number
calculate(1, 2, 4); // TypeError: "value" must be a string
Dmitry.
On 01.09.2011 22:28, Dmitry A. Soshnikov wrote:
On 31.08.2011 18:38, Allen Wirfs-Brock wrote:
On Aug 31, 2011, at 1:57 AM, Dmitry A. Soshnikov wrote:
On 30.08.2011 20:39, Allen Wirfs-Brock wrote:
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript. There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime.
Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs. Sure, though I don't think that application level code have to be dependent in logic on documentation string. The doc-string is just the addition for debug. No more, no less. It's when you use a new lib (my example with Redis'sget
method which should accept callback), you just get this help directly "from here" without going to some websites (which can even be down for this time) and solve the problems much faster. And for minified scripts, I think the option "Remove doc-comments" is the case. When designing language features, a designer always has some specific use cases in mind. But that doesn't restrict users of the language from discovering other use cases. Once a feature is incorporated into a language, implementation have to support all possible use cases, not just the original intended use case. That's why it is important for language designer to think broadly about all conceivable uses and feature interaction when extending a language. Sure, no application is required to have logic dependencies upon the content of a documentation string but once they exist any application could have such dependencies. As a language designers you have to think about the implications of such usages. It isn't good enough to just say that wasn't the intended use of the feature.Yes, of course it's true, we have to consider the most hardcore use-cases of our programs. Well, then I have to think on it, but the first thing which comes in mind -- already mentioned "Remove doc-comments" option for minifiers, i.e. a user himself is responsible for this action.
By the way, another solution is to use another prologue directive:
"use documentation";
or vice-versa, "no documentation"; since it would be better to turn-off the feature, but by default to have it in the console, tests and debug.
P.S.: want to embed the source-code transformation script I gave into Narcissus as an extension (e.g. an option for compilation)?
njs script.js --with-doc
Dmitry.
I also prototyped this idea mainly for node. Also, I figured out a hack with a label, that make it work for spidermonkey.
-- Irakli Gozalishvili Web: www.jeditoolkit.com Address: 29 Rue Saint-Georges, 75009 Paris, France goo.gl/maps/3CHu
On Thursday, 2011-09-01 at 20:28 , Dmitry A. Soshnikov wrote:
On 31.08.2011 18:38, Allen Wirfs-Brock wrote:
On Aug 31, 2011, at 1:57 AM, Dmitry A. Soshnikov wrote:
On 30.08.2011 20:39, Allen Wirfs-Brock wrote:
On Aug 30, 2011, at 12:39 AM, Dmitry A. Soshnikov wrote:
OK, let's up the topic. Seems there are no technical issues in the proposed thing -- we can simply either accept it or not. The question is whether it's needed and sound proposal and the feature to have in ECMAScript.
There certainly are technical issues.
To start, by making this part of the core language you would be defining a feature that can be used by any application code. For example, a program might choose to encode essential runtime metadata as its "documentation". Because of that possibility, the [[Documentation]] information must always be available at runtime. Minimizers or obfuscators could not remote documentation comments because of the possibility of breaking such programs.
Sure, though I don't think that application level code have to be dependent
in logic on documentation string. The doc-string is just the addition for
debug. No more, no less. It's when you use a new lib (my example with
Redis's get
method which should accept callback), you just get this help
directly "from here" without going to some websites (which can even be down
for this time) and solve the problems much faster. And for minified scripts,
I think the option "Remove doc-comments" is the case.
When designing language features, a designer always has some specific use cases in mind. But that doesn't restrict users of the language from discovering other use cases. Once a feature is incorporated into a language, implementation have to support all possible use cases, not just the original intended use case. That's why it is important for language designer to think broadly about all conceivable uses and feature interaction when extending a language. Sure, no application is required to have logic dependencies upon the content of a documentation string but once they exist any application could have such dependencies. As a language designers you have to think about the implications of such usages. It isn't good enough to just say that wasn't the intended use of the feature.
Yes, of course it's true, we have to consider the most hardcore use-cases of our programs. Well, then I have to think on it, but the first thing which comes in mind -- already mentioned "Remove doc-comments" option for minifiers, i.e. a user himself is responsible for this action. Moreover, minification is again mostly for browser-scripting, where this help(...) functionality isn't so required as it is for the server programming in the console.
And for the later case I wrote a simple version of such a pre-processor: gist.github.com/1186853
Simple example:
/**
- sum
- @param {Number} x
- @param {Number} y
- The function of summation. */ function sum(x, y) { return x + y; }
/**
- multSum
- @param {Number} x
- @param {Number} y
- @param {String} value
- The function of calculation. */ function calculate(x, y, value) { // comment print(value + ": " + sum(x, y)); }
// ---------- Test documentation ----------
// Result:
// Help on "sum" function: // // /** // * sum // * @param {Number} x // * @param {Number} y // * The function of summation. // */ help(sum);
// Help on "calculate" function: // // /** // * multSum // * @param {Number} x // * @param {Number} y // * @param {String} value // * The function of calculation. // */ help(calculate);
// ---------- Test guards ----------
sum(1, 2); // 3, OK
calculate(1, 2, "value"); // value: 3
sum(1, "2"); // TypeError: "y" must be a number
calculate(1, 2, 4); // TypeError: "value" must be a string
Dmitry.
On Friday, 2011-09-02 at 22:28 , Brendan Eich wrote:
Can you show your label hack for SpiderMonkey to es-discuss?
Ahh sorry I did not realized I forgot to post link:
On Sep 2, 2011, at 3:29 PM, Irakli Gozalishvili wrote:
On Friday, 2011-09-02 at 22:28 , Brendan Eich wrote:
Can you show your label hack for SpiderMonkey to es-discuss?
Ahh sorry I did not realized I forgot to post link:
Cool! Permit me to cite some of your README content to help promote to those who did not click ;-) -- here it is:
var doc = require('doc').doc doc(doc) // Prints following output:
/* function doc(source) { ... }
Prints documentanion of the given function */
// You can also document your own functions:
function compose() {
doc: "Returns the composition of a list of functions, where each function"
| "consumes the return value of the function that follows. In math"
| "terms, composing the functions f()
, g()
, and h()
produces"
| "f(g(h()))
."
| "Usage:"
| "var greet = function(name) { return 'hi: ' + name }"
| "var exclaim = function(statement) { return statement + '!' }"
| "var welcome = compose(exclaim, greet)"
| "welcome('moe')"
| "//> 'hi: moe!'"
var funcs = Array.prototype.slice.call(arguments) return function composed() { var args = slice.call(arguments) var i = funcs.length while (0 <= --i) args = [ funcs[i].apply(this, args) ] return args[0] } }
On Sep 2, 2011, at 3:41 PM, Brendan Eich wrote:
On Sep 2, 2011, at 3:29 PM, Irakli Gozalishvili wrote:
On Friday, 2011-09-02 at 22:28 , Brendan Eich wrote:
Can you show your label hack for SpiderMonkey to es-discuss?
Ahh sorry I did not realized I forgot to post link:
Cool! Permit me to cite some of your README content to help promote to those who did not click ;-) -- here it is:
var doc = require('doc').doc doc(doc) // Prints following output:
/* function doc(source) { ... }
Prints documentanion of the given function */
// You can also document your own functions:
function compose() { doc: "Returns the composition of a list of functions, where each function" | "consumes the return value of the function that follows. In math" | "terms, composing the functions
f()
,g()
, andh()
produces" | "f(g(h()))
." | "Usage:" | "var greet = function(name) { return 'hi: ' + name }" | "var exclaim = function(statement) { return statement + '!' }" | "var welcome = compose(exclaim, greet)" | "welcome('moe')" | "//> 'hi: moe!'"
Function.toString isn't standardised, and I recall that in the past SM did elide dead code, multiple engines reformat code, so in general this doesnt seem reliable at a library level. It also doesn't work for builtin functions, and I feel we'd want a solution that allows documentation for builtin functions as well.
From the pov of runtime identification, we'd be in the odd position of having to try and identify valid code as being documentation.
This shouldn't be taken as support for this idea (documentation as part of the language) as I feel that this is the type of feature i'd associate with the development environment rather than part of the language.
While we're sharing... I've also written an implementation, albiet much simpler in form then Dmitry's or Irakli's, but this will work in ES today as is, using old-time multi-line and single line comments.
Also, mine will suffer from the same downfall noted in Oliver's comments.. "Function.toString isn't standardised, and I recall that in the past SM did elide dead code"
On Sep 2, 2011, at 3:55 PM, Oliver Hunt wrote:
Function.toString isn't standardised, and I recall that in the past SM did elide dead code, multiple engines reformat code
Also really old builds of JSC had "interesting" output from function.toString()
On Sep 2, 2011, at 3:55 PM, Oliver Hunt wrote:
Function.toString isn't standardised, and I recall that in the past SM did elide dead code, multiple engines reformat code, so in general this doesnt seem reliable at a library level. It also doesn't work for builtin functions, and I feel we'd want a solution that allows documentation for builtin functions as well.
Oh sure, Irakli is just making do with what happens to work. SpiderMonkey doesn't do source recovery, so it needs something special and the label hack Irakli found is "it" (for now!).
This thread was about standardizing something guaranteed that does not depend on toString, but we would love to have a winning library approach whose API we can standardize. Such a library could be scrappy (I hope not crappy ;-) about using "what happens to work" on particular engines and even versions.
This shouldn't be taken as support for this idea (documentation as part of the language) as I feel that this is the type of feature i'd associate with the development environment rather than part of the language.
That's the other thing about a library approach: it doesn't need to land in language, but then the IDE may have to grow a full parser (probably good IDEs already have one).
On Saturday, 2011-09-03 at 24:55 , Oliver Hunt wrote:
On Sep 2, 2011, at 3:41 PM, Brendan Eich wrote:
On Sep 2, 2011, at 3:29 PM, Irakli Gozalishvili wrote:
On Friday, 2011-09-02 at 22:28 , Brendan Eich wrote:
Can you show your label hack for SpiderMonkey to es-discuss?
Ahh sorry I did not realized I forgot to post link:
Gozala/doc Cool! Permit me to cite some of your README content to help promote to those who did not click ;-) -- here it is:
var doc = require('doc').doc doc(doc) // Prints following output: /* function doc(source) { ... } ----------------------------------------------- Prints documentanion of the given function */ // You can also document your own functions: function compose() { doc: "Returns the composition of a list of functions, where each function" | "consumes the return value of the function that follows. In math" | "terms, composing the functions
f()
,g()
, andh()
produces" | "f(g(h()))
." | "Usage:" | "var greet = function(name) { return 'hi: ' + name }" | "var exclaim = function(statement) { return statement + '!' }" | "var welcome = compose(exclaim, greet)" | "welcome('moe')" | "//> 'hi: moe!'"Function.toString isn't standardised, and I recall that in the past SM did elide dead code, multiple engines reformat code, so in general this doesnt seem reliable at a library level.
Yeap toString is not reliable, but hey it works just fine on most engines I know ;) Also SM strips out dec code that's why I use label hack so that it keeps it :)
It also doesn't work for builtin functions, and I feel we'd want a solution that allows documentation for builtin functions as well.
In fact doc
function will work with built-ins just fine, even though output is not that useful:
On Sep 2, 2011, at 3:29 PM, Irakli Gozalishvili wrote:
I want to second Brendan on the coolness of this. It really is a JavaScript derived solution rather than simply copying what somebody else did for some other language.
I have a couple of thoughts to throw in.
The documentation description doesn't need to be a single structured string. It could also be something like:
function sum(x, y) { doc: { name: "sum"; parameters: { x: "first value to sum"; expected: "Number"; y: "second value to sum"; expected: "Number"; } returns: "Number"; description: "Compute the sum of two numbers"; docUrl: "maysite.com/myapp/designDoc.html#sum"; }
return x + y; }
This doc blocks looks something like an object literal but is really composed of nested labelled blocks and statements. However, it would be pretty easy to translate such a structure into a JSON object structure.
Whether a string or a structured labeled block is used, if we hope for multiple tools and IDEs to support it then somebody needs to propose and promote a specific documentation schema.
Oliver is correct that you can't depend upon Function.prototype.toString actually reproduce such a structure, but it is a pretty good fallback source for a runtime tool to try when it can't find the documentation anywhere else.
Earlier in this thread I was stressing that it is desirable to make it impossible for application logic to have dependencies upon such metadata. This approach satisfies that requirement. In fact, if this convention was established I would expect minimizers to strip out any such documentation.
Given that you can't depend upon the documentation to be in the actual downloaded source code runtime tools need a convention for finding the documentation. I suggest above that such a documentation block could easily be translated into JSON. Such a convention might be something like that given a executed file located at "http:somepath/foo.js" or "http:somepath/foo.min.js" the file "http:somepath/foo.js.doc.json" contains the documentation objects for the file. Of source, someone needs to write a documentation extractor to produce such files.
On Sep 6, 2011, at 12:36 AM, Dmitry Soshnikov wrote:
So, IMO we can't build a "library on hacks" which we then standardize. If to implement, then the best approach at the lower level, as Python did (with processing at compilation time).
I see no reason why we can't build up a library or competing libraries first, then standardize. Sure, it would be easier for the library implementors if there were standard low-level doc-string or doc-comment (or doc-data as Allen just posted) plumbing.
But JS can host all sorts of hacks to front a uniform cross-browser/version API -- this has been going on at least since the "Ajax" days.
The hard thing about standardizing first is that TC39 won't do as good a job as competing library authors contending for users, but TC39 de-jure standardization will result in everyone being stuck with whatever we invent.
Did you have a concrete lower-level suggestion that does not depend on normative language prohibiting dead code elimination combined with Function.prototype.toString() standardization?
On Sep 6, 2011, at 1:52 PM, Dmitry Soshnikov wrote:
(1) to standardize
toString
(for this particular case -- do not remove comments inside);If the (1) is not possible (why by the way?),
Because comments are not saved in the compilation process and doing so would slow parsing down and take more space. It's not obvious this would matter in head-to-head competition with other browsers (esp. with minified benchmarks) -- we would have to find out.
Switching to source recovery will entrain more space but may be tolerable -- except that switching to source recovery is work, competing with other demands. There's no free lunch.
Anyway, that is the SpiderMonkey perspective. I'm not sure about Rhino or other engines that do not already retain the source buffer rather than implement a decompiler.
At a higher level, ECMA-262 has underspecified Function.prototype.toString forever. Jumping to the other extreme when there's a simpler and more direct solution seems like a mistake.
then we can:
(2) either to change grammar for function declarations/expression (to add Documentation production before the
function
token; though, not so convinient for FE, when e.g. foo: function () { ... })
I'm not sure what you mean in that parenthetical aside. We could say something much simpler, e.g. any labeled expression statement in prologue position must be retained (by whatever means) such that it appears in the toString result for the function containing that prologue.
Simpler specs that do not imply one and only one implementation technique are usually better.
(3) make a pre-parsing of doc-comments and to inject them as a properties of functions. I gave an example of such a pre-processor (just a small prototype, but it can generate the docs and type-guards) -- gist.github.com/1186853
How would this be part of the standard?
Remember browsers are not IDEs. They have to load code, including some comments, very quickly. They currently either parse and drop comments (which lexically are space)
On 04.09.2011 23:45, Brendan Eich wrote:
On Sep 6, 2011, at 1:52 PM, Dmitry Soshnikov wrote:
(1) to standardize
toString
(for this particular case -- do not remove comments inside);If the (1) is not possible (why by the way?), Because comments are not saved in the compilation process and doing so would slow parsing down and take more space. It's not obvious this would matter in head-to-head competition with other browsers (esp. with minified benchmarks) -- we would have to find out.
Switching to source recovery will entrain more space but may be tolerable -- except that switching to source recovery is work, competing with other demands. There's no free lunch.
Anyway, that is the SpiderMonkey perspective. I'm not sure about Rhino or other engines that do not already retain the source buffer rather than implement a decompiler.
Yes, thanks; sounds logical and quite obvious. Though, one can save only doc-comment at compilation; not all.
At a higher level, ECMA-262 has underspecified Function.prototype.toString forever. Jumping to the other extreme when there's a simpler and more direct solution seems like a mistake.
Well, then we don't have a reliable way of implementing such a runtime
library -- no matter how syntactically to represent the documentation:
vars, labels, "labels aks JSON", etc -- if some implementation is
allowed to return any result in toString
decompilation, we can't be
sure. But, for such implementation we may simply show "No documentation
is available" of course, which will be a signal to fix an implementation
then -- in case if a feature is wide-spread.
then we can:
(2) either to change grammar for function declarations/expression (to add Documentation production before the
function
token; though, not so convinient for FE, when e.g. foo: function () { ... }) I'm not sure what you mean in that parenthetical aside.
I meant grammar for function declarations -- to add optional
Documentation
production.
FunctionDeclaration : Documentation<opt> function Identifier ( FormalParameterListopt ) {
FunctionBody }
where Documentation is /* doc string */
However, it's just a variant. That's said, it's not so easy to catch function expressions in this way, but they are also need somehow to be documented.
var foo = { /** * my funciton */ bar: function () { ... } }
on typing help(foo.bar)
I'd like to see "/**\n* my function\n*/", but
it's not obvious for grammar at all. Moreover, a function can be
assigned to a property (being defined in completely different syntactic
position).
From this viewpoint of course inner comments (from which I actually
started the topic showing Python's triple-doc-strings inside methods) is
better. This approach even allow not to define own doc property, but
place internal method on Function.prototype
and activation get
this.[[Doc]]
.
All discussed "hacks" -- var doc = " ...", or label doc: " ..." are
fine. Though if we had have toString
's guarantee it would be even better.
We could say something much simpler, e.g. any labeled expression statement in prologue position must be retained (by whatever means) such that it appears in the toString result for the function containing that prologue.
Maybe. But the main thing -- is a convenience of a programmer. An application, practical programmer should type less, do more. In the best variant, old code should just work also. I switched to JavaDocs since, as mentioned, many of source are written using it (though, initially talked about inner-docs). If we can provide a convenient way of describing docs, as I said, exact syntax is not so essential for me; the convenient feature -- that's what is essential.
Simpler specs that do not imply one and only one implementation technique are usually better.
(3) make a pre-parsing of doc-comments and to inject them as a properties of functions. I gave an example of such a pre-processor (just a small prototype, but it can generate the docs and type-guards) -- gist.github.com/1186853 How would this be part of the standard?
No, not this; this is just a prototype. Though, it relates to (1) -- an
optional Documentation
non-terminal for FD, which is just parsed.
However, that's said, it's hard for FE, so I don't completely like it
yet -- just wanted to provide a variant which would support all old code
also.
Remember browsers are not IDEs. They have to load code, including some comments, very quickly. They currently either parse and drop comments (which lexically are space) on the floor, or else retain whole source buffers or slices of source buffers for F.p.toString. TC39 is not going to add overhead to this process lightly.
OK, I understand and think it's sensible. And as I mentioned in previous letter, I'd better recommend the feature to Node.js only for now. Not to ECMAScript. Since the most use-cases are related with Node's console.
Dmitry.
On 4 September 2011 21:45, Brendan Eich <brendan at mozilla.com> wrote:
On Sep 6, 2011, at 1:52 PM, Dmitry Soshnikov wrote:
(1) to standardize
toString
(for this particular case -- do not remove comments inside);If the (1) is not possible (why by the way?),
Because comments are not saved in the compilation process and doing so would slow parsing down and take more space. It's not obvious this would matter in head-to-head competition with other browsers (esp. with minified benchmarks) -- we would have to find out.
Switching to source recovery will entrain more space but may be tolerable -- except that switching to source recovery is work, competing with other demands. There's no free lunch.
Plus, it breaks all function-based data abstraction if you can reliably reflect on its implementation and then even reify it through eval.
I am indifferent about the general idea of a doc interface, but: having to peek at the implementation of something (which is what toString does) in order to gather its interface description sounds like a fundamental violation of basic principles and exactly the wrong way to go about it.
On 05.09.2011 13:26, Andreas Rossberg wrote:
On 4 September 2011 21:45, Brendan Eich <brendan at mozilla.com <mailto:brendan at mozilla.com>> wrote:
On Sep 6, 2011, at 1:52 PM, Dmitry Soshnikov wrote: > (1) to standardize `toString` (for this particular case -- do not remove comments inside); > > If the (1) is not possible (why by the way?), Because comments are not saved in the compilation process and doing so would slow parsing down and take more space. It's not obvious this would matter in head-to-head competition with other browsers (esp. with minified benchmarks) -- we would have to find out. Switching to source recovery will entrain more space but may be tolerable -- except that switching to source recovery is work, competing with other demands. There's no free lunch.
Plus, it breaks all function-based data abstraction if you can reliably reflect on its implementation and then even reify it through eval.
In other words, if to rephrase you, you nevertheless are talking about
standardization of toString
, but to make it some like this, right? :
Function.prototype.toString = function () { throw "Don't break out abstractions, malicious hacker!" };
;)
I am indifferent about the general idea of a doc interface,
Then I don't think incorrect judging about the concept of an "abstraction" is a good topic in this thread (you may open a new one). Abstraction is about abstraction, it's not about "security". Especially in the interpreted dynamically typed language with embedded reflection. Abstractions are for programmers. Not for "hackers".
Anyway. It's a completely different story.
but: having to peek at the implementation of something (which is what toString does) in order to gather its interface description sounds like a fundamental violation of basic principles and exactly the wrong way to go about it.
Fundamental principle of an abstraction, once again, is the abstraction. That is, to provide convenient black box with API hiding implementation details. Thus, nobody says it's about security and that we can't use things directly if needed. Of course it's a bad practice to rely on the internal state. But in case of to string it's not even about application level. It's just a meta-reflection. Yes, achieved via such way. If it had been implemented at lower level (about what I was talking about), then we could get it via reflection API, e.g. Function.getDoc(fn).
However, I already mentioned, I already think it makes sense only for technologies such as Node.js (i.e. with big standard library on which it's needed often to get quickly help), not ECMAScript at its core.
Dmitry.
On 5 September 2011 16:12, Dmitry Soshnikov <dmitry.soshnikov at gmail.com> wrote:
** On 05.09.2011 13:26, Andreas Rossberg wrote:
I am indifferent about the general idea of a doc interface,
Then I don't think incorrect judging about the concept of an "abstraction" is a good topic in this thread (you may open a new one). Abstraction is about abstraction, it's not about "security". Especially in the interpreted dynamically typed language with embedded reflection. Abstractions are for programmers. Not for "hackers".
Well, properly maintaining language-level abstractions is a prerequisite for security, but security is by no means the only reason to want abstraction. Reliability and modularity are others (and the ones that I personally care about even more). In any case, I don't think you want to introduce or bless another feature that interferes badly with either of these concerns -- it's almost guaranteed to lead to legacy that will cause a big headache later.
but: having to peek at the implementation of something (which is what toString does) in order to gather its interface description sounds like a fundamental violation of basic principles and exactly the wrong way to go about it.
Fundamental principle of an abstraction, once again, is the abstraction. That is, to provide convenient black box with API hiding implementation details.
I was specifically referring to the separation of interface and implementation. Documentation is interface description, intended to avoid having to look at the implementation. Wouldn't it be ironic if the mandatory way to get to it was by looking at the implementation?
What about a simple property?
readFile.documentation = { name: 'readFile', arguments: [ { name: 'path', type: 'string' }, { name: 'encoding', type: 'string', optionnal: true }, { name: 'callback', type: 'function' } ], description: 'Reads a file and calls the provided callback with the content of the file as first argument.' }; function readFile( path, encoding/?/, callback ) { var length = arguments.length; if ( length <= 1 ) { throw new TypeError( ); } else if ( length <= 2 ) { callback = encoding; encoding = 'utf-8'; } // do some stuff console.log( path, encoding, callback ); }
And that would also work on objects:
var API = { documentation: { name: 'API', properties: [ { name: 'url', type: 'string', description: 'The current url.'; }, { name: 'goToGoogle', type: 'function', description: 'Change current url to Google.' } ] }, url: '', goToGoogle: function ( ) { this.url = 'www.google.com'; } };
In the example implementation that I created, toString() was used exclusively because it "worked" in the single environment that I tested in and gave me the minimum that I needed to produce a working proof of concept - not to show any kind of support for a better specified Function.prototype.toString()
-- Sent from my Palm Pre On Sep 5, 2011 5:27 AM, Andreas Rossberg <rossberg at google.com> wrote:
On 4 September 2011 21:45, Brendan Eich <brendan at mozilla.com> wrote:
On Sep 6, 2011, at 1:52 PM, Dmitry Soshnikov wrote:
(1) to standardize
toString
(for this particular case -- do not remove comments inside);
If the (1) is not possible (why by the way?),
Because comments are not saved in the compilation process and doing so would slow parsing down and take more space. It's not obvious this would matter in head-to-head competition with other browsers (esp. with minified benchmarks) -- we would have to find out.
Switching to source recovery will entrain more space but may be tolerable -- except that switching to source recovery is work, competing with other demands. There's no free lunch.
Plus, it breaks all function-based data abstraction if you can reliably reflect on its implementation and then even reify it through eval. I am indifferent about the general idea of a doc interface, but: having to peek at the implementation of something (which is what toString does) in order to gather its interface description sounds like a fundamental violation of basic principles and exactly the wrong way to go about it.
On 03.09.2011 3:07, Brendan Eich wrote:
On Sep 2, 2011, at 3:55 PM, Oliver Hunt wrote:
Function.toString isn't standardised, and I recall that in the past SM did elide dead code, multiple engines reformat code, so in general this doesnt seem reliable at a library level. It also doesn't work for builtin functions, and I feel we'd want a solution that allows documentation for builtin functions as well. Oh sure, Irakli is just making do with what happens to work. SpiderMonkey doesn't do source recovery, so it needs something special and the label hack Irakli found is "it" (for now!).
This thread was about standardizing something guaranteed that does not depend on toString, but we would love to have a winning library approach whose API we can standardize. Such a library could be scrappy (I hope not crappy ;-) about using "what happens to work" on particular engines and even versions.
If it had been the main goal, I wouldn't start the topic. Because I remember perfectly we already found before several hacks which "just work" right now in all (modern) engines. One of those:
function foo(x, y) { var doc = " ... "; }
though, of course places additional variable inside. However, from this viewpoint it doesn't matter about what developers will deal -- whether to write doc: " ..." | " ..." | " ..." or var doc = " ...". Completely doesn't matter. Since if they deal, they will just use an approach without worrying that e.g. name doc can be used inside. It can't if the deal.
So all these hacks are fine for ad-hoc local experiments. Though, of course may lead to wide-spreading. In this case we should think which format of a documentation is better to chose (will you explain later why so "strange" syntactic form of a doc-string was chosen -- that's because we couldn't find better "hack" when designing a "library"?).
Considered JavaDoc that's said would "just work" for tones of old code and automatically get support of the documentation and type-guards. OTOH, non-comment docs (such as vars or lables) allow not to worry about minifiers which won't remove them. But from this viewpoint it again doesn't matter how to leave this doc-string in the source code -- either with an option of a minifier or with a non-doc construct which will be skipped.
So, IMO we can't build a "library on hacks" which we then standardize. If to implement, then the best approach at the lower level, as Python did (with processing at compilation time).
If course we don't plan to include the feature for standardization (as Oliver noted; though, I'm not sure he worked with Python and couldn't appreciate its simplicity of getting the documentation on exactly built-ins and also frameworks), they yes -- it can be a simple ad-hoc library for local projects (because it will be hard to switch people from used JavaDocs to "hack"-versions of documentation; besides, an old code no one will want to rewrite for this).
Dmitry.
On 03.09.2011 22:53, Allen Wirfs-Brock wrote:
On Sep 2, 2011, at 3:29 PM, Irakli Gozalishvili wrote:
I want to second Brendan on the coolness of this. It really is a JavaScript derived solution rather than simply copying what somebody else did for some other language.
Ah, come ooooon! >_* Are we here talking about a "coolness"?
Seems we started from the beginning that we have some problems that:
(1) toString
is not standardized and (2) a consequence from (1)
On 04.09.2011 0:31, Brendan Eich wrote:
On Sep 6, 2011, at 12:36 AM, Dmitry Soshnikov wrote:
So, IMO we can't build a "library on hacks" which we then standardize. If to implement, then the best approach at the lower level, as Python did (with processing at compilation time). I see no reason why we can't build up a library or competing libraries first, then standardize. Sure, it would be easier for the library implementors if there were standard low-level doc-string or doc-comment (or doc-data as Allen just posted) plumbing.
But JS can host all sorts of hacks to front a uniform cross-browser/version API -- this has been going on at least since the "Ajax" days.
The hard thing about standardizing first is that TC39 won't do as good a job as competing library authors contending for users, but TC39 de-jure standardization will result in everyone being stuck with whatever we invent.
Sure, it's true, the way "to standardize de-facto things" is very efficient and practical. Though, sometimes without low-level resources it's not always possible to implement a library in an elegant way. And to standardize "hacks"... don't know. Not sure it's the best idea.
Did you have a concrete lower-level suggestion that does not depend on normative language prohibiting dead code elimination combined with Function.prototype.toString() standardization?
I have several concrete lower-level proposals:
(1) to standardize toString
(for this particular case -- do not remove
comments inside);
If the (1) is not possible (why by the way?), then we can:
(2) either to change grammar for function declarations/expression (to
add Documentation production before the function
token; though, not so
convinient for FE, when e.g. foo: function () { ... })
(3) make a pre-parsing of doc-comments and to inject them as a properties of functions. I gave an example of such a pre-processor (just a small prototype, but it can generate the docs and type-guards)
What about to standardize (Python's) "doc"-umentation comments for JS? Not exactly in this name with underscores, but the idea itself. I remember recent discussion on Twitter where Brendan proposed to use a prologue string for that:
function foo() { "this is the super function" }
then having some util which can analyze string decompilation of the function shows the documentation: Object.getDocumentation(foo) -> "this
is the super function".
However, engines usually optimize this case and remove "dead-code". Moreover,
toString
for functions is not standardized also. So all these are just toys and examples. In contrast -- Python's doc make this documentation as a property of function. It can be some internal property -- no matter, for that Object.getDoc(fn) will read this [[Documentation]] stuff.Why did I recall it? -- a real practical case -- just installed redis k-w database to play and wanted to see the documentation right from the node's console -- but... can't. There's no such an ability in JavaScript -- which is the best in Python.
So if to accept triple-string and comments we catch two things at once (after all, triple-strings are useful by themselves):
function foo() { """ Still the stuff is super. @author Me And double "quotes" are not needed to escape, as well as 'single' """
// implementation
}
Object.getDoc(foo) -> "Still the stuff is super.\n at author MeAnd double "quotes" are not needed to escape,\nas well as 'single'"
Huh?
Dmitry.