Proposal: safeEval

# doodad-js Admin (a month ago)

I take a chance to valorize "eval" again by proposing "safeEval".

function safeEval(expression, [locals], [options]) {

            ......

};

So that you can:

safeEval("1 + a", {a: 2}); // returns "3"

safeEval("1 + a()", {a: function() {return 2}}, {allowFunctions: true}); // also returns "3"

but:

safeEval("1 + a()", {a: function() { return 2}}); // throws whatever you want because "allowFunctions" is denied

etc.

Note that local variables are specified in argument. Also note that "options" mainly gives/denies permissions. I'm not sure if we should be whitelisting or blacklisting features there though, or a mix of default enabled and disabled ones...

Very incomplete, but as for inspiration (and very useful to me): www.npmjs.com/package/@doodad-js/safeeval

Claude


This email has been checked for viruses by AVG. www.avg.com

# Jordan Harband (a month ago)

What value does this add, when you can already do function safeEval(...args) { return Function(...args)(); }, or similar?

# doodad-js Admin (a month ago)

Thanks,

There is an option to allow/disallow some aspects of the language, and you can provide the local variables you want to the expression.

Claude

From: Jordan Harband <ljharb at gmail.com>

Sent: Tuesday, June 19, 2018 10:59 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: es-discuss <es-discuss at mozilla.org>

Subject: Re: Proposal: safeEval

What value does this add, when you can already do function safeEval(...args) { return Function(...args)(); }, or similar?

On Tue, Jun 19, 2018 at 7:29 PM, doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

Hi,

I take a chance to valorize “eval” again by proposing “safeEval”.

function safeEval(expression, [locals], [options]) {

            ......

};

So that you can:

safeEval(“1 + a”, {a: 2}); // returns “3”

safeEval(“1 + a()”, {a: function() {return 2}}, {allowFunctions: true}); // also returns “3”

but:

safeEval(“1 + a()”, {a: function() { return 2}}); // throws whatever you want because “allowFunctions” is denied

etc.

Note that local variables are specified in argument. Also note that “options” mainly gives/denies permissions. I’m not sure if we should be whitelisting or blacklisting features there though, or a mix of default enabled and disabled ones...

Very incomplete, but as for inspiration (and very useful to me): www.npmjs.com/package/@doodad-js/safeeval

Claude

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com

# Mike Samuel (a month ago)

How would this compare to tc39/proposal-frozen-realms ?

I'm not sure how to run @doodad-js/safeeval in node since require doesn't provide obvious access to safeeval, but the code seems to do AST filtering. What does it do for inputs like

safeEval(' 0..constructor.constructor("alert(1)")() ')
safeEval(' 0[x][x]`alert(1)`() ', { x: 'constructor' })
safeEval(' 0[x][y] = null ', { x: 'prototype', y: 'toString' })
# doodad-js Admin (a month ago)

From: Claude Petit <petc at webmail.us>

Sent: Wednesday, June 20, 2018 2:23 PM To: 'mikesamuel at gmail.com' <mikesamuel at gmail.com>

Cc: 'es-discuss' <es-discuss at mozilla.org>

Subject: RE: Proposal: safeEval

Thanks,

How would this compare to tc39/proposal-frozen-realms ?

I was not aware of that proposal or didn’t pay attention.I think “safeEval” provides ACLs, while your proposal don’t.

I'm not sure how to run @doodad-js/safeeval in node

That’s a Doodad module, and it must be loaded by Doodad before usage. Anyway, that’s not important.

What does it do for inputs like

Sure, as I told you, that’s very incomplete. I’m just bringing the idea. But I’ll fix that issues if they are present.

Claude

From: Mike Samuel <mikesamuel at gmail.com <mailto:mikesamuel at gmail.com> >

Sent: Wednesday, June 20, 2018 9:51 AM To: doodadjs at gmail.com <mailto:doodadjs at gmail.com>

Cc: es-discuss <es-discuss at mozilla.org <mailto:es-discuss at mozilla.org> >

Subject: Re: Proposal: safeEval

How would this compare to tc39/proposal-frozen-realms ?

I'm not sure how to run @doodad-js/safeeval in node since require doesn't

provide obvious access to safeeval, but the code seems to do AST filtering.

What does it do for inputs like

safeEval(' 0..constructor.constructor("alert(1)")() ')

safeEval(' 0[x][x]`alert(1)`() ', { x: 'constructor' })

safeEval(' 0[x][y] = null ', { x: 'prototype', y: 'toString' })

On Tue, Jun 19, 2018 at 10:29 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

Hi,

I take a chance to valorize “eval” again by proposing “safeEval”.

function safeEval(expression, [locals], [options]) {

            ......

};

So that you can:

safeEval(“1 + a”, {a: 2}); // returns “3”

safeEval(“1 + a()”, {a: function() {return 2}}, {allowFunctions: true}); // also returns “3”

but:

safeEval(“1 + a()”, {a: function() { return 2}}); // throws whatever you want because “allowFunctions” is denied

etc.

Note that local variables are specified in argument. Also note that “options” mainly gives/denies permissions. I’m not sure if we should be whitelisting or blacklisting features there though, or a mix of default enabled and disabled ones...

Very incomplete, but as for inspiration (and very useful to me): www.npmjs.com/package/@doodad-js/safeeval

Claude

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com

# Mike Samuel (a month ago)

On Wed, Jun 20, 2018 at 2:26 PM doodad-js Admin <doodadjs at gmail.com> wrote:

I was not aware of that proposal or didn’t pay attention.I think “safeEval” provides ACLs, while your proposal don’t.

Neither the realms proposal nor the frozen realms proposal include ACLs.

Where are the ACLs in safeeval? I see some privileges via options at L72-75 doodadjs/doodad-js-safeeval/blob/e895e88ff6100dcdd15edfcedff4bd85aaec099c/src/common/Tools_SafeEval.js#L72-L75 : const preventAssignment = types.get(options, 'preventAssignment', true), allowFunctions = types.get(options, 'allowFunctions', false), // EXPERIMENTAL allowNew = types.get(options, 'allowNew', false), // EXPERIMENTAL allowRegExp = types.get(options, 'allowRegExp', false); // EXPERIMENTAL but, as I understand the term, ACLs are usually the set of privileges available to a principal, the rows in an access control matrix. How are you defining "principal?"

# Isiah Meadows (a month ago)

Just a quick read, but that's a terrible set of ACLs, and I strongly dislike the idea in general. That utility is trivially broken in multiple ways (won't announce them on-list, but I've emailed Claude privately), and I'm pretty convinced the idea itself is broken. Limiting syntax is incredibly ineffective for anything security-related, because there are an infinite number of ways to express something in JS, but only a finite number of ways you can realistically limit it without breaking it for normal users or just disabling scripting altogether. It also doesn't stop them from accessing various globals to screw with you.

To give a concrete example of why syntactic analysis is a bad idea for security, let's consider eBay's encounter with JSFuck 1, 2. Because that literally uses only six seemingly benign characters, [, ], !, +, (, and ), you can only protect against it by disallowing calls, which make general use nearly impossible. It was difficult enough that eBay initially gave up 2, until it resulted in rampant, virtually untraceable fraud in the wild 3.

Now, if you disallow parentheses, you also have to ban assignment if any of your scripts has an ID 4, because attackers can use that to their advantage to accomplish the same objective. Claude has an option for that in his library, but it's not especially obvious you'd need it to prevent arbitrary code execution.

Frozen realms together with closures provide privilege separation through offering capabilities, which addresses who can read and/or write what. Capabilities are better than ACLs when it comes to security, because if you limit what they can try, they can't do what they can't try. They can't read what they can't even try to access.

If you want real security, focus on what people can try, not what they can do. And this is why I say this entire proposal is complete and utter crap.


Isiah Meadows me at isiahmeadows.com, www.isiahmeadows.com

# doodad-js Admin (a month ago)

I don't want to propose you my library, I want to propose you the idea.

# Mike Samuel (a month ago)

How can we discuss your idea separately from the library?

You talk about options and ACLs but the only hint as to what those might mean is the library.

How would the idea work if not by tree filtering? AdSAFE did that but writing AdSAFE was very different from writing vanilla JS.

# Isiah Meadows (a month ago)

I did get a bit out of hand with a few parts. I wanted to clarify that the idea itself is IMHO a really bad idea. To emphasize a part:

That utility is trivially broken in multiple ways (won't announce them on-list, but I've emailed Claude privately), and I'm pretty convinced the idea itself is broken.

That last "and" should've been a "but". I meant to place greater emphasis on the second part, not the first.


Isiah Meadows me at isiahmeadows.com, www.isiahmeadows.com

# doodad-js Admin (a month ago)

Thanks

How can we discuss your idea separately from the library?

I’m more thinking at the runtime level than at the “user land”. To be honest, I don’t care of “safeEval” on “user land”.

You talk about options and ACLs but the only hint as to what those might mean is the library

How would the idea work if not by tree filtering? AdSAFE did that but writing AdSAFE was very different from writing vanilla JS.

Yeah, sorry. The purpose is to offer something like “opcode” filtering, but in a more expressive and user-friendly way.

Claude

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Wednesday, June 20, 2018 9:37 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

How can we discuss your idea separately from the library?

You talk about options and ACLs but the only hint as to what those might mean is the library.

How would the idea work if not by tree filtering? AdSAFE did that but writing AdSAFE was very different from writing vanilla JS.

On Wed, Jun 20, 2018, 9:12 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

I don't want to propose you my library, I want to propose you the idea.

# Mike Samuel (a month ago)

On Wed, Jun 20, 2018 at 9:52 PM doodad-js Admin <doodadjs at gmail.com> wrote:

Thanks

How can we discuss your idea separately from the library?

I’m more thinking at the runtime level than at the “user land”. To be honest, I don’t care of “safeEval” on “user land”.

You seem to be asking for criticism, but seem to not want criticism of the only thing that has enough detail for criticism.

You talk about options and ACLs but the only hint as to what those might mean is the library

How would the idea work if not by tree filtering? AdSAFE did that but writing AdSAFE was very different from writing vanilla JS.

Yeah, sorry. The purpose is to offer something like “opcode” filtering, but in a more expressive and user-friendly way.

EcmaScript is specified as a tree interpreter that produces completion records, not in terms of an ISA. The spec does not define opcodes, and you've provided no reason to believe that opcode filtering would provide a better balance between security and ease of writing than AST filtering.

Having written a JS sandbox, I'm skeptical that either approach would work. All successful approaches have combined static analysis with at least 2 of

  1. large dedicated runtime libraries,
  2. source code rewriting, and
  3. separation/isolation via realm/origin/worker. Any pair of these are going to require detailed correctness arguments to pass muster.

I don't see how we could compare the benefits of your proposal to any other without a lot more detail.

# doodad-js Admin (a month ago)

Fixed at 4.1.0-beta.2.

doodadjs/doodad-js-safeeval/commit/e6b05a4b07e834f260faeab4890f57e9eaabf763

Thank you for reporting.

Claude


This email has been checked for viruses by AVG. www.avg.com

# doodad-js Admin (a month ago)

Thanks,

You seem to be asking for criticism, but seem to not want criticism of the only thing that has enough detail for criticism.

I was not asking for criticism, I was just submitting an idea (with an opened, non-destructive discussion).

The spec does not define opcodes

I know

you've provided no reason to believe that opcode filtering would provide a better balance between security and ease of writing than AST filtering

AST filtering is fragile because every change on the language can break it.

Claude

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Thursday, June 21, 2018 9:04 AM To: doodadjs at gmail.com Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

On Wed, Jun 20, 2018 at 9:52 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

Thanks

How can we discuss your idea separately from the library?

I’m more thinking at the runtime level than at the “user land”. To be honest, I don’t care of “safeEval” on “user land”.

You seem to be asking for criticism, but seem to not want criticism of the only thing that has enough detail for criticism.

You talk about options and ACLs but the only hint as to what those might mean is the library

How would the idea work if not by tree filtering? AdSAFE did that but writing AdSAFE was very different from writing vanilla JS.

Yeah, sorry. The purpose is to offer something like “opcode” filtering, but in a more expressive and user-friendly way.

EcmaScript is specified as a tree interpreter that produces completion records, not in terms of an ISA.

The spec does not define opcodes, and you've provided no reason to believe that opcode filtering would provide a better balance between security and ease of writing than AST filtering.

Having written a JS sandbox, I'm skeptical that either approach would work.

All successful approaches have combined static analysis with at least 2 of

  1. large dedicated runtime libraries,

  2. source code rewriting, and

  3. separation/isolation via realm/origin/worker.

Any pair of these are going to require detailed correctness arguments to pass muster.

I don't see how we could compare the benefits of your proposal to any other without a lot more detail.

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com


This email has been checked for viruses by AVG. www.avg.com

# Mike Samuel (a month ago)

On Fri, Jun 22, 2018, 4:21 PM doodad-js Admin <doodadjs at gmail.com> wrote:

you've provided no reason to believe that opcode filtering would provide a better balance between security and ease of writing than AST filtering

AST filtering is fragile because every change on the language can break it.

If you blacklist.

Yet you're providing a library that does just that and have still provided no reason to believe that an opcode filtering proposal would be both more secure and less brittle.

# doodad-js Admin (a month ago)

Thanks,

If you blacklist.

Blacklisting or whitelisting, that’s an open discussion.

Yet you're providing a library that does just that

Because that’s a “user land” library and currently the only way is with “AST filtering”, apart from compiling a complete runtime, with Emscripten or else.

Claude

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Friday, June 22, 2018 4:46 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

On Fri, Jun 22, 2018, 4:21 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

you've provided no reason to believe that opcode filtering would provide a better balance between security and ease of writing than AST filtering

AST filtering is fragile because every change on the language can break it.

If you blacklist.

Yet you're providing a library that does just that and have still provided no reason to believe that an opcode filtering proposal would be both more secure and less brittle.

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com


This email has been checked for viruses by AVG. www.avg.com

# Mike Samuel (a month ago)

On Fri, Jun 22, 2018, 4:56 PM doodad-js Admin <doodadjs at gmail.com> wrote:

Thanks,

If you blacklist.

Blacklisting or whitelisting, that’s an open discussion.

It really isn't.

Yet you're providing a library that does just that

Because that’s a “user land” library and currently the only way is with “AST filtering”, apart from compiling a complete runtime, with Emscripten or else.

No it isn't. As I mentioned earlier, a combination of source code rewriting, out of language isolation, and special purpose libraries have a better track record than AST filtering for general purpose programming languages.

# doodad-js Admin (a month ago)

“Blacklisting or whitelisting, that’s an open discussion”: It really isn't.

So for you, blacklisting or whitelisting is not opened to a discussion?

No it isn't. As I mentioned earlier, a combination of source code rewriting, out of language isolation, and special purpose libraries have a better track record than AST filtering for general purpose programming languages.

So, you don’t want JS code interpretation inside “user reports formulas”, “template engines”, “compiler tools”, ...?

Claude

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Friday, June 22, 2018 5:06 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

On Fri, Jun 22, 2018, 4:56 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

Thanks,

If you blacklist.

Blacklisting or whitelisting, that’s an open discussion.

It really isn't.

Yet you're providing a library that does just that

Because that’s a “user land” library and currently the only way is with “AST filtering”, apart from compiling a complete runtime, with Emscripten or else.

No it isn't. As I mentioned earlier, a combination of source code rewriting, out of language isolation, and special purpose libraries have a better track record than AST filtering for general purpose programming languages.

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com


This email has been checked for viruses by AVG. www.avg.com

# Mike Samuel (a month ago)

On Fri, Jun 22, 2018, 5:30 PM doodad-js Admin <doodadjs at gmail.com> wrote:

“Blacklisting or whitelisting, that’s an open discussion”: It really isn't.

So for you, blacklisting or whitelisting is not opened to a discussion?

No. Case based reasoning doesn't work when the partition of cases can't be enumerated so if we want confidence in our tools we ought prefer whitelisting.

*No it isn't. As I mentioned earlier, a combination of source code

rewriting, out of language isolation, and special purpose libraries have a better track record than AST filtering for general purpose programming languages.*

So, you don’t want JS code interpretation inside “user reports formulas”, “template engines”, “compiler tools”, ...?

This is silly. I can want these without wanting them built using substandard tools.

# doodad-js Admin (a month ago)

This is silly. I can want these without wanting them built using substandard tools.

That’s the point why I bring it to ES. Nothing on the “user land” can provide something reliable, apart a complete JS runtime library compiled to “WASM” or “asm.js”. And... that’s silly.

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Friday, June 22, 2018 6:04 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

On Fri, Jun 22, 2018, 5:30 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

“Blacklisting or whitelisting, that’s an open discussion”: It really isn't.

So for you, blacklisting or whitelisting is not opened to a discussion?

No.

Case based reasoning doesn't work when the partition of cases can't be enumerated so if we want confidence in our tools we ought prefer whitelisting.


This email has been checked for viruses by AVG. www.avg.com

# Mike Samuel (a month ago)

On Fri, Jun 22, 2018, 6:51 PM doodad-js Admin <doodadjs at gmail.com> wrote:

This is silly. I can want these without wanting them built using substandard tools.

That’s the point why I bring it to ES. Nothing on the “user land” can provide something reliable, apart a complete JS runtime library compiled to “WASM” or “asm.js”. And... that’s silly.

For the last time, why do you believe opcode filtering can?

# doodad-js Admin (a month ago)

For the last time, why do you believe opcode filtering can?

Because, at my knowledge, AST filtering is more subject to break than “opcode” filtering. If that’s not the case, please help me to provide a better “safeEval” by reporting issues of my library directly to me. But I doubt it can be full proof without runtime’s help.

Claude

From: Mike Samuel <mikesamuel at gmail.com>

Sent: Friday, June 22, 2018 6:53 PM To: doodad-js Admin <doodadjs at gmail.com>

Cc: Isiah Meadows <isiahmeadows at gmail.com>; es-discuss <es-discuss at mozilla.org>

Subject: Re: FW: Proposal: safeEval

On Fri, Jun 22, 2018, 6:51 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

This is silly. I can want these without wanting them built using substandard tools.

That’s the point why I bring it to ES. Nothing on the “user land” can provide something reliable, apart a complete JS runtime library compiled to “WASM” or “asm.js”. And... that’s silly.

For the last time, why do you believe opcode filtering can?


This email has been checked for viruses by AVG. www.avg.com

# doodad-js Admin (18 days ago)

But I doubt it can be full proof without runtime’s help.

I found a way: AST filtering with rewriting. So that “obj[key]” will get rewritten to “safeEval.get(obj, key)”. That is now part of my TODO list for “@doodad-js/safeeval”. For the moment, I block the dynamic property accessor operator (“obj[key]”), and the rewriting must be manual.

Claude

From: doodad-js Admin <doodadjs at gmail.com>

Sent: Friday, June 22, 2018 7:29 PM To: mikesamuel at gmail.com Cc: 'Isiah Meadows' <isiahmeadows at gmail.com>; 'es-discuss' <es-discuss at mozilla.org>

Subject: RE: FW: Proposal: safeEval

For the last time, why do you believe opcode filtering can?

Because, at my knowledge, AST filtering is more subject to break than “opcode” filtering. If that’s not the case, please help me to provide a better “safeEval” by reporting issues of my library directly to me. But I doubt it can be full proof without runtime’s help.

Claude

From: Mike Samuel <mikesamuel at gmail.com <mailto:mikesamuel at gmail.com> >

Sent: Friday, June 22, 2018 6:53 PM To: doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> >

Cc: Isiah Meadows <isiahmeadows at gmail.com <mailto:isiahmeadows at gmail.com> >; es-discuss <es-discuss at mozilla.org <mailto:es-discuss at mozilla.org> >

Subject: Re: FW: Proposal: safeEval

On Fri, Jun 22, 2018, 6:51 PM doodad-js Admin <doodadjs at gmail.com <mailto:doodadjs at gmail.com> > wrote:

This is silly. I can want these without wanting them built using substandard tools.

That’s the point why I bring it to ES. Nothing on the “user land” can provide something reliable, apart a complete JS runtime library compiled to “WASM” or “asm.js”. And... that’s silly.

For the last time, why do you believe opcode filtering can?

www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

Virus-free. www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient, www.avg.com


This email has been checked for viruses by AVG. www.avg.com