info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Re: Web Compat Service & Static Analysis
19 views
Skip to first unread message
Luke Crouch
Nov 7, 2014, 2:56:30 PM11/7/14
to David Bruant, Justin Crawford, dev-stati...@lists.mozilla.org
Thanks David, that's very helpful.
I'm adding dev-static-analysis to this to see if anyone on that list
(Erik maybe?) has any ideas on this.
In short, we're learning that JS will certainly be the most challenging
language for which to create a compatibility linter/scanner. ;)
-L
On 11/7/14 10:18 AM, David Bruant wrote:
> Le 07/11/2014 16:42, Luke Crouch a écrit :
>> +Justin
>>
>> Thanks for the info David, sorry you coudn't make it.
>>
>> In addition to the updated notes in the etherpad [1], Justin and I met
>> [2] with Thijs Busser who created http://jscc.info/ that mashes up data
>> from Kangax & Can I Use with a bunch of regexs.
>>
>> He mentioned wanting to use esprima.org? Do you like that approach too?
> Totally. Anything that relies on regexps to do anything with code is
> doomed to fail at some point. AST-based analysis are the way to go
> (fundamentally, code as a data structure is a trees and regexps cannot
> work on trees).
> And esprima is the state-of-the-art JS parser that lots of static
> analysis tools are based on (escodegen, eslint, etc. The list of
> dependent tools on npm is saying https://www.npmjs.org/package/esprima
> ). Esprima is fully ES5-compatible and they're actively working on an
> ES6 branch (harmony), so that would be the parser I'd bet on.
>
> But in my opinion, it still won't be enough. A naive tool based on
> esprima won't be able to understand that the following is not
> browser-compatible (yet, because Object.assign is introduced in ES6):
> ----
> var O = Object;
> var assign = O.assign;
> assign({}, {});
> ----
> (and some forms of this will occur in real-life code, because on the
> web, everything happens)
> To understand that this code is not ES5 compatible, one has to track
> that the O variable has such type (and such static functions attached to
> it), etc. and that's tedious work. The good news is that it's exactly
> the sort of work the TypeScript compiler was designed to do.
> Other compiler like the Closure Compiler do this kind of work too, but
> they force you into writing your JavaScript in a particular style (in
> order to make the code more easliy analysable). The reason I'm so fan of
> TypeScript is that they've found a great balance of analysing types in
> JS while letting authors write their code in whatever style fits them.
>
> Hope that helps,
>
> David
>
>>
>> Thanks,
>> -L
>>
>>
>> [1] https://etherpad.mozilla.org/compat-data-for-static-analysis
>> [2] https://devengage.etherpad.mozilla.org/jscc-mdn
>>
>>
>> On 11/7/14 8:58 AM, David Bruant wrote:
>>> Hi Luke,
>>>
>>> I'm very sorry, I couldn't attend the meeting. I was catching up after
>>> MozFest.
>>>
>>> As far as JavaScript static analysis is concerned, I'm sure there is a
>>> "quick" win to be won hacking the TypeScript compiler (especially now
>>> that they've re-written it entirely and that it's apparently much
>>> cleaner).
>>>
>>> Tools like JSLint/JSHint/ESLint (3rd is my favorite because of its very
>>> modular rule architecture) can verify some aspects of a program, but are
>>> really syntax-based. For instance, these tools cannot tell whether
>>> you're using setTimeout with a string or a function as first argument.
>>> On the other hand, TypeScript can and does and you can be sure it does,
>>> because that's what it's been written for (track types across variables
>>> and function arguments).
>>> TypeScript is initialized with a declaration file so that the compiler
>>> knows about the "standard library" (functions exposed per ECMA and W3C
>>> specs). I'm sure it would be easy to change these declaration files to
>>> see if some code is using non-standard and/or edgy (or prefixed)
>>> functions in their code.
>>>
>>> rhaa... I'm babbling and it's complicated to explain in an email.
>>>
>>> If you're interested, maybe we can call some time next week so I explain
>>> all that (unless it's too early-stage discussion in which case, we can
>>> postpone to a more appropriate time).
>>>
>>> David
>>>
>>>
>>> Le 27/10/2014 18:32, publicc...@mozilla.com a écrit :
>>>> The following is a new meeting request:
>>>>
>>>> Subject: Web Compat Service & Static Analysis
>>>> Organizer: publicc...@mozilla.com
>>>> Sent By: "Luke Crouch" <lcr...@mozilla.com>
>>>>
>>>> Location: VidyoRoom - Luke Crouch
>>>> Time: Wednesday, October 29, 2014, 11:00:00 AM - 12:00:00 PM GMT
>>>> -06:00 US/Canada Central
>>>> Invitees: jo...@factorialfive.com; hoos...@mozilla.com;
>>>> er...@mozilla.com; jpato...@mozilla.com; brua...@gmail.com;
>>>> lcr...@mozilla.com
>>>>
>>>>
>>>> *~*~*~*~*~*~*~*~*~*
>>>>
>>>> Vidyo: Luke Crouch
>>>> Room Link:
>>>> https://v.mozilla.com/flex.html?roomdirect.html&key=k3IKWMEdaAsz
>>>> Webcast Link:
>>>> https://vreplay.mozilla.com/replay/webcastShow.html?key=SbB5sPpfCaIQ8b8
>>>> Dial-In: 1-650-903-0800, x92; Conference #: 99163
>>>> +1 800 707 2533, pin 369; Conference #: 99163
>>>> IRC: #mdndev
>>>>
>>>> Agenda: https://etherpad.mozilla.org/compat-data-for-static-analysis
>
--
Q: Why is this email five sentences or less?
A: http://five.sentenc.es
I'm adding dev-static-analysis to this to see if anyone on that list
(Erik maybe?) has any ideas on this.
In short, we're learning that JS will certainly be the most challenging
language for which to create a compatibility linter/scanner. ;)
-L
On 11/7/14 10:18 AM, David Bruant wrote:
> Le 07/11/2014 16:42, Luke Crouch a écrit :
>> +Justin
>>
>> Thanks for the info David, sorry you coudn't make it.
>>
>> In addition to the updated notes in the etherpad [1], Justin and I met
>> [2] with Thijs Busser who created http://jscc.info/ that mashes up data
>> from Kangax & Can I Use with a bunch of regexs.
>>
>> He mentioned wanting to use esprima.org? Do you like that approach too?
> Totally. Anything that relies on regexps to do anything with code is
> doomed to fail at some point. AST-based analysis are the way to go
> (fundamentally, code as a data structure is a trees and regexps cannot
> work on trees).
> And esprima is the state-of-the-art JS parser that lots of static
> analysis tools are based on (escodegen, eslint, etc. The list of
> dependent tools on npm is saying https://www.npmjs.org/package/esprima
> ). Esprima is fully ES5-compatible and they're actively working on an
> ES6 branch (harmony), so that would be the parser I'd bet on.
>
> But in my opinion, it still won't be enough. A naive tool based on
> esprima won't be able to understand that the following is not
> browser-compatible (yet, because Object.assign is introduced in ES6):
> ----
> var O = Object;
> var assign = O.assign;
> assign({}, {});
> ----
> (and some forms of this will occur in real-life code, because on the
> web, everything happens)
> To understand that this code is not ES5 compatible, one has to track
> that the O variable has such type (and such static functions attached to
> it), etc. and that's tedious work. The good news is that it's exactly
> the sort of work the TypeScript compiler was designed to do.
> Other compiler like the Closure Compiler do this kind of work too, but
> they force you into writing your JavaScript in a particular style (in
> order to make the code more easliy analysable). The reason I'm so fan of
> TypeScript is that they've found a great balance of analysing types in
> JS while letting authors write their code in whatever style fits them.
>
> Hope that helps,
>
> David
>
>>
>> Thanks,
>> -L
>>
>>
>> [1] https://etherpad.mozilla.org/compat-data-for-static-analysis
>> [2] https://devengage.etherpad.mozilla.org/jscc-mdn
>>
>>
>> On 11/7/14 8:58 AM, David Bruant wrote:
>>> Hi Luke,
>>>
>>> I'm very sorry, I couldn't attend the meeting. I was catching up after
>>> MozFest.
>>>
>>> As far as JavaScript static analysis is concerned, I'm sure there is a
>>> "quick" win to be won hacking the TypeScript compiler (especially now
>>> that they've re-written it entirely and that it's apparently much
>>> cleaner).
>>>
>>> Tools like JSLint/JSHint/ESLint (3rd is my favorite because of its very
>>> modular rule architecture) can verify some aspects of a program, but are
>>> really syntax-based. For instance, these tools cannot tell whether
>>> you're using setTimeout with a string or a function as first argument.
>>> On the other hand, TypeScript can and does and you can be sure it does,
>>> because that's what it's been written for (track types across variables
>>> and function arguments).
>>> TypeScript is initialized with a declaration file so that the compiler
>>> knows about the "standard library" (functions exposed per ECMA and W3C
>>> specs). I'm sure it would be easy to change these declaration files to
>>> see if some code is using non-standard and/or edgy (or prefixed)
>>> functions in their code.
>>>
>>> rhaa... I'm babbling and it's complicated to explain in an email.
>>>
>>> If you're interested, maybe we can call some time next week so I explain
>>> all that (unless it's too early-stage discussion in which case, we can
>>> postpone to a more appropriate time).
>>>
>>> David
>>>
>>>
>>> Le 27/10/2014 18:32, publicc...@mozilla.com a écrit :
>>>> The following is a new meeting request:
>>>>
>>>> Subject: Web Compat Service & Static Analysis
>>>> Organizer: publicc...@mozilla.com
>>>> Sent By: "Luke Crouch" <lcr...@mozilla.com>
>>>>
>>>> Location: VidyoRoom - Luke Crouch
>>>> Time: Wednesday, October 29, 2014, 11:00:00 AM - 12:00:00 PM GMT
>>>> -06:00 US/Canada Central
>>>> Invitees: jo...@factorialfive.com; hoos...@mozilla.com;
>>>> er...@mozilla.com; jpato...@mozilla.com; brua...@gmail.com;
>>>> lcr...@mozilla.com
>>>>
>>>>
>>>> *~*~*~*~*~*~*~*~*~*
>>>>
>>>> Vidyo: Luke Crouch
>>>> Room Link:
>>>> https://v.mozilla.com/flex.html?roomdirect.html&key=k3IKWMEdaAsz
>>>> Webcast Link:
>>>> https://vreplay.mozilla.com/replay/webcastShow.html?key=SbB5sPpfCaIQ8b8
>>>> Dial-In: 1-650-903-0800, x92; Conference #: 99163
>>>> +1 800 707 2533, pin 369; Conference #: 99163
>>>> IRC: #mdndev
>>>>
>>>> Agenda: https://etherpad.mozilla.org/compat-data-for-static-analysis
>
--
Q: Why is this email five sentences or less?
A: http://five.sentenc.es
Erik Rose
Nov 7, 2014, 3:33:21 PM11/7/14
to Luke Crouch, Justin Crawford, Marcell Vazquez-Chanlatte, David Bruant, dev-stati...@lists.mozilla.org
Everything David says is on the level. For analyzing code out in the wild, esprima might be a good candidate to build on. For analyzing Firefox's internal JS (which is what we're trying to do in DXR), it's not suitable, since Spidermonkey went off in all sorts of non-standardized directions, and no off-the-shelf tool supports ES6 yet, let alone our pre-standardization dalliances.
Marcell, who interned on DXR this summer, might have something to add here, as he did a great deal of shopping for JS analysis tools before coming up empty. He even had a plugin running for awhile based on ternjs, which does some neat object-shape-tracing tricks. If we're thinking about combining our efforts, I see two possibilities: go with something like esprima and give up ES6 and Spidermonkey compatibility for the near future, or add analysis capabilities to Spiderflunky (https://github.com/erikrose/spiderflunky), which is based on Spidermonkey's own JS parser (with the advantage that we don't have to write it in JS).
Can anyone who works on Firefox say whether we're headed toward ES6 conformance internally and, if so, what the timeline might be?
Cheers,
Erik
> _______________________________________________
> dev-static-analysis mailing list
> dev-stati...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-static-analysis
Marcell, who interned on DXR this summer, might have something to add here, as he did a great deal of shopping for JS analysis tools before coming up empty. He even had a plugin running for awhile based on ternjs, which does some neat object-shape-tracing tricks. If we're thinking about combining our efforts, I see two possibilities: go with something like esprima and give up ES6 and Spidermonkey compatibility for the near future, or add analysis capabilities to Spiderflunky (https://github.com/erikrose/spiderflunky), which is based on Spidermonkey's own JS parser (with the advantage that we don't have to write it in JS).
Can anyone who works on Firefox say whether we're headed toward ES6 conformance internally and, if so, what the timeline might be?
Cheers,
Erik
> dev-static-analysis mailing list
> dev-stati...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-static-analysis
David Bruant
Nov 8, 2014, 4:54:10 PM11/8/14
to Erik Rose, Luke Crouch, Justin Crawford, Marcell Vazquez-Chanlatte, dev-stati...@lists.mozilla.org
Le 07/11/2014 21:32, Erik Rose a écrit :
> For analyzing Firefox's internal JS (which is what we're trying to do
> in DXR), it's not suitable, since Spidermonkey went off in all sorts
> of non-standardized directions, and no off-the-shelf tool supports ES6
> yet, let alone our pre-standardization dalliances.
But for a service checking browser compatibility for web content, that
> For analyzing Firefox's internal JS (which is what we're trying to do
> in DXR), it's not suitable, since Spidermonkey went off in all sorts
> of non-standardized directions, and no off-the-shelf tool supports ES6
> yet, let alone our pre-standardization dalliances.
shouldn't be a concern.
> Can anyone who works on Firefox say whether we're headed toward ES6
> conformance internally and, if so, what the timeline might be?
https://bugzilla.mozilla.org/show_bug.cgi?id=867609
https://bugzilla.mozilla.org/show_bug.cgi?id=867617
I think Firefox aims at removing all deviations from ES6.
https://bugzilla.mozilla.org/show_bug.cgi?id=611388
https://bugzilla.mozilla.org/show_bug.cgi?id=1001090
were important steps in that direction, for instance.
I haven't heard of a timeline. Among other reasons, changing the
semantics of chrome-level JS bears a compatibility risk for addons, so
that'll all take time and it'd be hard for anyone to tell how long.
David
> <mailto:lcr...@mozilla.com>> wrote:
>
>> Thanks David, that's very helpful.
>>
>> I'm adding dev-static-analysis to this to see if anyone on that list
>> (Erik maybe?) has any ideas on this.
>>
>> In short, we're learning that JS will certainly be the most challenging
>> language for which to create a compatibility linter/scanner. ;)
>>
>> -L
>>
>> On 11/7/14 10:18 AM, David Bruant wrote:
>>> Le 07/11/2014 16:42, Luke Crouch a écrit :
>>>> +Justin
>>>>
>>>> Thanks for the info David, sorry you coudn't make it.
>>>>
>>>> In addition to the updated notes in the etherpad [1], Justin and I met
>>>> [2] with Thijs Busser who createdhttp://jscc.info/that mashes up data
>
>> Thanks David, that's very helpful.
>>
>> I'm adding dev-static-analysis to this to see if anyone on that list
>> (Erik maybe?) has any ideas on this.
>>
>> In short, we're learning that JS will certainly be the most challenging
>> language for which to create a compatibility linter/scanner. ;)
>>
>> -L
>>
>> On 11/7/14 10:18 AM, David Bruant wrote:
>>> Le 07/11/2014 16:42, Luke Crouch a écrit :
>>>> +Justin
>>>>
>>>> Thanks for the info David, sorry you coudn't make it.
>>>>
>>>> In addition to the updated notes in the etherpad [1], Justin and I met
>>>> from Kangax & Can I Use with a bunch of regexs.
>>>>
>>>> He mentioned wanting to useesprima.org <http://esprima.org/>? Do
>>>>
>>>> you like that approach too?
>>> Totally. Anything that relies on regexps to do anything with code is
>>> doomed to fail at some point. AST-based analysis are the way to go
>>> (fundamentally, code as a data structure is a trees and regexps cannot
>>> work on trees).
>>> And esprima is the state-of-the-art JS parser that lots of static
>>> analysis tools are based on (escodegen, eslint, etc. The list of
>>> dependent tools on npm is sayinghttps://www.npmjs.org/package/esprima
>>> Totally. Anything that relies on regexps to do anything with code is
>>> doomed to fail at some point. AST-based analysis are the way to go
>>> (fundamentally, code as a data structure is a trees and regexps cannot
>>> work on trees).
>>> And esprima is the state-of-the-art JS parser that lots of static
>>> analysis tools are based on (escodegen, eslint, etc. The list of
>>>>>> The following is a new meeting request:
>>>>>>
>>>>>> Subject: Web Compat Service & Static Analysis
>>>>>> Organizer: publicc...@mozilla.com
>>>>>> <mailto:publicc...@mozilla.com>
>>>>>>
>>>>>> Subject: Web Compat Service & Static Analysis
>>>>>> Organizer: publicc...@mozilla.com
>>>>>> Sent By: "Luke Crouch" <lcr...@mozilla.com
>>>>>> <mailto:lcr...@mozilla.com>>
>>>>>>
>>>>>> Location: VidyoRoom - Luke Crouch
>>>>>> Time: Wednesday, October 29, 2014, 11:00:00 AM - 12:00:00 PM GMT
>>>>>> -06:00 US/Canada Central
>>>>>> Invitees: jo...@factorialfive.com
>>>>>> <mailto:jo...@factorialfive.com>; hoos...@mozilla.com
>>>>>> Location: VidyoRoom - Luke Crouch
>>>>>> Time: Wednesday, October 29, 2014, 11:00:00 AM - 12:00:00 PM GMT
>>>>>> -06:00 US/Canada Central
>>>>>> Invitees: jo...@factorialfive.com
>>>>>> <mailto:hoos...@mozilla.com>;
>>>>>> er...@mozilla.com <mailto:er...@mozilla.com>;
>>>>>> jpato...@mozilla.com <mailto:jpato...@mozilla.com>;
>>>>>> brua...@gmail.com <mailto:brua...@gmail.com>;
>>>>>> lcr...@mozilla.com <mailto:lcr...@mozilla.com>
>>>>>>
>>>>>>
>>>>>> *~*~*~*~*~*~*~*~*~*
>>>>>>
>>>>>> Vidyo: Luke Crouch
>>>>>> Room Link:
>>>>>> https://v.mozilla.com/flex.html?roomdirect.html&key=k3IKWMEdaAsz
>>>>>> Webcast Link:
>>>>>> https://vreplay.mozilla.com/replay/webcastShow.html?key=SbB5sPpfCaIQ8b8
>>>>>> Dial-In: 1-650-903-0800, x92; Conference #: 99163
>>>>>> +1 800 707 2533, pin 369; Conference #: 99163
>>>>>> IRC: #mdndev
>>>>>>
>>>>>> Agenda: https://etherpad.mozilla.org/compat-data-for-static-analysis
>>>
>>
>> --
>> Q: Why is this email five sentences or less?
>> A:http://five.sentenc.es <http://five.sentenc.es/>
>>>>>>
>>>>>> *~*~*~*~*~*~*~*~*~*
>>>>>>
>>>>>> Vidyo: Luke Crouch
>>>>>> Room Link:
>>>>>> https://v.mozilla.com/flex.html?roomdirect.html&key=k3IKWMEdaAsz
>>>>>> Webcast Link:
>>>>>> https://vreplay.mozilla.com/replay/webcastShow.html?key=SbB5sPpfCaIQ8b8
>>>>>> Dial-In: 1-650-903-0800, x92; Conference #: 99163
>>>>>> +1 800 707 2533, pin 369; Conference #: 99163
>>>>>> IRC: #mdndev
>>>>>>
>>>>>> Agenda: https://etherpad.mozilla.org/compat-data-for-static-analysis
>>>
>>
>> --
>> Q: Why is this email five sentences or less?
>> _______________________________________________
>> dev-static-analysis mailing list
>> dev-stati...@lists.mozilla.org
>> <mailto:dev-stati...@lists.mozilla.org>
>> dev-static-analysis mailing list
>> dev-stati...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-static-analysis
>
Luke Crouch
Nov 19, 2014, 7:49:22 AM11/19/14
to David Bruant, Erik Rose, Justin Crawford, Marcell Vazquez-Chanlatte, dev-stati...@lists.mozilla.org
Kicking up this thread again because Facebook did this:
http://flowtype.org/
-L
>>>>> [2] with Thijs Busser who created http://jscc.info/ that mashes up data
http://flowtype.org/
-L
>>>>> from Kangax & Can I Use with a bunch of regexs.
>>>>>
>>>>> He mentioned wanting to use esprima.org <http://esprima.org/>? Do
>>>>>
>>>>> you like that approach too?
>>>> Totally. Anything that relies on regexps to do anything with code is
>>>> doomed to fail at some point. AST-based analysis are the way to go
>>>> (fundamentally, code as a data structure is a trees and regexps cannot
>>>> work on trees).
>>>> And esprima is the state-of-the-art JS parser that lots of static
>>>> analysis tools are based on (escodegen, eslint, etc. The list of
>>>> dependent tools on npm is saying https://www.npmjs.org/package/esprima
>>>> Totally. Anything that relies on regexps to do anything with code is
>>>> doomed to fail at some point. AST-based analysis are the way to go
>>>> (fundamentally, code as a data structure is a trees and regexps cannot
>>>> work on trees).
>>>> And esprima is the state-of-the-art JS parser that lots of static
>>>> analysis tools are based on (escodegen, eslint, etc. The list of
0 new messages