Few more results: the answer for the flowing question: How much time spend spidermonkey to compile js files at the sturtup of firefox

[riadh chtara]

Hi guys
I tried to calculate how much time spend spidermonkey to compile js files at the sturtup of firefox:
So I changed jsapi.api and get the follwing results
Changes made to jsapi (you can see here the hole new version https://docs.google.com/folder/d/0B9VtjieUwlMDNzJVM3c1UkM0RVE/edit)
Then I recompiled firefox and relaunched it,than I got these results
I calculate three things:
Time needed for only compiling:
JSScript *
JS::Compile(JSContext *cx, HandleObject obj, CompileOptions options,
const jschar *chars, size_t length)

Time needed for comping and encoding:
JSScript *
JS::Compile(JSContext *cx, HandleObject obj, CompileOptions options,
const char *bytes, size_t length)


Time needed for runnig the script:
JS_NEVER_INLINE JS_PUBLIC_API(JSBool)
JS_ExecuteScript(JSContext *cx, JSObject *objArg, JSScript *scriptArg, jsval *rval)

The changes I made:

JSScript *
JS::Compile(JSContext *cx, HandleObject obj, CompileOptions options,
const char *bytes, size_t length)
{
FILE * pFile;
int64_t time = PRMJ_Now();
pFile = fopen ("/home/riadh/jsdata.txt","a");


jschar *chars;
if (options.utf8)
chars = InflateString(cx, bytes, &length, CESU8Encoding);
else
chars = InflateString(cx, bytes, &length);
if (!chars)
return NULL;

JSScript *script = Compile(cx, obj, options, chars, length);
js_free(chars);JS_NEVER_INLINE JS_PUBLIC_API(JSBool)
JS_ExecuteScript(JSContext *cx, JSObject *objArg, JSScript *scriptArg, jsval *rval)
if (pFile!=NULL) {
fprintf(pFile, "compiling and encoding time %s : %.3f ms \n",options.filename, double(PRMJ_Now() - time) / PRMJ_USEC_PER_MSEC);

fclose (pFile);
}
return script;
}

JSScript *
JS::Compile(JSContext *cx, HandleObject obj, CompileOptions options,
const jschar *chars, size_t length)
{
FILE * pFile;
int64_t time = PRMJ_Now();
pFile = fopen ("/home/riadh/jsdata.txt","a");
JSScript * script;


Maybe<AutoVersionAPI> mava;
if (options.versionSet) {
mava.construct(cx, options.version);
// AutoVersionAPI propagates some compilation flags through.
options.version = mava.ref().version();
}

JS_THREADSAFE_ASSERT(cx->compartment != cx->runtime->atomsCompartment);
AssertHeapIsIdle(cx);
CHECK_REQUEST(cx);
assertSameCompartment(cx, obj);
JS_ASSERT_IF(options.principals, cx->compartment->principals == options.principals);
AutoLastFrameCheck lfc(cx);
script = frontend::CompileScript(cx, obj, NULL, options, chars, length);
if (pFile!=NULL) {
fprintf(pFile, "compiling time %s : %.3f ms \n",options.filename, double(PRMJ_Now() - time) / PRMJ_USEC_PER_MSEC);

fclose (pFile);
}
return script;
}

JS_NEVER_INLINE JS_PUBLIC_API(JSBool)
JS_ExecuteScript(JSContext *cx, JSObject *objArg, JSScript *scriptArg, jsval *rval)
{
JS_PUBLIC_API(JSBool) js;
FILE * pFile;
int64_t time = PRMJ_Now();
pFile = fopen ("/home/riadh/jsdata.txt","a");
RootedObject obj(cx, objArg);
RootedScript script(cx, scriptArg);

JS_THREADSAFE_ASSERT(cx->compartment != cx->runtime->atomsCompartment);
AssertHeapIsIdle(cx);
CHECK_REQUEST(cx);
assertSameCompartment(cx, obj);
if (cx->compartment != obj->compartment())
*(volatile int *) 0 = 0xf0;
AutoLastFrameCheck lfc(cx);

/*
* Mozilla caches pre-compiled scripts (e.g., in the XUL prototype cache)
* and runs them against multiple globals. With a compartment per global,
* this requires cloning the pre-compiled script into each new global.
* Since each script gets run once, there is no point in trying to cache
* this clone. Ideally, this would be handled at some pinch point in
* mozilla, but there doesn't seem to be one, so we handle it here.
*/
if (script->compartment() != obj->compartment()) {
script = CloneScript(cx, NullPtr(), NullPtr(), script);
if (!script.get())
return false;
} else {
script = scriptArg;
}

js = Execute(cx, script, *obj, rval);
if (pFile!=NULL) {
fprintf(pFile, "execution time : %.3f ms \n", double(PRMJ_Now() - time) / PRMJ_USEC_PER_MSEC);

fclose (pFile);
}
return js;
}









The result are also here https://docs.google.com/folder/d/0B9VtjieUwlMDNzJVM3c1UkM0RVE/edit.
There is a text file (raw result) and an excel file
I calculate the sums:
The time needed for compiling and encoding is 1550ms

The time needed for running execution is 1245ms

What do you think about that?
If some of you has time, it would be greate if he replace his jsapi.cpp with my jsapi.cpp, (not forget to replace in my jsapi,all the "/home/riadh/jsdata.txt" by the address of the file you would like to save results in it),recompile firefox, open it, then send me your file.
But before that, could you check please if I did a mistake when making this test(I'm newbie to the code, so I can make stupid mistake)
I will use this test to get more data about time spend by spidermonkey (in real website like facebook, google, and websites with jquery)

Cheers
Riadh

[riadh chtara]

I rerun the test a second time.
I get
execution time 229,442
compiling time 177,229
encoding & compile was < compiling time(same files are compiled without being encoded)
I will run other test: but if I get the same result:I can confirm that saving compiled js files will not reduce that much the launch time of firefox (before that I will need to get the exact launch time of firefox)

[riadh chtara]

Hi guys
I get also new data about compiling time and runnig time of javascript in firefox.
I hack firefox (jsapi.cpp)
I changed evaluate method to:

extern JS_PUBLIC_API(bool)
JS::Evaluate(JSContext *cx, HandleObject obj, CompileOptions options,
const jschar *chars, size_t length, jsval *rval)
{
FILE * pFile, *pFilew;
int64_t compile, run;
pFile = fopen ("/home/riadh/evaldata.log","a");
pFilew = fopen ("/home/riadh/evaldataw.log","a");

Maybe<AutoVersionAPI> mava;
if (options.versionSet) {
mava.construct(cx, options.version);
// AutoVersionAPI propagates some compilation flags through.
options.version = mava.ref().version();
}

JS_THREADSAFE_ASSERT(cx->compartment != cx->runtime->atomsCompartment);
AssertHeapIsIdle(cx);
CHECK_REQUEST(cx);
assertSameCompartment(cx, obj);
JS_ASSERT_IF(options.principals, cx->compartment->principals == options.principals);

AutoLastFrameCheck lfc(cx);

options.setCompileAndGo(true);
options.setNoScriptRval(!rval);
compile = PRMJ_Now();
RootedScript script(cx, frontend::CompileScript(cx, obj, NULL, options, chars, length));
if (!script)
return false;
compile = PRMJ_Now() - compile;
run = PRMJ_Now();
JS_PUBLIC_API(bool) tmp = Execute(cx, script, *obj, rval);
run = PRMJ_Now() - run;
JS_ASSERT(script->getVersion() == options.version);
if ((pFile!=NULL)&&(pFilew!=NULL)) {
fprintf(pFile, "\n\nevalution time: compile: %.3f ms run : %.3f ms of \n\n %s \n\n",
double(compile) / PRMJ_USEC_PER_MSEC,
double(run) / PRMJ_USEC_PER_MSEC,
DeflateString(cx, chars, length));
fprintf(pFilew, "compile: %.3f ms run : %.3f ms of \n",
double(compile) / PRMJ_USEC_PER_MSEC,
double(run) / PRMJ_USEC_PER_MSEC);

fclose (pFile);
fclose (pFilew);
}


return tmp;
}




Could you check if I made any stupid mistake in time measurement.

I recompile and run firefox.
I went to these websites
gmail(inbox page),google,google drive (files view page),facebook (wall page),
and jquery.
Each time I open firefox, delete log files, then open a website, I save the new logs file.
I did that three time for each website
I got the following result:in average more than 40% of time is spent in compiling
More details about the results I got
https://docs.google.com/open?id=0B9VtjieUwlMDSUkydko2S3gtWTQ
Logs file I got
https://docs.google.com/open?id=0B9VtjieUwlMDWVpmYi1SbGxKUTg

As I said early, I'm newbie to the code.
But, I think that the time running time I measure doesn't involve running time of a JavaScript function attached to a event like mouse click because I didn't do that.
If I'm not wrong the time I get is only start up time for a webpage
So maybe saving compiled file will increase the speed of the start up of page
What do you think?
I'm waiting for your answer
Chaeers
Riadh

[Nicolas B. Pierron]

Hi Riadh,

On 10/17/2012 01:16 AM, riadh chtara wrote:
> JS_NEVER_INLINE JS_PUBLIC_API(JSBool)
> JS_ExecuteScript(JSContext *cx, JSObject *objArg, JSScript *scriptArg, jsval *rval)

> {
> FILE * pFile;
> int64_t time = PRMJ_Now();
> pFile = fopen ("/home/riadh/jsdata.txt","a");
>

> …

>
> if (pFile!=NULL) {
> fprintf(pFile, "execution time : %.3f ms \n", double(PRMJ_Now() - time) / PRMJ_USEC_PER_MSEC);
>
> fclose (pFile);
> }
> return js;
> }
>
>
> The result are also here https://docs.google.com/folder/d/0B9VtjieUwlMDNzJVM3c1UkM0RVE/edit.
> There is a text file (raw result) and an excel file
> I calculate the sums:
> The time needed for compiling and encoding is 1550ms
>
> The time needed for running execution is 1245ms
>
> What do you think about that?

Can you try to avoid manipulating a file, you should better allocate
some-kind of static queue containing time-stamps of events, such as
enterExecute, enterCompile, leaveCompile and leaveExecute with the stamp of
each of them. Then you can add a function to the jsapi.h and map it in JS
such as you can call it from the web console to dump the queue of events to
a file. You might even set another function to reset the queue for testing
websites.

Take care, I don't know if you noticed it, but compartment might be executed
in parallel, which means that registering a time-stamp / writing to a file
might cause some race condition issues, especially when opening a file
twice, which might happens if a Compile happens under an Execute.

--
Nicolas B. Pierron

[riadh chtara]

Hi Nicalas,

> Can you try to avoid manipulating a file, you should better allocate
>
> some-kind of static queue containing time-stamps of events, such as
>
> enterExecute, enterCompile, leaveCompile and leaveExecute with the stamp of
>
> each of them. Then you can add a function to the jsapi.h and map it in JS
>
> such as you can call it from the web console to dump the queue of events to
>
> a file. You might even set another function to reset the queue for testing
>
> websites.

Thank you for your suggestions
I think can do that. But before that, I'd like to know what is the size of the bytecode.
If the bytecode is very big,it's would be a waste of time to save to hard drive and opened from there.
I was doing test to estimate the size of bytecode for jquery(jquery-1.8.2.min.js).
It was very complicated to do tha

Cheers
Riadh

[riadh chtara]

Hi Nicalas,
> Can you try to avoid manipulating a file, you should better allocate
>
> some-kind of static queue containing time-stamps of events, such as
>
> enterExecute, enterCompile, leaveCompile and leaveExecute with the stamp of
>
> each of them. Then you can add a function to the jsapi.h and map it in JS
>
> such as you can call it from the web console to dump the queue of events to
>
> a file. You might even set another function to reset the queue for testing
>

> websitesThank you for your suggestions

I think can do that. But before that, I'd like to know what is the size of the bytecode.
If the bytecode is very big,it's would be a waste of time to save to hard drive and opened from there.
I was doing test to estimate the size of bytecode for jquery(jquery-1.8.2.min.js).

It was very complicated to get the exact size of generated bytecode.
So I did the following thing:
I made a program that run a script.This program stops just before script is compiled and also just after that (the program waits for an keyword input to continue)
waitingBefore();
script = Compile(cx, *obj, *options, buffer->begin(),buffer->length());
waitingAfter();
I used linux command pmap to get the memory usage by the process before and after compiling the script.
The bytecode size for me is the difference between these value
I did the same thing with a profiler (valgrind)
For jquery-1.8.2.min.js, I get 3 MB of augmentation in memory usage. So jquery-1.8.2.min.js bytecode file will need 3 MB in hard drive to be stored.
I'm not very happy with that because it seems very big for me:
With 3 MB, reading byte code file from hard drive takes in average 215ms (even if we repeat it many time and we start using hard drive cache).
So in that case, saving bytecode to hard drive is useless.
So could you tell me please if there is another way to check the size of the bytecode (jsscript object)?
Or even if the size of the bytecode is 3 MB, could you tell since you have much experience with this code, if there are large parts of it that could be calculated very fast or doesn't need to be calculated(a very big vector of zeros)


Cheers
Riadh

[Nicolas B. Pierron]

Hi Riadh,

You are probably not only looking only at the bytecode but also at all the
analysis which happen with it, and the memory management that we have might
also allocate more space less frequently. If you only want the bytecode
size alone, you should look at all “script->length” produced, but be aware
that this does not include any of the analysis and that these will have to
be computed/restored as they are needed for running the bytecode.

--
Nicolas B. Pierron

[riadh chtara]

Hi guys,

> You are probably not only looking only at the bytecode but also at all the
>
> analysis which happen with it, and the memory management that we have might
>
> also allocate more space less frequently.

I hope I made a big mistake in estimating the bytecode size, because if my estimation is true, saving bytecode will be useless.

> If you only want the bytecode
>

> size alone, you should look at all �script->length� produced, but be aware

>
> that this does not include any of the analysis and that these will have to
>
> be computed/restored as they are needed for running the bytecode.

I tried using script->length (with jquery), I get 18. I don't know what that means.

When I was talking about saving the bytecode, I was thinking about saving the byte code and the results of analysis, this means saving jsscript object returned by compile function.
Besides, I was searching for the place in which compile function spend the most of its time. I found that it spend most of its time in (jquery-1.4.min.js):
*pn = parser.statement(); (55% of compiling and analyse time)
*EmitTree(cx, &bce, pn); (28% of compiling and analyse time
*parser.freeTree(pn);
*NameFunctions(cx, pn)
(fuctions used in frontend::CompileScript - bytecodecompiler.cpp)
Could you tell please for each of these 4 functions, what happens exactly? is is it compiling or analyzing? This is very important for me.
If is only compiling I think we will need to save only bytecode.
If they involve analyzing (especially parser.statement(), EmitTree(cx, &bce, pn)) we will need to save not only bytecode but also analysis results
I'm wondering if we could have a chat with skype for example: it would be much easier to talk this thing: I have also same questions and it would be easier for me get more information.
Cheers
Riadh

[Nicolas B. Pierron]

On 10/19/2012 01:06 PM, riadh chtara wrote:
> Hi guys,
>> You are probably not only looking only at the bytecode but also at all the
>>
>> analysis which happen with it, and the memory management that we have might
>>
>> also allocate more space less frequently.
> I hope I made a big mistake in estimating the bytecode size, because if my estimation is true, saving bytecode will be useless.
>
>> If you only want the bytecode
>>

>> size alone, you should look at all �script->length� produced, but be aware

>>
>> that this does not include any of the analysis and that these will have to
>>
>> be computed/restored as they are needed for running the bytecode.
> I tried using script->length (with jquery), I get 18. I don't know what that means.

JQuery is composed of multiple scripts, and length is the number of bytes
used by the bytecode of one script.

> When I was talking about saving the bytecode, I was thinking about saving the byte code and the results of analysis, this means saving jsscript object returned by compile function.
> Besides, I was searching for the place in which compile function spend the most of its time. I found that it spend most of its time in (jquery-1.4.min.js):
> *pn = parser.statement(); (55% of compiling and analyse time)

> *EmitTree(cx,&bce, pn); (28% of compiling and analyse time

> *parser.freeTree(pn);
> *NameFunctions(cx, pn)
> (fuctions used in frontend::CompileScript - bytecodecompiler.cpp)
> Could you tell please for each of these 4 functions, what happens exactly? is is it compiling or analyzing? This is very important for me.

I have no idea, I think we ran the analysis once we have finished producing
the bytecode or before the first time it is ran. I am not entirely sure. You
can look at all the analysis function of JSScript in jsscript.h.

> If is only compiling I think we will need to save only bytecode.

> If they involve analyzing (especially parser.statement(), EmitTree(cx,&bce, pn)) we will need to save not only bytecode but also analysis results

> I'm wondering if we could have a chat with skype for example: it would be much easier to talk this thing: I have also same questions and it would be easier for me get more information.

IRC (irc.mozilla.org #jsapi) sounds like a better solution to ask such
questions than skype because it is asynchronous, many people are following,
and the right person can answer each question in a less formal way.

--
Nicolas B. Pierron

[riadh chtara]

Hi guys
First of all, I think I have much much over estimate the size od jsscript object for jquery.
So I used today a function I created that us exactly the same way like in JSScript * js::CloneScript (defined in JSScript.cpp) to calculate JSScript size.
I got as a result for jquery 62 bytes which is unfortunately very small for me.
But, I feel quite happy today because I discover this CloneScript method. Because it does almost the same thing I want to do but save the result in memory instead hard drive.
So I will try to understand it and do just the same.
Best
Riadh

[Steve Fink]

Just be sure you're getting all of the scripts you care about. For
example, consider this example.js:

var x = 1;
function f() {
return x;
}
function g() {
return eval("x + 1");
}

That's three scripts -- f, g, and the global script -- and another one
at runtime every time g() is called (though it gets cached.)

I'd guess that jquery has hundreds of scripts. The size of the global
script is generally not that interesting, especially for a module where
almost everything is wrapped up in functions.

[riadh chtara]

Hi Steve,

Just be sure you're getting all of the scripts you care about. For example,
> consider this example.js:
>
> var x = 1;
> function f() {
> return x;
> }
> function g() {
> return eval("x + 1");
> }
> That's three scripts -- f, g, and the global script -- and another one at
> runtime every time g() is called (though it gets cached.)
>

Thanks a lot : I didn't know that.
If you have a js file with your following code in it

var x = 1;
function f() {
return x;
}
function g() {
return eval("x + 1");
}

adn C++ program with the following code,


JSScript *allscript;
allscript = Compile(cx,obj,options, "js script path")

Could you tell me please how to get the scripts list (the global script, f
and g)
cheers
Riadh

[Nicolas B. Pierron]

Hi Riadh,

On 10/21/2012 09:13 AM, riadh chtara wrote:

> I got as a result for jquery 62 bytes which is unfortunately very small for me.

if this is was 62 bytes, then this would be an extreme compression
algorithm. If you ever got a size below the size of the initial script, you
probably made a mistake at measuring the size, especially if you are using a
minimized version.

As Steve mentioned, JQuery is composed of hundreds of functions. Each
function has its own script. So you might want to look at this collection of
function and not only one.

--
Nicolas B. Pierron