Profiling premake

[Mihai Sebea]

Hi 

I'm trying to profile a bit premake as it takes around 2.5 sec to genereate projects for a rather "simple" ish project .. probably a few hundred files and 6 configurations 

Btw I fixed the pepper fish profiler 

checkout https://github.com/premake/premake-core/pull/1164

Now ...it seems most of the time is spent 

-- Translate

if field and p.field.translates(field) then

value = p.field.translate(field, value)

end

 

--------------------- L:compile@src/base/configset.lua:212 ---------------------

Sample count:          425

Time spend total:       1.987s

Time spent in children: 1.064s

Time spent in self:     0.923s

Time spent per sample:  0.00468s/sample

Time spent in self per sample: 0.00217s/sample

Child L:compile@src/base/configset.lua:212      sampled    224 times. Took 1.05s

Child L:__index@src/base/context.lua:58         sampled      2 times. Took 0.01s

and the fields translate ... well I have no idea what it does ...

function field.translate(f, value)

local processor = field.accessor(f, "translate")

if processor then

return processor(f, value, nil)[1]

else

return value

end

end

function field.translates(f)

return (field.accessor(f, "translate") ~= nil)

end

So first question ... are there any other methods of profiling ?

and second ... what does this translate do ? and how can I improve this ?

Later edit 

It seems there are only 4 fields that implement the translate function 

directory 

file

keyed

list

and 1 sec is spent in child calls so i assume the rest is spent in just checking if the field has the translate function ?

Even later edit 

So i tried to add another check in the accessorForKind function 

if no processor is found i fill the cache with an empty string because well nil already means that we haven't searched for it ...

so i added an extra check if we already search this field type and the cache is "" then I return nil 

This should in theory avoid the cost of 

local thisKind = kind:match('(.-):') or kind

local nextKind = kind:sub(#thisKind + 2)

But in practice it does seem to make a difference ?

any ideas aside from moving the whole field class in c ?

or maybe that would be even slower because of the overhead of the c function call and mashaling params ?

 

Latest edit 

i also tried to call the accessor only once thinging that reducing a function call will be less expensive but since the the function is already cached it doesn't do much difference

local translator = field and p.field.accessor(field, "translate") or nil

  if  translator then

So finally i removed the translator call and the time reduced from 2000ms to 410ms

Now ... it seems the actual call to the translator is the issue so I tried to time that with os.clock and from about 3000 calls very few go above 0.001 most of them are 0 ...

 

Not sure what to do next ... any tips ?

  

[starkos]

field.translate() handles translating the path separators contained within the value of a field, swapping out the standard "/" for "\\" as needed. The `processor()` stuff abstracts away the underlying type of field. i.e. is it just a string? An array of strings? A table of arrays of strings? The processor function is responsible for walking the data in a way that's appropriate for the data structure.

Since the translate function itself seems fast, it will probably need an algorithmic improvement to reduce the overall number of translate calls. It might be helpful to see where it is getting called the most? I wouldn't be surprised to find out that we are processing paths more than once during baking, for instance.

–st.

[Mihai Sebea]

I'm attaching the whole profiling information here ...maybe some one can shed a light on how to better understand it 

I but some prints in the translator call and it seems we are calling it for each configuration 

even though the files are the same in all configurations .

Is there a way to cache the results maybe ?

Thanks ! 

Cheers

Mihai. 

[starkos]

Caching the shared paths seems like an excellent idea. You would just have to take care to ensure they don't contain any tokens (most won't) that could produce different output from configuration to configuration.

–st.

[Mihai Sebea]

why thank you ! I know it's a great idea ... :)

I'm just not sure where should i cache this though ... should i do it the field translate method ? 

and should i split all the paths and check for tokens ?

thanks !