Performance Concerns in Hosted environment

[pingm...@gmail.com]

From some basic testing, when loading scripts in a hosted environment, and assemblies are pre-compiled and cached, the css engine will still perform file checks to see if changes have been made.   In high volume scenarios where scripts are frequently loaded, this could lead to excessive overhead.   Some basic loading comparison of this vs asp.net's LoadControl functionality showed remarkably slow results (eg: 1second for csc vs 50 ms for asp.net).  

Are there any suggestions to avoid this overhead?  Are there any plans for CSS to enhance this model to improve speed?  One possible alternative maybe to check dependency time stamps once at startup, and have a watch on dependencies to detect and invalidate changes when files changes are detected? 

Thanks!

[Oleg Shilo]

It is really hard for me to comment the performance figures while I do not know the nature of the conducted test.

However the loading time  you quoted indicate that the caching mechanism does not work in your case. Most likely it is because some of the script dependencies are changed (even if the script is not) and this triggers script full recompilation. Interestingly enough I just answered a similar question/concern from the another thread: https://groups.google.com/forum/?fromgroups=#!topic/cs-script/IqjYl7i9BNc

In fact I have implemented the work around for cases like yours: when the default CS-Script caching may not be adequate. In such cases you can define your own cache probing mechanism taking the full control of the assembly/script loading. 

That post also contains the VS2012 sample demonstrating the simplified cache probing based only on the file timestamps (without analyzing the script dependencies as CS-Script does). This allowed to minimize the overhead down to 20 (sometimes just 6) milliseconds per assembly. Of course the overhead time depends on the system spec though it does not depend on the compiled script size.

I cannot exclude that in the future releases I will make the simplified cache probing as a default option depending how successful it is with the users. So please provide the feedback on the feature.

Regards,

Oleg Shilo

--------------------------------------------------------------------------------------------
Internet: http://www.csscript.net
E-Mail: csscript...@gmail.com

--
You received this message because you are subscribed to the Google Groups "CS-Script" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cs-script+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[pingm...@gmail.com]

Hi Oleg I should have clarified my testing a little more. All I did was use CSS to load a script file using the codedom approach, 1000 times, and do the same iterations to call asp.net load control. I believe CSS does cache ( only one assembly file created) but the file dependency probing creates noticeable overhead.

Having the custom probing as you mention does help. However, it would be a big productivity boost if CSS supported the alternative filesystemwatcher solution (or any other faster way) out of the box. I was looking through the source and It looks like the dependency checking is quite thorough but... Take a decent amount of time for the average dev to reuse.

Thanks for the explanations! :)

[Oleg Shilo]

>However, it would be a big productivity boost if CSS supported the alternative filesystemwatcher solution (or any other faster way) out of the box..

This is exactly what I meant by "...I will make the simplified cache probing as a default option depending how successful it is with the users."

Though you just convinced me and I have embedded the two implementations of the IsOutOfDate algorithm: 

Simplified - based on the timestamps of the script and the assembly

Advanced - based on the timestamps of the script, the assembly and all script dependencies

And starting from v3.5.7 Simplified is default caching algorithm.

While I always felt that analyzing dependencies in many cases would be an overkill I could not just ignore them as that would be a ticking bomb. This should solve the problem as now the default behavior is: if the script and its assembly have the same LastWriteTimeUtc the assembly loaded and returned immediately with Assembly.Load.

The fix is available from NuGet or from the HotFix page: http://csscript.net/LatestBuild.html

Regards,

Oleg

[pingm...@gmail.com]

Just my two cent but the advance checking approach would probably be better as default. I would guess most use case scenario don't require super high performance requirements -- hence more accurate dependency check would avoid unecessary questions as to why the default behavior does not cause scripts to reload correctly.

I was hoping I'd convince you to implement the advance approach as filesystemwatcher to avoid having to rewrite my own custom version specific that would be more limited :)

In case you were curious how I was planning to solve:
1. Add filesystemwatcher watcher on all scripts/assembles
2. When change is fired, purge script and all parent scripts upward.
3. Next time scripts are loaded, they won't be in cache and hence forced to compile.

[Oleg Shilo]

Please see my inline comments.

On Fri, May 3, 2013 at 3:08 PM, <pingm...@gmail.com> wrote:

Just my two cent but the advance checking approach would probably be better as default. I would guess most use case scenario don't require super high performance requirements -- hence more accurate dependency check would avoid unecessary questions as to why the default behavior does not cause scripts to reload correctly.

Agree, I was not sold on the default for the caching algorithm neither. I just wanted the HotFix to have the fastest option just out of box and review the defaults again before merging the HotFix with the main stream codebase. And you are right the more conservative probing ("advanced") seems safer.

 

I was hoping I'd convince you to implement the advance approach as filesystemwatcher to avoid having to rewrite my own custom version specific that would be more limited :)

It is actually easy to implement but I am very skeptical about the use-case for this feature

In case you were curious how I was planning to solve:
1.  Add filesystemwatcher watcher on all scripts/assembles
2.  When change is fired, purge script and all parent scripts upward.
3. Next time scripts are loaded, they won't be in cache and hence forced to compile.

It is trickier with this one. This sort of functionality is not a responsibility of the script engine. Also it will require heavy integration of CS-Script with the target OS (e.g. installing service or daemon). I deliberately do not want CS-Script to go this way and limit OS integration with the shell-extensions only.

Saying that, implementing of this sort of "script cache manager" makes very good sense on the host application side. Particularly because "cache discovery" it is an extremely easy task. The core implementation only takes 4 lines of code:  

static void VerifyScriptCache(string script)
{
    IsOutOfDateResolver isOutOfDate = CSScript.CachProbing.Advanced;

     string assembly = CSScript.GetCachedScriptPath(script);
 
     if (File.Exists(assembly) && isOutOfDate(script, assembly))
         File.Delete(assembly); 
} 

You will only need to provide the concurrency and error handling and of course a file system monitoring to trigger the cache verification.


I have prepared for you the simple/prototype to demonstrate the idea https://dl.dropboxusercontent.com/u/2192462/pingmustard/pingmustard.7z



Just be careful with FileSystemWatcher as it is not as practical as it seems. For example VS does not updated source files when you edit them. Instead it keeps your changes in the temp file and when you click "save" it deletes the original file and renames the temporary one. Thus the FileSystemWatched never raises "Update" but "Delete" and "Create" instead.