Why is XSLCompiledTransform in 2.0 3x slower than XslTransform in 1.1

[AndrewF]

Hi there,

I'm hoping someone from MS can answer me this as I can't find any
reasoning behind this.

I have a C# application that has been in service for about 3 years and
because of other features requiring .NET 2.0 functionality it was
decided to port everything over to 2.0 from 1.1

Basically the app takes a load of XML files, processes them using an
XSL template and spits out some rendered documents. Repeat until
finished.

Now the interesting thing is that up until this week I had *left* the
transform code using the old 1.1 classes because there is a lot of
pfaffing required to use the new ones - it isn't as straightforward as
"load this file into an object, transform against this document".

I decided though that after seeing lots of items about it being better
supporting of the new XSLT features and apparently quicker I would
migrate it.

Imagine my surprise when the whole process takes 3x longer to execute
than it did before!

The reason I believe is to do with the *extra* work you have to do to
make the thing work. I know from profiling that the *actual*
transformation is quicker but the prework required creates an overhead
that in high volume file usage like my app means a huge bottleneck is
created.

I think most of this is down to not being able to use the old types of
document for the XSL and XML documents anymore and thus are slower.

instead of:

XslTransform xsldoc = new XslTransform();
xsldoc.Load(file);

We now end up with

XmlReaderSettings settings = new XmlReaderSettings();

settings.ProhibitDtd = false;
settings.IgnoreWhitespace = false;
settings.IgnoreComments = false;
settings.IgnoreProcessingInstructions = false;

_xslt.Load(XmlTextReader.Create(value, settings));

In order to get the same whitespace behaviour and this has a
computationally higher load.

We also need to pass the transform function an input stream rather than
a string or file so this has higher load also [particularly if being
done many times].

In .net 1.1 we did:

_xsl.Transform(_xmldoc, xslArgs, stringwriter, null); as the transform
call which was perfect if you had manipulated your XML before
transforming.

Now with 2.0 we end up with:

StringReader srxml = new StringReader(_xmlstring);
_xmlt = new XmlTextReader(srxml);

_xslt.Transform(_xmlt, xslArgs, stringwriter);

Certainly if someone knows of a better way to do this than I have I'd
be much appreciated as I need to deal with my XML documents as
documents before sending them to transform.

On tests using the old binaries a two hundred files took approximately
12 seconds process and transform. On the same test case data with the
new binaries the same task took 38 seconds to process. The exact same
code is executed everywhere else other than the transformation step
which uses the two versions I documented above. Nothing else is
different between the classes.

Has anyone else had any experience of this or can anyone from MS from
the XSL team shed any light on this? I'm willing to be proven wrong on
this, however my impressions of XSLT in .net 2.0 are not particularly
astounding given the hype.

For the time being I'm rolling back my XSLT work to keep my clients
happy.

Cheers
AndrewF

[AndrewF]

Supplementary to this I did some more testing and there was something
that I didn't notice before which is a speed increase on subsequent
executions.

Digging back into the documentenation illustrated that the XSLT
instructions are in fact compiled in order to elicit quicker response
times later on. Whilst this was evident in my testing, creating
approximately a 20% reduction on subsequent execution time, the 300%
increase in execution time by virtue of the code changes far outweight
this benefit.

Again, I'm willing to be told I've executed my code wrongly if someone
has noticed something I haven't...

Cheers
AndrewF

[Kevin Spencer]

Hi Andrew,

I have used the new classes as well, and what I realized was that since the
XSLT that I use doesn't change often, I could add it to my project as a
string resource. That's neither here nor there, but it relates to the
following code. The file could just as easily be kept in the file system of
the app. Then, I created a field and a property which is
XslCompiledTransform, and initialize it when the app starts:

private XslCompiledTransform _HtmlTransform;
/// <summary type="System.Xml.Xsl.XslCompiledTransform">
/// <c>System.Xml.Xsl.XslCompiledTransform</c> for
/// converting serialized PDD (XML) to XHTML
/// </summary>
public XslCompiledTransform HtmlTransform
{
get { return _HtmlTransform; }
}

I call the following method in the constructor of the class it resides in:

// Initializes the XSL Transforms for this instance.
private void SetupTransforms()
{
if (_HtmlTransform == null)
{
try
{
XmlReaderSettings readerSettings = new XmlReaderSettings();
readerSettings.ProhibitDtd = false;
XsltSettings transformSettings = new XsltSettings(false, true);
_HtmlTransform = new XslCompiledTransform();
TextReader reader = new StringReader(
Properties.Resources.HtmlTransform);
XmlReader htmlReader =
XmlReader.Create(reader, readerSettings);
_HtmlTransform.Load(htmlReader, transformSettings, null);
reader.Close();
}
catch (Exception ex)
{
Utilities.HandleError(ex);
throw;
}
}
}

In a sense, the prepared XslCompiledTransform is "cached" as a
field/property of the class, for the lifetime of the class. It only has to
be created once. After that, it's a simple matter of calling the Transform
method, as in the example below. Note again, that in my case, I needed to
serialize an object itself to an XML document in a MemoryStream, and then
Transform it, which you don't need to do if you're working with files.:

/// <summary>
/// Serializes this instance as a PDD (XML) document, then
/// Transforms it using the <see cref="HtmlTransform">HtmlTransform</see>
/// instance.</summary>
/// <param name="filePath">Path to save XHTML document to.</param>
/// <param name="overwrite">If exists, overwrite?</param>
public void ExportAsHtml(string filePath, bool overwrite)
{
XmlReader pddReader;
XmlReaderSettings readerSettings = new XmlReaderSettings();
readerSettings.ValidationType = ValidationType.None;
XmlSerializer serializer;

if (File.Exists(filePath) && !overwrite)
throw new IOException("File " + filePath +
"exists. Parameter \"overwrite\" must be true to overwrite");
try
{
PrepareToSerialize();
using (MemoryStream ms = new MemoryStream())
{
serializer = new XmlSerializer(typeof(PrinterDocument));
serializer.Serialize(ms, this);
ms.Seek(0, SeekOrigin.Begin);
pddReader = XmlReader.Create(ms, readerSettings);
TextWriter writer = new StreamWriter(filePath);
_HtmlTransform.Transform(pddReader, null, writer);
}
}
catch (Exception ex)
{
Utilities.HandleError(ex, "filePath: " + filePath);
throw;
}
}

I think you'll find this methodology much more efficient than re-creating
your XslCompiledTransform each time you need to use it.
--
HTH,

Kevin Spencer
Microsoft MVP
Short Order Coder
http://unclechutney.blogspot.com

The devil is in the yada yada yada

"AndrewF" <and...@transitionkiteboarding.com> wrote in message
news:1162046317.5...@k70g2000cwa.googlegroups.com...

[Dimitre Novatchev]

I think Kevin said it very well.

In addition, do read Anton Lapounov's weblog on this topic -- straight from
the horse's mouth :o) :

"XslCompiledTransform Performance: Beating MSXML 4.0"
http://blogs.msdn.com/antosha/archive/2006/07/24/677560.aspx

and

XslCompiledTransform slower than XslTransform?
http://blogs.msdn.com/antosha/archive/2006/07/16/667221.aspx

Cheers,
Dimitre Novatchev

"Kevin Spencer" <sp...@uce.gov> wrote in message
news:eDloqaq%23GHA...@TK2MSFTNGP03.phx.gbl...

[AndrewF]

Hi Guys,

Thanks for both of your input - very much appreciated and I'd seen
those documents already Dimitre in my search for optimisation.

Kevin, my approach isn't that dissimilar to yours when I'm working with
the files essentially my own translation class gets created and as part
of the constructor of that class one needs to pass in the XSL document
which then gets loaded. The next step is to pass in the list of XML
documents that need to be transformed which then get put through some
washing code then finally transformed against the XSL document.

I can see the caching working happily as the first call on the first
document in the test set takes nearly a half a second to do the
transform and then virtually nothing after that as the rest are then
transformed using the compiled and cached code. When run a second time
there are certain optimisations in play as well as the whole process is
sped up by about 20% as well - probably because of the JIT and the
compiled and cached XSL for read etc as also.

I'm going to do some further profiling on this today and see the
performance hits on the individual blocks of code. I can't see that it
will be due to the read of the XSL document as this only needs to be
done once for any "n" files so it's a static factor. I personally think
it is down to the way one has to cast the working XML document to an
XML reader before doing the transformation on it as this is an
additional step you don't need in the older version.

I'll post the results later when I've run them experimentally.

Also out of interest Dimitre, I tried to NGEN the XSL processor as
mentioned in Anton's blog and it made a difference of exactly 0 in
terms of execution time so that leads me to believe the error is coming
in the execution loop rather than start up times.

Cheers
AndrewF

[Kevin Spencer]

Keep us posted, Andrew!

--
HTH,

Kevin Spencer
Microsoft MVP
Short Order Coder
http://unclechutney.blogspot.com

The devil is in the yada yada yada

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162279691.5...@e3g2000cwe.googlegroups.com...

[ElenaD...@gmail.com]

Your suspicions about new ways of creating XmlReader are baseless --
new implementation of XmlReader (one that returned to you by
XmlReader.Create()) is faster then XmlTextReader. You still can use
XmlTextReader as well.

XmlReaderSettings give you better control over reader creation but
there is no need to use it if you don't need this control:
XslCompiledTransform xsl = new XslCompiledTransform();
xsl.Load(file);
Would do work just well.
The same about executing transform:
xsl.Transform(_xmldoc, xslArgs, stringwriter, null);
would work either.

All known differences between XslTransform and XslCompiledTransform are
outlined in the:
http://blogs.msdn.com/xmlteam/articles/Introducing_XslCompiledTransform.aspx.

Old version of your application (you run for 3 years in .Net 1.1) would
also benefit if you reuse XslTransform.

[AndrewF]

Hi all,

First up, thanks Elena for the heads up on the XML Reader class - to be
honest I wasn't expecting there to be a huge problem with it but
couldn't see such a massive performance drop being based solely on the
XSLCompiledTransform class.

Also thanks for noting that I can in fact pass in the original
XMLDocument object rather than casting it to a text reader first. I've
done that now as a point of optimisation and it will make it easier to
do a direct comparison on execution times.

So here's what I did - I threw a stopwatch timer around the specific
segments of code so I could see how long it takes to do things. I put
it around the XSL load code at the start, the actualy Transform()
method itself and also around the processing code I to produce the XML
just for point of comparison so I can make sure the code was equal in
terms of general performance.

I ran this 50 times for each form of the code so I could get the
benefits of any caching. The machine was also rebooted when switching
from the old code to the new code so it was running "fresh" each time.

The test case was processing 350 XML files using a single XSL template
which comprised 8 individual "included" files. The XSL is quite
complext with a lot of control statements in it to generate branching
and heavy template re-use. IE it is a real-life example of the XSL
required to produce the templates for a medium sized web site.

Old code first:

First load of the XSL template: 233ms
Subsequent loads of XSL template: 11ms average

XML Processing Time: 18ms

Transform() time: 5ms average

New code:

First load of the XSL template: 71ms
Subsequent loads of XSL template: 30ms average

XML Processing Time: 18ms

Transform() time: 105ms average

This is a *direct* comparison of the XslCompiledTransform class to the
XslTransform class in a production environment using real-life code. As
you can see the averages of the XML processing time are equal and
therefore the binary isn't different enough to cause execution time
changes from the old to the new code.

What is interesting is that the first load is substantially quicker
using CompiledTransform as well though subsequent ones aren't so quick.
Note that this test didn't compare the "load times" of the actual
XSLDOC.Load() method, rather it test the "process times" of the load
process. This is because in XSL Tranform you can just pass a file name
to the document and it will open it. In the compiled transform you have
to do other work as indicated in my first post *in order to get the
same behaviour* in terms of white space etc.

The direct comparison between the Transforms though is staggering.
Whatis even more interesting is that I initially ran this with my
"casting" code taking the XML document and turning it into an
XMLTextReader and this didn't really alter the performance of the new
code. There was about a millisecond of difference in real terms as far
as performance goes.

So the next question is why is this the case? Has MS not tested this on
really difficult XSL? Are there optimisations we can do with the XSL.
In all seriousness the XSL that I saw used on Anton's blog with the
chess stuff isn't particularly difficult code, it is just executing
lots of times in loops and maybe this is where we get some performance
increase...

Any ideas guys?

Cheers
AndrewF

[Oleg Tkachenko [MVP]]

AndrewF wrote:
> New code:
>
> First load of the XSL template: 71ms
> Subsequent loads of XSL template: 30ms average
>
> XML Processing Time: 18ms
>
> Transform() time: 105ms average

It would be helpful to see your code, but why do you need "Subsequent
loads of XSL template" stage? The whole point of XslCompiledTransform
class is that you reuse it. Compilation takes time you know, don't
recompile stylesheet without necessity, cache it.
Try to test with cached XslTransform vs XslCompiledTransform to see the
real difference in performance.

--
Oleg Tkachenko [XML MVP, MCPD]
http://blog.tkachenko.com | http://www.XmlLab.Net | http://www.XLinq.Net

[Kevin Spencer]

You should only be loading the XSL one time, as per my original reply. Here
is a repost of the example I gave you:

XmlReaderSettings readerSettings = new XmlReaderSettings();
readerSettings.ProhibitDtd = false;
XsltSettings transformSettings = new XsltSettings(false, true);
_HtmlTransform = new XslCompiledTransform();
TextReader reader = new StringReader(Properties.Resources.HtmlTransform);
XmlReader htmlReader = XmlReader.Create(reader, readerSettings);
_HtmlTransform.Load(htmlReader, transformSettings, null);
reader.Close();

--
HTH,

Kevin Spencer
Microsoft MVP
Short Order Coder
http://unclechutney.blogspot.com

The devil is in the yada yada yada

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162461523.4...@e3g2000cwe.googlegroups.com...

[AndrewF]

Hi guys,

Thanks for the feedback.

In the application, I am only loading the XSL once at the start and
then I transform as many times as is needed for the number of XML
documents that need processing. In this case 350 XML documents. So
there is absolutely only 1 XSL document load which is then used 350
times in a transform.

After the application has finished processing it then exits.

Note that this is a standalone console app it isn't used as a web
script etc. It does some batch processing and then quits.

What I found when I ran it was that the first time I ran the
application it took longer to load the XSL document using both the new
and the old code but on subsequent runs it then took the same amount of
time which was shorter to load it afterwards. Hence the information was
being cached and reused on subsequent executions.

Thus when I ran each version of the application 50 times, the 1st time
it was run I got the first load times [233ms and 71ms] and then the
other 49 times I got more or less the same but quicker loading times of
11ms and 30ms.

Oleg, The pertinent bits of my source code are displayed in my first
post, and the classes are really too big to throw into here.
Realistically, all I've done is compare the old with the new versions
of XSL transformation using an existing class that does a *lot* of
other processing.

In real terms the bottleneck in this application is the time required
to process the XML which can be up to 1 or 2 seconds depending on the
complexity of the created documents. However that is equal from one run
to the next so if I'm introducing a bottleneck using the compiled
Transformation class of an extra 100ms for the transformation then that
starts to cause problems down the line when I'm transforming several
thousand XML documents in one go...

If any of you guys have any ideas for optimisation I'll gladly hear
them and I'm willing to be swayed - to be honest I'd prefer to use
XslCompiledTransform because I get extra XSL options that you don't get
with the XslTransform class and that would be good, but the time
penalty is still too high to use this in production at the moment.

Cheers
Andrew

The

[Kevin Spencer]

I think I understand now. You load the XSL one time and re-use it to process
several hundred to thousands of XML documents in a "batch." It is
interesting that the XslCompiledTransform is doing the job more slowly than
the XslTransform class. Without seeing the code, it would not be possible to
offer any optimization ideas, though.

If it's any consolation, the XslTransform class may be deprecated, but it
will remain available until you move to the .Net 3.0 platform, which could
be quite awhile.

--
HTH,

Kevin Spencer
Microsoft MVP
Ministry of Software Development
http://unclechutney.blogspot.com

I just flew in from Chicago with
a man with a wooden leg named Smith
who shot an elephant in my pajamas.
So I bit him.

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162646191.1...@e3g2000cwe.googlegroups.com...

[AndrewF]

Hi Kevin,

Yes this is exactly what I'm doing with it and it does seem strange to
me that on a transformation only basis the new code is slower but I
wonder whether the MSIL that gets created during the compilation stage
in the new class is optimised for a certain "style" or method of
building XSLT and our stuff isn't in line with that so the raw
interpreter works better than the compiled instructions which are
perhaps adding redundant steps...

What would be interesting would be to see if there is a way we can see
the compiled instructions and see if there is anything obvious. When
I've run this across different stylesheets it is interesting that a
couple of the templates I personally thought were quite complex
actually performed better than the test sample indicates which I think
of as less complex XSL. This is what leads me to the "style" idea
above...

I'm going to try and construct a stripped down example of my code and
run the tests again and then post the code up for additional scruitny.
At the moment there is too much code and too many dependencies to get
my application running to post it up here but I'll get a test case
working that uses these different XSL documents so we can do comparison
of style on the two transform classes.

Cheers
AndrewF

[Kevin Spencer]

There is definitely an element of optimization to writing XSL. That much I
can tell you.

--
HTH,

Kevin Spencer
Microsoft MVP
Ministry of Software Development
http://unclechutney.blogspot.com

I just flew in from Chicago with
a man with a wooden leg named Smith
who shot an elephant in my pajamas.
So I bit him.

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162915031....@m73g2000cwd.googlegroups.com...

[AndrewF]

Hi Mate,

Is there a way to get the code that is compiled from the stylesheet as
I think that would be the next logical step in terms of looking at the
differences between the two and looking at optimisation paths.

Does anyone know of any optimisation rules etc that have been designed
for XSLT with the .NET 2 class? If anyone from the XML team is reading
this - what design descisions have you taken on the compilation stage
of the XSL?

Cheers
AndrewF

[Sergey Dubinets]

Show the money. It hard to say what wrong without looking on your code.

It shouldn't take long to compose small app that runs both transforms and
compare the time. Can be done in two dozens lines.
You don't need thousands of source documents to proof your statement. It
would be enough to provide a few that really make the difference.

To see what XslCompiledTransform is doing you can compile stylesheet with
debugger flag set to true (new XslCompiledTransform(true)) and then use
ildasm or reflector to disassemble the assembly generated in temp dir. Good
luck with this.
Even non optimized XslCompiledTransform should over perform XslTransform.

Sergey Dubinets

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162978938....@i42g2000cwa.googlegroups.com...

[Kevin Spencer]

Hi Andrew,

Here are some resources I have found on the subject:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/ScaleNetChapt09.asp
http://support.microsoft.com/default.aspx?scid=kb;en-us;320847
http://delivery.acm.org/10.1145/1100000/1096618/p55-kenjii.pdf?key1=1096618&key2=0203103611&coll=&dl=ACM&CFID=15151515&CFTOKEN=6184618

--
HTH,

Kevin Spencer
Microsoft MVP
Ministry of Software Development
http://unclechutney.blogspot.com

I just flew in from Chicago with
a man with a wooden leg named Smith
who shot an elephant in my pajamas.
So I bit him.

"AndrewF" <and...@transitionkiteboarding.com> wrote in message

news:1162978938....@i42g2000cwa.googlegroups.com...

[Anton Lapounov]

Hi Andrew,

I am one of XslCompiledTransform developers. If I understand correctly you
call Load once, then Transform 350 times. You may double-check your findings
with XsltPerf utility linked at the bottom of my blog post:

http://blogs.msdn.com/antosha/archive/2006/07/16/667221.aspx

Then you may send me your XSLT and some of XML files (not necessary all 350
of them), and I will take a look. My work email may be found here:

http://antosha.com/workemail.gif

Thanks,
Anton Lapounov
System.Xml developer

[AndrewF]

Hi Sergey, Kevin and Anton,

Thanks to you all for your responses.

Kevin, I had come across the first of those links in my travels but the
others I am going to wade through today and see what can be done.

Sergey - the problem was more writing something to output the XML that
I'm inputting into the XSL transformation rather than anything else as
the transform bit of my code as you rightly say is only a dozen lines
but the XML processing code was several hundred.

I've managed to sort that out though and have used Anton's XsltPerf
tool to run it against a few XML docs using XSL.

What I found was really interesting and we'll forget about Loading
Times of the XSL documents because as I've stated above, I am loading
the doc once into memory and then transforming up to a few thousands
times so that isn't really an issue.

I used Anton's code as my basis of where I put my stopwatch timers so I
was comparing like with like and we are both doing exactly the same
thing starting on one side of the Transorm and stopping on the other.

In my app the average Transform times were [across 50 iterations]:

XT: 20.32ms
XCT: 32.78ms

In Anton's app the average Transform() times were [across 50
iterations]:

XT: 59.48ms
XCT: 40.44ms

I should note that these were run on two different machines however the
only difference between them is processor speed.

Consistently in my app the new transform function is slower than the
old one, though in Anton's app his is consistently faster.

What is very strange is that on the old version of the transform
function the deviation from the mean is minimal. IE when you run this
50 times you get very small changes in the execution time of the
transform function. However with XCT you can get some wildly differrent
results.

Using Anton's tool as it is decimally more accurate and seems to be
good for benchmarking and leaving aside the 1st transformation which
always seems to take longer.

I found that achieving a 40ms average using XCT could give you a lot of
variation from 8ms up to 102ms which equates to 20% of the average time
up to 250% of the average time. Using XT you would get less variation
- 39ms to 137ms which equates to 65% of the average to 2.3% of the
average.

So it seems to me that XCT can give you a lot more variation in results
than XT which never seems to go that fast in real terms but is more of
a slow and steady candidate overall.

Finally, somethign else I tried to see if it had an effect was to make
Anton's tool use DTDs. The XSL in question actually has some entity
declarations in them and I had to remove them to make it work using
XsltPerf. I then decided to make XsltPerf work using DTDs by adding an
XmlReaderSettings object and loading the XSL that way - which is the
same as I need to do in my app.

Interestingly there was a performance hit on this and it put the
average execution time up from 40.44 ms to 42.67ms across 50 runs.
Extremely interestingly, the variation I indicated earlier disappeared
at this point and we still got a minimum of about 8ms but the max
dropped to 60ms which is a much less erratic way of doing it.

I wonder then if some of the performance hit I'm getting could be borne
out by the fact I'm using XSL Attributes and DTDs? Both of which aren't
present by default in Anton's benhmark app...

Any thoughts gratefully received.

Cheers
AndrewF