turns out XSLT transformation is very CPU-intensive

41 views
Skip to first unread message

Martijn van de Rijdt

unread,
Jan 30, 2013, 11:57:38 AM1/30/13
to formh...@googlegroups.com
I did some testing on the performance of the XSLT transformation on a very large form (Bao's). Turns out that this takes up 100% CPU utilization for an extended period. 

AWS t1.micro instance: 10 minutes
AWS t1.small instance: 3 minutes
local VM with one 3.1GHz core: 36 seconds

Memory upgrade has no noticeable (under light load) impact (612Mb seems fine on local VM), nor has multiple cores (on local VM).

There are likely to be further improvements to be made in the XSL stylesheet. The key remaining issue is the way text strings are included in the form format, which is just very inefficient for XSLT to deal with. This takes up 85% of the transformation time. So a solution may be to move this work to the client (after XPath performance has been improved). Caching of the transformation results will help a lot but for every first time a long form is requested from enketo as well as for all preview loads, the transformation time is not really acceptable for forms with a lot of text strings.

Can we upgrade enketo.formub.org to t1.medium ($826/year) or probably even better to a high CPU-medium ($1050) instance, for the near future?

Matt Berg

unread,
Jan 30, 2013, 12:05:06 PM1/30/13
to formh...@googlegroups.com
Martijn,

How is performance on most of the average forms we're dealing with.  This is all important but want to make sure we're not focusing a lot of time and budget if it's just for forms that are real outliers.

In general, enketo feels very performant.  For bigger forms like a health facility form, it does take longer but I also expect it too so I don't really mind.

What matters more to me (not sure if this is related) is how responsive is the form when it's loaded in the browser.  As we are considering enketo to replace our forms in india this is a major concern / priority.

Not saying we don't upgrade just trying to get some better perspective on all of this.   

Matt


--
You received this message because you are subscribed to the Google Groups "Formhub Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to formhub-dev...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Martijn van de Rijdt

unread,
Jan 30, 2013, 5:33:27 PM1/30/13
to formh...@googlegroups.com
The 'average form' is transformed quickly, e.g guita.enketo.formhub.org/webform takes less than a second. I think special (vietnamese) characters play a performance role in XSLT too which would explain the extreme results. Nevertheless, I think enketo should be able to do serve this form faster (than 10 minutes in which the server is probably not very responsive to other users) and a server upgrade would be a temporary 'solution'. 

Yes, client-side processing performance needs a lot of improvement (and I have 1 idea that could make a huge difference if it works). Let me know if you want me to prioritize this over file uploads and/or authentication.

Matt Berg

unread,
Jan 31, 2013, 2:36:02 AM1/31/13
to formh...@googlegroups.com
Martijn,

I'd push on attachments a bit more especially if its showing progress then shift to performance.  It's a major priority but we have a bit of time to pursue it.

What is enketo running on again now?

Thanks,

Matt 

Martijn van de Rijdt

unread,
Jan 31, 2013, 10:14:57 AM1/31/13
to formh...@googlegroups.com
Yes, sounds good. I think it's running on t1.micro. Is it, Mark?

Mark E Johnston

unread,
Jan 31, 2013, 11:48:45 AM1/31/13
to formh...@googlegroups.com
looks like enketo is a m1.small and enketo-dev is a t1.micro, let me know if you want me to adjust

mark

Martijn van de Rijdt

unread,
Jan 31, 2013, 12:12:17 PM1/31/13
to formh...@googlegroups.com
Thanks. If we can afford it, a (temporary) upgrade to t1.medium, or better to high CPU-medium, would have my vote! 

(the vietnam form takes 4.4 mins to be transformed and served on prod, tested dropping the xml file here: http://enketo.formhub.org/formtester?debug=true)

Martijn van de Rijdt

unread,
Jan 31, 2013, 3:56:47 PM1/31/13
to formh...@googlegroups.com
I see now they're called m1.medium or c1.medium
mark


Matt 

To unsubscribe from this group and stop receiving emails from it, send an email to formhub-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Formhub Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to formhub-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Aid Web Solutions - www.aidwebsolutions.com


--
You received this message because you are subscribed to the Google Groups "Formhub Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to formhub-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Formhub Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to formhub-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply all
Reply to author
Forward
0 new messages