performance optimization

[matt]

Does anyone have a feel for the anything I could do to improve
performance for this exhibit ...
http://www.educause.edu/E2009/EDUCAUSE2009Programtest/171882

I'd really like to add several hundred more records (for individual
speakers) and several more discrete data elements per type (extra
facets), but I feel like I've already exceeded reasonable performance
thresholds and possible reached the upper bound of records that I
could reasonably look to use. Once the data loads, performance is
okay on FF 3.5b4, Chrome, and Safari 4 beta, but not really production
ready for IE and even older versions of FF that I've checked. Is
there anything that I should be doing to optimize my exhibit ... and
are there any rules of thumb related to number of facets, lenses, etc
that one might want to consider while looking to develop a high
performance widget. For instance, I was wondering if it would be more
efficient to have store formatted fields (dates, urls, etc) in the
json source as compared to formatting them on the fly.

Thanks in advance for any guidance ....

Matt

[John Callahan]

Hi Matt,

Nothing comes up on my screen using the latest FF and Chrome. Is your
Exhibit currently functioning?


It's tough to give benchmarks but I've had very good success with
several hundred records with many facets and views. This will of course
depend on the amount of data being passed and amount of post-processing
done via javascript. I would make sure at least your dates are in the
proper ISO8601 format. I don't think the concat expression adds to much
to the processing time. My guess is that adding many different types of
facets (list + slide + image + etc...) could increase processing time
more than several facets of the same type (all list.) Just a guess though.


I see you're using Drupal. Just as FYI, I did a simple test using the
Devel generate module to create 3000 random records with taxonomy.
Surprisingly, I was satisfied with the performance using all 3000 loaded
into the Exhibit module. I'm not sure if it sped things up or not, but
I saved the View (sql) output to a json file and loaded the data file
directly into Exhibit.


Using a combination of David's last email about loading the Exhibit
API's after the initial page load


http://groups.google.com/group/simile-widgets/browse_thread/thread/2ec56bfe32a6d851?hl=en

and another of his emails about loading additional data streams form
user input (Course Picker example)


http://groups.google.com/group/simile-widgets/browse_thread/thread/03a85b0fbc5cfaad?hl=en

could be extremely valuable when dealing with datasets in the several
thousands. If you're in the tens of thousands, that's a bit much an
Exhibit only solution.


- John

**************************************************
John Callahan
Geospatial Application Developer
Delaware Geological Survey, University of Delaware
227 Academy St, Newark DE 19716-7501
Tel: (302) 831-3584
Email: john.c...@udel.edu
http://www.dgs.udel.edu
**************************************************

[Matt Pasiewicz]

Interesting that folks are having trouble with the url ... can you check again?  Everything seems to be functional, but w/ two reports of inaccessibility, I hesitate to suggest it was just a blip.  And yes, I am using drupal, but I'm not (yet) using the exhibit module/views  ... the json is static.  I only have 400+ records, so perhaps all the other javascript on the page is exaggerating the problem.  

I'll also take a peak at the other two messages you mentioned.    

Let me know if you're still unable to access this page ...

http://www.educause.edu/E2009/EDUCAUSE2009Programtest/171882

Many thanks!

Matt

[David R. Karger]

I've just tried it from a different computer at a different institution
and it is still blank.
firefox both times.

[David Huynh]

Is any of the URL within your HTML intranet-only?

[Matt Pasiewicz]

Ah drat ... just realized that I'd been pointing a file to one of our dev sites, so it was resolving fine for me here locally, but would fail for everyone else ... sorry about that!  Should be back in action now.

http://www.educause.edu/E2009/EDUCAUSE2009Programtest/171882

[David R. Karger]

I'm finding it pretty snappy on my (new) machine.
One suggestion: exhibit facets are collapsable, and we've done some work
to make facets that are collapsed not bother computing. I'm not sure
it's in the released version, but while I check, would that be useful to
you?

Prerendering your html as json properties as you proposed would probably
give you good payback in speed too.

[David Huynh]

The tabular view is known to be slow because it renders all the items at
once and there is no setting to break the items into pages. It's one of
those long-standing issues that eventually I'll try to get to. Of
course, patches are always welcomed :) If you use the tile (default)
view instead, it should be slightly faster.

David

[Matt Pasiewicz]

> I'm not sure it's in the released version, but while 

> I check, would that be useful to you?

It could be ... it would certainly be interesting to see if it makes an impact on render time.  

> Prerendering your html as json properties as you proposed 

> would probably give you good payback in speed too.

If I can get comfortable with some baseline performance, I'll look into tweaking various options and share the results ... guess I was hoping for some known benchmarks or rules of thumb based on existing practice.  I'd think that there has to be some relationship between size of the file (with less client-side manipulation) vs. the smaller size w/ more client side parsing (when representing the same data).  Not sure which is more efficient ... probably based on a range of factors which is why I imagine it is harder to benchmark ... I imagine it depends greatly on where one wants their pain points ... on initial load or on certain events (sorting, etc).  

Thanks David!

Matt

[Matt Pasiewicz]

>  If you use the tile (default)
> view instead, it should be slightly faster.

Thanks!  I'll try that.  

> Of course, patches are always welcomed 

You bet.  If I can make the case for deployment and integration in lots of areas of our org, I can easily see us submitting patches ... we do this for other projects we've been involved with ... I just have to get us passed the initial boundaries.

[John Callahan]

The Tile View also offers an abbreviatedCount option to only display the
first 20 or so items, with a link to "show all results".
(http://simile.mit.edu/wiki/Exhibit/2.0/Tile_View)

Something else to try is pre-selecting a facet option. For example, a
random/featuered item from the Topic or Type facets could be
pre-selected. Or the current day of the week in the Day facet.

Drupal is known for pages taking a while to load. There are a lot of
factors that determine this. I would suggest testing the Exhibit
completely separate from Drupal in a static HTML page. I'd also test
the Drupal page without the Exhibit (use the Devel module) to test
loading times of different components of the page.


Something I would love to investigate further (based on those two links
in my last email) is a trick I've used often with online mapping.
(Before Google Maps, and even somewhat now, online maps are notorious
for long page loads.) I would load a static jpeg map while the real
maps and functions would continue to load. The user would think the map
loaded immediately and by the time they glanced at the map and through a
few options around the page, the full mapping data would be ready to go.

In this case for the EDUCAUSE Conference, maybe instead of the "Working"
flash screen, there would be text to select among the days of the week,
presentation types, and/or topics. The api would load in the
background; maybe the data as well. A selection would then generally
load the data with a pre-selected facet. Obviously, other possibilities
are there. The thought being to distract the user while the page first
loads. It worked great with mapping. :-)


Just a quick note...I'm a bit familiar with EDUCAUSE and the speaker
university/organization might be a good facet to use here. Good luck
with the rest. I think the site looks and feels great!

- John

**************************************************
John Callahan
Geospatial Application Developer
Delaware Geological Survey, University of Delaware
227 Academy St, Newark DE 19716-7501
Tel: (302) 831-3584
Email: john.c...@udel.edu
http://www.dgs.udel.edu
**************************************************

[contemplative]

I just took a look at you exhibit using FF3.0.1 and with firebug NET
enabled... It appears that the other code loading does
have an impact on the page load, but the queing of css and js seems to
be what is eating up most of the time.. there is
only 2meg actually being delivered, which indicates to me that the
rest of the time is probably spent processing and
displaying... total load time for me was 25.6 seconds. One thing I
noticed was that you have some .png files that take a long
time to que. the 'blackcheck.png' file and the 'downarrow.png' files
are taking 4 to 5 seconds each to que...

Hope that helps.

Nice looking page though!

wjw

[mleden]

Hi Matt,

Very nice looking Exhibit. I was particularly interested in how you
handled the "iPhone Version". However, the "iPhone Version" of the
URL does not appear to render on my iPhone, although it does on FF 3.0/
Win XP. Also, I noticed that with IE 8.0/Win XP I get an error on
your "main URL":
The JSON data file...contains errors...Syntax Error: Expected '}'...

-Mark

> > >     > Matt- Hide quoted text -
>
> - Show quoted text -

[matt]

Heh, I'd love to say that I worked up the iphone version, but I just
used the code David posted.
See http://groups.google.com/group/simile-widgets/msg/ee8af28a006a204a?dmode=source

Looks like it might be using an older version of IUI and Exhibit, but
it was just fun to see something close to working and know that with a
little more polish, it could be feasible to put together.

I haven't brought myself to look at it in IE yet, but I'll have to if
I wanna actually deploy this.

Thanks!

Matt

[Sergey Chernyshev]

The problem is simple - exhibit is not optimized for front-end performance - you can see the it on this waterfall diagram:
http://webpagetest.org/result/090520_266054de45d8f10d3461ae7e7dc0ecaf/

Biggest delay (18.5 seconds) is happening before the data in JSON is even requested on 43-rd request (http://webpagetest.org/result/090520_266054de45d8f10d3461ae7e7dc0ecaf/1/details/#request43).

Basically all of the code should be requested in one JS request and all styles in one CSS request, more over in ideal situation no JavaScript should even be loaded or executed before HTML and CSS loads, but it might be harder to achieve.

I think bundling techniques that SIMILE projects use (I experience same issues with just Timeline and Timeplot) can be dramatically improved to achieve better performance.

Thank you,

        Sergey


--
Sergey Chernyshev
http://www.sergeychernyshev.com/