Hi Jennifer, Jordan and Peter
I have been working hard on a fix this weekend and i have found some solutions that i would like to discuss before implementing, basically because it implies a "big change".
With the current architecture is pretty easy to make Solr Views (and Views in general) go out of memory, timeout or even surpass the max number of Input variables in php. Sadly it is related on how Views module fetches and renders the fields to be used. The ideal situation would be to enable a pager in then add field popup, but looking closely at the Views module code it's a very big deal and could easily introduce a lot more problems and also does not assure a quicker load, because we would be basically altering that form (which would be constructed anyway by the module the first time).
That said: this is my proposal (i have it almost working, lots of code, i like it).
Instead of having all Solr fields as individual "fields" to be added and listed, i modified the whole mechanic to use only a few generic ones, The base ones (PID, score, islandora_manage, solr_collection_count, query_luce, query_dismax) + a few configurable ones based on the Solr field type: Solr sortable, Solr general, Solr Date, (we could add some more if needed).
This reduces the number of fields our modules exposes to the minimum. Each of this last fields have now an extra configuration option, the real Solr field name to be used (defaults to PID) as source. So you add a Solr general field, you edit it, you have a Luke autocomplete field and you choose which field you will be querying. This autocompletes are modal, means "at Sortable one's edit form" you get fields that are single valued and indexed only, etc.
Pros: quick loading of fields, you don't need to remove fields and re add them if you need to change a source solr field (as before), means you can use an existing Solr view in a particular view and fine tune the source fields without having to remove and re-add fields.
Cons: you have to rebuild all your existing Solr views, because they will be no longer compatible!
Since the cons means remaking your already existing Solr views and i'm sure this will scare a lot of people!, we(or i) could make a transform routine, basically i just need to fetch the field name and put it into an option inside one of those generic Solr views fields. This could be implemented in a update function in the install file of the module, as a drush script, whatever you like more.
What do you think? It's an open discussion of course, but having a regex that removes some fields makes me also not happy, because only if you are using a very standard solr schema, it's very possible that we end stripping fields that we could need or not stripping anything. I prefer to give users the full power to decide what to use and where.
Best.
Diego
PS: copying this also in the JIRA to get some feedback.