Umlaut's solr index feature?

[Jonathan Rochkind]

Hi all,

Umlaut presently has a feature where it will build a Solr index for you
to power a local A-Z index. The Solr index is based off the SFX db, but
then Umlaut can send queries to the local Solr index instead of the SFX db.

While a neat idea, it's proven somewhat hard to maintain, and both me
and Scot are generally trying to move away from depending on Umlaut or
SFX for "A-Z list" purposes (at my institution, I'm trying to focus on
letting the catalog serve those purposes).

Is anyone using this feature in Umlaut? We're thinking about removing
it. One way or another, it probably won't get a whole lot of support
attention from either of us, but if people are using it, we'll try to
provide a less abrupt ripping out if we can.

Thanks for any feedback!

Jonathan

[PHILLIPS M.E.]

Hi

Does this mean that the A-Z display would be removed altogether from the Umlaut interface? We have not implemented one here yet because we do not have SFX, but I had wondered about using the Umlaut one with a different back-end so that we could improve on our rather nasty A-Z lists on our home page (done using PHP/MySQL and based on an export from the catalogue:
https://www.dur.ac.uk/library/resources/online/ejournals/

They have the A-Z journal index in use at the Danish Royal Library:
http://e-tidsskrifter.kb.dk/?umlaut.locale=en

I know what you mean about getting rid of A-Z. Our serials librarian here thinks it is a silly way to provide journal access and I'd tend to agree. However, it is enormously popular still. The Umlaut interface scores over our current listing in having paging for each letter and a search box with auto-complete. Some of my colleagues think we should rely on a restricted search scope in Primo, but some users really seem to prefer point and click to any typing. The Umlaut A-Z interface is still superior to any alternative we have here so I had been planning to implement it in due course. So if you can retain code that's needed for A-Z, we'd appreciate it. Is it the Solr aspect, or the interaction with SFX that's hard to maintain?

On a separate matter, we now have some web developer effort going into our local Umlaut implementation again. One of our developers put in a minor pull request a few weeks ago (changing the name of Web of Knowledge to Web of Science) and I hope we will soon be able to contribute back some other code, once we have Primo launched for the start of the new academic year.

Matthew

[Jonathan Rochkind]

Nope, we're not planning on removing the A-Z display altogether from the
Umlaut interface. It will remain, there are no plans to remove it!

The A-Z list is architected to support different plug-in modules for
sources of info to look up/display.

We're just talking about removing the module that uses a local Solr
index, built from a database dump from SFX. Are you using that module,
Matthew? If you don't know whether you are or not, you're probably not.

The module that queries the SFX database directly will remain. That will
in fact be the only module distributed with Umlaut, although others are
possible are in the future, especially for use with non-SFX environments.

I hadn't noticed that pull request from your developer -- feel free to
bring these to my attention if I've missed em. This is like the fourth
time they've changed the name of that product! I will change the
defaults for the upcoming Umlaut 5.0 release -- but in the meantime, you
can easily customize this label yourself in the
./config/umlaut_services.yml file (or in upcoming Umlaut 5.0, in Rails
i18n locale files). Let me know if you need help doing that.

Also, Matthew, I am still interested in seeing some of the custom work
you have done on custom adapters etc shared back with the community. I
think it's part of being a good community member when you are benefiting
from others work shared open source.

Jonathan

[PHILLIPS M.E.]

> The A-Z list is architected to support different plug-in modules for
> sources of info to look up/display.
>
> We're just talking about removing the module that uses a local Solr
> index, built from a database dump from SFX. Are you using that module,
> Matthew? If you don't know whether you are or not, you're probably not.

No, we're not using it. Sounds like we can simply write something to get it from our own database, which is nice.

> I hadn't noticed that pull request from your developer -- feel free to
> bring these to my attention if I've missed em. This is like the fourth
> time they've changed the name of that product! I will change the
> defaults for the upcoming Umlaut 5.0 release -- but in the meantime, you
> can easily customize this label yourself in the
> ./config/umlaut_services.yml file (or in upcoming Umlaut 5.0, in Rails
> i18n locale files). Let me know if you need help doing that.

We're OK about modifying it locally. Our web developers prefer to pass stuff back up to the main branch where possible as it reduces local maintenance needs. We're thinking about adding some sort of global flag for removing debug data from the page -- our developer would like to retain it on our development and test instance, but remove it completely from the production system (not just hide it with CSS but remove all traces from what gets passed to the browser) -- so they may submit some code to provide for this at some stage. It's not a high priority at the moment though.

> Also, Matthew, I am still interested in seeing some of the custom work
> you have done on custom adapters etc shared back with the community. I
> think it's part of being a good community member when you are benefiting
> from others work shared open source.

I fully agree and I am getting more and more embarrassed about this. I have to get sign-off from the legal support person here bit it is taking a while as they have to get approval from senior university management I think. Our web developers are keen to do pass everything back which we can.

I'm not sure how much will be generally useful to SFX users as quite a bit of it is to do with making Umlaut work without SFX, so we've improved the CrossRef connector, and built a Millennium connector. But we've also written an Altmetric connector which might be nice for general use, and my colleague wants to explore DOI look-ups to the DataCite service.

Once we go live, later this month, I'll be able to catch up a bit and tackle the legal people.

All the best,

Matthew

[Jonathan Rochkind]

Thanks Matthew!

I am keenly interested in making Umlaut more useful to those without SFX
-- even though we still use SFX here (for now), I have been gradually
reducing the parts of SFX that our own Umlaut configuration uses,
replacing with more dependable parts built into Umlaut (for instance,
working on switching to Umlaut-generated links out to ILLiad instead of
relying on SFX for them).

Much of the stuff you've done sounds pretty interesting to me, for sure.
Some of it, it might make more sense to release as an extra plug-in gem,
rather than built into Umlaut. (No difference in how you write the code,
just a question of where it lives).

I totally agree and appreciate your pushing fixes upstream to Umlaut
instead of just making local changes. However, in this case, local
configuration of display text and labels like this is totally expected
for Umlaut installations, everyone won't agree on these. (although also
in this case it's totally appropriate to update the defaults, thanks for
the pull request; it's hard to keep it up to date when the company keeps
switching the name back and forth every couple years!).

I am curious why you are concerned with the debug info being included on
the page? Although I have no problems with a pull request to make it
suppressable with configuration, would be happy to review that. For us
here, we find that debug info truly invaluable for troubleshooting
access problems. But if you think it's a security concern, for instance,
I am very interested to know why in case it's something I should be
concerned about too!

Also, just to make sure you know, since you are about to go live with
Umlaut, I am currently working on the next major relase of Umlaut, which
among other things will support (and may require) Rails 4.x.

Jonathan

[PHILLIPS M.E.]

> I am curious why you are concerned with the debug info being included on
> the page? Although I have no problems with a pull request to make it
> suppressable with configuration, would be happy to review that. For us
> here, we find that debug info truly invaluable for troubleshooting
> access problems. But if you think it's a security concern, for instance,
> I am very interested to know why in case it's something I should be
> concerned about too!

I have not investigated the debug stuff thoroughly yet: it was just that one of our web developers voiced concern at some of the content. I think there was debug information that included the URL being used to connect to an API, and the URL contained our username/password. I suspect this was not in anything you had written and supplied with Umlaut, but probably something I had bunged in myself when developing the CrossRef service adaptor.

I'll try to look into what there was that might be a concern. I'll let you know if it was anything in the official Umlaut release, and try to reduce the debug I'm producing to sensible levels before contributing back any code.

Matthew