Re: For DSpace 8: update Handle server

[Mark H. Wood]

I forgot one thing. There is no official Handle resolver Maven
artifact at Central (or anywhere else that I know of). Long ago,
probably around DSpace 4.0, when we switched to Maven, someone packed
up the then-current version of the resolver and submitted
org.dspace:handle. If we continue in this manner, we'll have to
maintain that G:A with newer versions from time to time. If the
resolver becomes a prerequisite, we can wave "goodbye" to another hunk
of software that we redistribute but don't control.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu

[Mark H. Wood]

We're still shipping a v6.1 Handle resolver as part of DSpace. v9.1
is current. AFTER Dspace 7 we ought to look at updating. I don't
recall the current Handle documentation even mentioning a release as
old as the one we're using.

But I'd suggest that we also look at changing the resolver from a
dependency to a prerequisite, as we're doing with Solr. Then sites
can upgrade their resolvers without having to sync. with DSpace
upgrades, and DSpace doesn't have to rush out a release if there's a
security fix for the resolver that has to go out quickly. It's not
difficult to install a Handle resolver -- the one manual thing that
has to be done during configuration for DSpace is to add the custom
database backend which we would still supply.

Another thing to consider, if we would *require* a more recent
resolver release, is that we could get out of the business of
providing the backend. I think I've seen that recent resolvers will
optionally generate the Handle local-part and return it, during a
create request. We could drop the Handle table, instead using the
resolver in a more normal way and just recording the generated Handles
in object metadata as now. One arguable downside, IIRC, is that the
generated localpart is a UUID, so new DSpace Handles would be much
longer.

[Ian Little]

Hi everyone,

I work on the Handle codebase at CNRI and we wanted to let you know that
we're interested in being helpful in DSpace's future work using Handle.
The suggested approach of making Handle server installation a
prerequisite is reasonable, and we'd be pleased to work with you to make
it easy for DSpace users. We've also considered a special drop-in
version of Handle built especially for DSpace, if that direction seems
better to you.

One suggestion made was getting "out of the business of providing the
backend". There could be value in letting Handle maintain its own
storage (identifier metadata could potentially persist even if the
DSpace instance itself were retired or unavailable), but you'd need to
consider many issues which arise when ensuring that two separate data
stores are in sync.

That said, recent versions of the Handle server support a feature called
"template handles" which could probably replace the DSpace storage
backend and wouldn't maintain any storage on the Handle side. We often
use this feature in projects which require resolvable handles where all
actual data is maintained by a separate service.

Don't hesitate to include me in future conversations about Handle in
DSpace, and please feel welcome to send any questions you have my way. I
and my colleagues are also happy to drop by your Slack or IRC channels
sometime to chat.

Ian Little
Software Developer
Corporation for National Research Initiatives

[Tim Donohue]

Hi Ian,

Thanks for reaching out.  A few honest notes here... first, Mark's initial emails here specifically mention DSpace 8 because, unfortunately, a Handle Server update isn't currently on the schedule for DSpace 7 (due out later this year -- likely late summer / early fall).  We (the volunteer developers / Committers) know that DSpace needs to update the Handle Server in use, but (as of yet) we haven't had anyone find time to devote mindspace / effort towards making that happen.  Mark kindly shared his brainstorms (for the future) here. I'm glad he did, since it elicited a response from you.

So, this is a long way of saying both "Thanks" and admitting that we may not be able to respond much on this topic in the near future.  Nearly all our volunteers are heavily concentrated on DSpace 7 at this time.  Once that's out the door, I expect we'll have breathing room to look up, and starting planning for what comes next in DSpace 8.

In the meantime, honestly, if there *is* a way to continue to have a "drop-in version of Handle build especially for DSpace", that may be the *easiest* initial upgrade for us (as it likely will require the least amount of DSpace code refactoring to support).  Longer term, I fully expect we'll want to eventually extract this dependency more (as not all DSpace users use Handles, even though many do) -- but that's a bigger effort on our side.

So, if there's any way to build a "drop-in version" (especially something that could be distributed via Maven Central, as that's easy for us to pull dependencies from), maybe there's a slight chance of getting this updating prior to DSpace 8.  I cannot promise that, as it's highly dependent on finding interested volunteers (on my side), and also dependent on what a "drop-in version" might look like, and what sort of upgrade process is necessary for existing Handles.  But, it at least seems like the "lowest hanging fruit" here for moving this topic forward in the nearterm.

Not sure if this message is at all helpful.  But, if you had further ideas of how a "drop-in version" could be created for DSpace, I'd welcome you sharing them here (either in an email, or in a brief write-up elsewhere -- you could even use the DSpace Wiki).  I could then bring that up in a future DSpace developer meeting to see if there's interest/feedback, and get a sense of how to move it forward (and whether it's anything we can do now, or if we'll really need to wait until DSpace 7.0 is out the door).

Thanks again for getting in touch on this mailing list!

Tim

--
All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "DSpace Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-devel...@googlegroups.com.
To post to this group, send email to dspace...@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-devel.
For more options, visit https://groups.google.com/d/optout.

--

Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org

[Ian Little]

Thanks, Tim. I understand that you all are super busy with DSpace 7 now.
I just wanted to reach out and let you know that DSpace is on our mind
as we work on improving the Handle software. I'll talk more with my
colleagues and get back to you about what exactly we might be able to do
to help with the upgrade, whether it be in DSpace 7 or DSpace 8.

Ian

> <mailto:dspace-devel%2Bunsu...@googlegroups.com>.

> To post to this group, send email to dspace...@googlegroups.com

> <mailto:dspace...@googlegroups.com>.

[Ian Little]

Hi,

So, I spent some time exploring what it would look like to update Handle in DSpace 7. I pulled your master branch and added our internal Maven repository so that I could use the latest Handle release. It went pretty smoothly: to get a running Handle server, I had to change one line of Java, add some dependencies, update one dependency (besides Handle), and fiddle the make_handle_config script. Once I made those changes, I was able to follow your regular install instructions (mvn package, ant fresh_install, make_handle_config, start_handle_server) to get a DSpace with Handle 9.1. I believe remote-handle-resolver storage module would also work with a trivial code change.

I did not try updating an existing Handle server, but we've put working into making sure things are backwards compatible. I believe it should work, though obviously that would need some testing. Mostly today I just wanted to see if this drop-in replacement idea was even feasible, and I've convinced myself that it is (at least from our end).

So, we at CNRI are wondering: What can we do to help get Handle 9 into DSpace 7? If we can get the Handle jars on Maven Central by a certain date and submit a PR on GitHub with the needed changes, would that increase the chances? Anything else we can do to be helpful? I think we're hoping to get Handle up on Maven Central "someday," but we might be able to accelerate that if it would facilitate DSpace 7 having Handle 9 built in. If it's just too late for this release, though, we're fine targeting DSpace 8. I realized looking at your wiki today that you're trying to get a preview out at the end of the month, so I understand we might have missed the boat on DSpace 7.

Ian

On 12/3/19 2:47 PM, Ian Little wrote:
> Thanks, Tim. I understand that you all are super busy with DSpace 7 now.
> I just wanted to reach out and let you know that DSpace is on our mind
> as we work on improving the Handle software. I'll talk more with my
> colleagues and get back to you about what exactly we might be able to do
> to help with the upgrade, whether it be in DSpace 7 or DSpace 8.
>
> Ian
>
> On 8/3/19 12:55 PM, Tim Donohue wrote:

> >     <mailto:dspace-devel%2Bunsu...@googlegroups.com>.

> >     To post to this group, send email to dspace...@googlegroups.com

> >     <mailto:dspace...@googlegroups.com>.

[Tim Donohue]

Hi Ian,

Actually, you've come in at just the right time for DSpace 7.  You are correct that DSpace 7 is aiming for a "Preview Release" by the end of March / early April (more likely to be the latter at this point).  But, the "Preview" release is not feature-complete and is just an early preview of some of the new features coming in DSpace 7. So, we will continue to make improvements and add more features *until the 7.0 Beta* (which is due in late May / early June).

So, there still is time to help us get DSpace 7 updated to use the latest Handle server.  I think the best way to do so would be to have your team release the latest Handle JARs to Maven central and submit a PR to DSpace (latest `master`).  I'd recommend trying to achieve both of those by mid-to-end of April (earlier is always better).   There's some "wiggle room" in that deadline though, as we still may be able to fit it in after that date.  But, ideally, the earlier the PR can be created, the more likely I can locate developers (on my end) to review/test/approve it prior to Beta.  So, sometime in April is ideal.  If it slips to early May, it'd still be possible to get in. But mid-to-late May could end up being too late (to review in time).

A few things to keep in mind here:

• When submitting a PR, it's useful to provide some basic details on how to test it (for reviewers of the PR).  If it's as easy as "create a few new objects and make sure Handles get assigned" that's great. If there's more detailed tests we should run (e.g. ensure an existing handle prefix still works), that'd also be good to know.  (If we need to do the latter, I might be able to get someone to test it on our demo.dspace.org server, which has an existing handle prefix.)
• It'd also be useful to know if there's any (manual) upgrade steps that we'd need to document for any DSpace users upgrading to DSpace 7.  My assumption is the Handle server upgrade should hopefully be "invisible" to them. But, if they need to tweak the Handle configuration files, or run some other upgrade script, that'd be useful to document in the PR too.
• If there's any other things that you feel we should document to DSpace users (any other ways the latest Handle Server is different from what we are using), those could also be noted in the PR description.

If you have other questions, definitely let us know.  Thanks in advance for being willing to contribute directly to DSpace to make this happen! It'll be a great benefit to the entire DSpace user community!

Tim