Dataverse 6.1 deployment challenges

[Philip Durbin]

Dear Dataverse enthusiasts,

Since the release of Dataverse 6.1 a few problems were found in it that prompted the project to produce a patch addressing the issues, described below. We will be releasing this as an upcoming 6.1.1 release.

Most importantly there were a couple of bugs found by the community members who were first to deploy Dataverse 6.1 on their servers. Some were affecting the display of thumbnails.  Another was breaking multi-file downloads in the UI in datasets with non-standard licenses. In addition to these we’ve addressed some less visible bugs found by the development team since v6.1 was released, such as a problem with the new Globus functionality and a couple of problematic database queries.

You can consult the individual PRs listed further below for all the details.

If you would like to use the patched war file right now, you can download it from https://github.com/donsizemore/dataverse_backports/releases/tag/6.1_20240130 (a "dvinstall.zip" installer bundle is included as well). It has been tested by the installations that first reported the problems and here at Harvard Dataverse.

Second, 6.1 includes a SQL script related to the GuestbookResponse (the table storing the access/download history of your instance) that can take a long time to run. On Harvard Dataverse, it took over 23 minutes, but we don't think any installations have as many guestbook responses as we have (59 million rows). We think for most installations, it should be safe to simply deploy the war file, which contains the (Flyway) SQL script. At Harvard Dataverse, we used the Flyway command line tool to run the script *before* deploying the war file. The concern is the asadmin deploy command will time out after 10 minutes otherwise. It is important to note that the deployment, and the already started Flyway migration will then continue running in the background! So, if asadmin does time out, please wait a bit longer and see if the deployment eventually succeeds, or until you see any error messages in the server log before attempting to deploy again.

The specific PRs included in the patched war file provided:

- Two fixes for the thumbnail display logic: https://github.com/IQSS/dataverse/pull/10258 and https://github.com/IQSS/dataverse/pull/10269
- Multi-file downloads in the UI: https://github.com/IQSS/dataverse/pull/10264
- Globus: https://github.com/IQSS/dataverse/pull/10201
- Two fixes for SQL exceptions in GuestbookResponseService: https://github.com/IQSS/dataverse/pull/10233 and https://github.com/IQSS/dataverse/pull/10244

Please note that the bugs in question were introduced in the parts of the application that were heavily re-worked in 6.1. These optimization and cleanup changes were in fact long overdue and are an overall great improvement to the application.

As always, if you have any concerns, please feel free to reply on this thread, email us at sup...@dataverse.org, or chat with us at https://chat.dataverse.org

Thanks,

Phil
(on behalf of the Dataverse team)

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Asbjørn KU-IT]

Thanks for this. We were experiencing this GuestbookResponseService issue  https://github.com/IQSS/dataverse/pull/10233 for a day or two and a great thing to wake up and see this resolved today. we had tested the issue on the demo.dataverse.org installation and did not encounter the issue, so we where hesitant to reach out about the issue.

[meghan.good...@gmail.com]

Hello Phil and team,

Just curious if you have an update on when 6.1.1 will be released? We're just trying to time our next upgrade to include this release.

Thanks,

Meghan

[Philip Durbin]

Hi! We're tracking the 6.1.1 release at https://github.com/IQSS/dataverse/issues/10289 and it's in the current sprint but no one has picked it up yet. Lots of us are busy preparing for the community meeting in Mexico, I'm afraid! Meanwhile, the patched war file I mentioned is what we're using at Harvard Dataverse: https://github.com/donsizemore/dataverse_backports/releases/tag/6.1_20240130

Thanks for your patience!

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/b29d6745-78df-4907-8700-29c5cf0d1e3cn%40googlegroups.com.

[Philip Durbin]

Just to close the loop on this, we released 6.2 and decided against releasing 6.1.1.

All of the fixes mentioned above are in 6.2.

You can find the 6.2 release announcement here: https://groups.google.com/g/dataverse-community/c/HCWP2LOfcBc/m/8cFBZPbnAAAJ

Thanks,

Phil