Skip to first unread message
Danny Brooke
Nov 4, 2019, 10:19:59 AM11/4/19
to Dataverse Users Community
Hi everyone, we'll have our regular call tomorrow at Noon EST. Topics are:
- Upcoming 4.18 Release
- #Dataverse2020 Dates
- Moving off Glassfish 4 (https://github.com/IQSS/dataverse/issues/6230)
- Community Questions
For connection information and notes documents, please check out https://dataverse.org/community-calls
Philip Durbin
Nov 5, 2019, 4:31:21 PM11/5/19
to dataverse...@googlegroups.com
Great call today! Here are the notes from https://docs.google.com/document/d/1q8Dxmh3jZ5c5Xf2Vhfb8KNpQtISE0vaPQw_EMwnTMY4/edit?usp=sharing
2019-11-05 Dataverse Community Call
Agenda
* Upcoming 4.18 Release
* #Dataverse2020 Dates - June 17th, 18th, 19th
* Moving off Glassfish 4 ( https://github.com/IQSS/dataverse/issues/6230 )
* Community Questions
Attendees
* Danny Brooke (IQSS)
* Gustavo Durand (IQSS)
* Phil Durbin (IQSS)
* Oliver Bertuch (Forschungszentrum Jülich GmbH)
* Jim Myers (QDR, TDL)
* Sherry Lake (UVA)
* Kaitlin, Jayanthy, Bikram, Sohaib, Meghan, Alan, Amaz (Scholars Portal)
* Paul Boon (DANS)
* Slava (DANS/DataverseEU)
* Deirdre Kirmis (Arizona State University)
* Laura Waugh (TDR - Texas State University)
Notes
* (Danny) 4.18
* File previews: https://github.com/IQSS/dataverse/issues/3758
* (Phil) You can get a sneak peek at https://dev2.dataverse.org/file.xhtml?fileId=66&version=1.0
* Microsoft Login: https://github.com/IQSS/dataverse/issues/6155
* Other stuff
* Upgrade to PrimeFaces: https://github.com/IQSS/dataverse/issues/5975
* API token management: https://github.com/IQSS/dataverse/issues/6266
* Search API now supports unpublished content: https://github.com/IQSS/dataverse/issues/6301
* Questions
* (Jim) Restricted content for file previews?
* (Danny) Right now we don't have previews for restricted content but you can still click "Explore" to launch the full tool. We'd like to address this in the future.
* (Sherry, Danny) How easy is it to make a previewer?
* (Jim) It's both easy and hard. We can help you do it.
* (Sherry) We have three dimensional data. Our researchers would love to turn and twist the image in the browser.
* (Scholars Portal) Is the file kept in cache by the tool?
* (Jim) It's up to the tool. Tools could stream data, if they wanted to.
* (Slava) Previewers will be available in Dataverse on Kubernetes, currently we’re experimenting on Azure Cloud.
* (Paul) Previewing of large files.
* (Danny) Right now we were focused on the initial implementation.
* (Slava) Paul knows that I'm working with a virtual reality player (100 GB file) as a future part of the TimeMachine project.
* (Danny) #Dataverse2020 Save the Date
* June 17, 18, 19
* https://projects.iq.harvard.edu/dcm2020
* (Gustavo) Moving off Glassfish 4 ( https://github.com/IQSS/dataverse/issues/6230 )
* The main thing is being on something that is currently maintained
* We also want MicroProfile support such as the Config API.
* Stepping stone for upgrading to Java EE 8, Java 11.
* Looking at Glassfish and Payara because they are easier to migrate to.
* Payara is based on Glassfish itself.
* We've seen some issues. Oliver and others are aware of them.
* Trying to figure out if the problems are Payara-specific but they seem to happen on Glassfish too.
* (Jim) Any sense on how significant those issues are and time scale?
* (Gustavo) Not yet. I'm seeing some strangeness with jsessionid. Also null primary keys in objects. Oliver took a stab at fixing this for the "create dataverse" page.
* (Jim) Conflicts with old libraries in Glassfish when working a feature. Keep going. Thanks for working on this.
* Community Questions
* FYI (Jim) - had success in TDL work to upload files directly to S3 from DVUploader - details in the big data forum. More work TBD... ( https://groups.google.com/forum/#!topic/dataverse-big-data/nWf57CXrRyc )
* Also FYI (Jim) - the issue with S3 access I reported/asked about last meeting was part of the AWS DDOS problem. I noticed that the newest Dataverse versions ‘break’ when S3 isn’t available now that file sizes are displayed. Older versions just lose download/preview capabilities, but the lastest gets an error in displaying file pages.
* (Danny) Sure, please feel free to go ahead and create the issue. Thanks.
* (Jim, later) Done: https://github.com/IQSS/dataverse/issues/6343
* (Amaz) We've been looking at DCM and talking to Globus. We're planning on setting up a Globus endpoint. How do we upload 60,000 little files? How do we upload large data? Globus seems to work well for this.
* (Jim) Right now Dataverse points at one store. You can't do one store at a time. For the TDL use case, we'll probably send some data to TACC, not all. It sounds like you might be doing something similar. Maybe this is a feature we should put into Dataverse.
* (Amaz) We aren't talking about multiple stores. We're thinking that Globus would be the primary storage. Globus has an S3 connector. Globus would be the utility to perform the transfer.
* (Danny) Helpful from an IQSS standpoint would be something for the design team to look at so they can start thinking about the UI.
* (Jim) I'm hoping that my changes for TDL won't require any UI changes. Bypass Glassfish. TBD.
* (Amaz) The Globus Personal Connect tool is what we're looking at. This wouldn't have any effect on the Dataverse UI.
* (Slava) We are working on http://weblate.dataverse.org for DataverseEU and get all languages synchronized. We’re planning a webinar on Weblate somewhere in the middle of December, all community members are welcome.
* (Slava) We are moving 100,000 datasets from EASY to Dataverse. We're working on the migration process now and trying to figure out the best infrastructure setup, to move data to the separate University Dataverses or by discipline. It’s still in the discussion.
* (Phil) Thanks Sherry for hacking on the new map: https://github.com/IQSS/dataverse-installations
* (Phil) Thanks Courtney for helping with DataverseTV. There are 35 videos listed already and every is welcome to add more just by adding a row to a Google doc spreadsheet: https://github.com/IQSS/dataverse-tv
* (Danny) We are going to be setting up multiple community calls for different time zones. We are a worldwide community and it's helpful to have call times for when people are awake.
Agenda
* Upcoming 4.18 Release
* #Dataverse2020 Dates - June 17th, 18th, 19th
* Moving off Glassfish 4 ( https://github.com/IQSS/dataverse/issues/6230 )
* Community Questions
Attendees
* Danny Brooke (IQSS)
* Gustavo Durand (IQSS)
* Phil Durbin (IQSS)
* Oliver Bertuch (Forschungszentrum Jülich GmbH)
* Jim Myers (QDR, TDL)
* Sherry Lake (UVA)
* Kaitlin, Jayanthy, Bikram, Sohaib, Meghan, Alan, Amaz (Scholars Portal)
* Paul Boon (DANS)
* Slava (DANS/DataverseEU)
* Deirdre Kirmis (Arizona State University)
* Laura Waugh (TDR - Texas State University)
Notes
* (Danny) 4.18
* File previews: https://github.com/IQSS/dataverse/issues/3758
* (Phil) You can get a sneak peek at https://dev2.dataverse.org/file.xhtml?fileId=66&version=1.0
* Microsoft Login: https://github.com/IQSS/dataverse/issues/6155
* Other stuff
* Upgrade to PrimeFaces: https://github.com/IQSS/dataverse/issues/5975
* API token management: https://github.com/IQSS/dataverse/issues/6266
* Search API now supports unpublished content: https://github.com/IQSS/dataverse/issues/6301
* Questions
* (Jim) Restricted content for file previews?
* (Danny) Right now we don't have previews for restricted content but you can still click "Explore" to launch the full tool. We'd like to address this in the future.
* (Sherry, Danny) How easy is it to make a previewer?
* (Jim) It's both easy and hard. We can help you do it.
* (Sherry) We have three dimensional data. Our researchers would love to turn and twist the image in the browser.
* (Scholars Portal) Is the file kept in cache by the tool?
* (Jim) It's up to the tool. Tools could stream data, if they wanted to.
* (Slava) Previewers will be available in Dataverse on Kubernetes, currently we’re experimenting on Azure Cloud.
* (Paul) Previewing of large files.
* (Danny) Right now we were focused on the initial implementation.
* (Slava) Paul knows that I'm working with a virtual reality player (100 GB file) as a future part of the TimeMachine project.
* (Danny) #Dataverse2020 Save the Date
* June 17, 18, 19
* https://projects.iq.harvard.edu/dcm2020
* (Gustavo) Moving off Glassfish 4 ( https://github.com/IQSS/dataverse/issues/6230 )
* The main thing is being on something that is currently maintained
* We also want MicroProfile support such as the Config API.
* Stepping stone for upgrading to Java EE 8, Java 11.
* Looking at Glassfish and Payara because they are easier to migrate to.
* Payara is based on Glassfish itself.
* We've seen some issues. Oliver and others are aware of them.
* Trying to figure out if the problems are Payara-specific but they seem to happen on Glassfish too.
* (Jim) Any sense on how significant those issues are and time scale?
* (Gustavo) Not yet. I'm seeing some strangeness with jsessionid. Also null primary keys in objects. Oliver took a stab at fixing this for the "create dataverse" page.
* (Jim) Conflicts with old libraries in Glassfish when working a feature. Keep going. Thanks for working on this.
* Community Questions
* FYI (Jim) - had success in TDL work to upload files directly to S3 from DVUploader - details in the big data forum. More work TBD... ( https://groups.google.com/forum/#!topic/dataverse-big-data/nWf57CXrRyc )
* Also FYI (Jim) - the issue with S3 access I reported/asked about last meeting was part of the AWS DDOS problem. I noticed that the newest Dataverse versions ‘break’ when S3 isn’t available now that file sizes are displayed. Older versions just lose download/preview capabilities, but the lastest gets an error in displaying file pages.
* (Danny) Sure, please feel free to go ahead and create the issue. Thanks.
* (Jim, later) Done: https://github.com/IQSS/dataverse/issues/6343
* (Amaz) We've been looking at DCM and talking to Globus. We're planning on setting up a Globus endpoint. How do we upload 60,000 little files? How do we upload large data? Globus seems to work well for this.
* (Jim) Right now Dataverse points at one store. You can't do one store at a time. For the TDL use case, we'll probably send some data to TACC, not all. It sounds like you might be doing something similar. Maybe this is a feature we should put into Dataverse.
* (Amaz) We aren't talking about multiple stores. We're thinking that Globus would be the primary storage. Globus has an S3 connector. Globus would be the utility to perform the transfer.
* (Danny) Helpful from an IQSS standpoint would be something for the design team to look at so they can start thinking about the UI.
* (Jim) I'm hoping that my changes for TDL won't require any UI changes. Bypass Glassfish. TBD.
* (Amaz) The Globus Personal Connect tool is what we're looking at. This wouldn't have any effect on the Dataverse UI.
* (Slava) We are working on http://weblate.dataverse.org for DataverseEU and get all languages synchronized. We’re planning a webinar on Weblate somewhere in the middle of December, all community members are welcome.
* (Slava) We are moving 100,000 datasets from EASY to Dataverse. We're working on the migration process now and trying to figure out the best infrastructure setup, to move data to the separate University Dataverses or by discipline. It’s still in the discussion.
* (Phil) Thanks Sherry for hacking on the new map: https://github.com/IQSS/dataverse-installations
* (Phil) Thanks Courtney for helping with DataverseTV. There are 35 videos listed already and every is welcome to add more just by adding a row to a Google doc spreadsheet: https://github.com/IQSS/dataverse-tv
* (Danny) We are going to be setting up multiple community calls for different time zones. We are a worldwide community and it's helpful to have call times for when people are awake.
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/d6207423-65bd-49d7-b735-daaf62cfb601%40googlegroups.com.
--
Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
Reply all
Reply to author
Forward
0 new messages