Skip to first unread message
maureen...@worldveg.org
Oct 20, 2016, 1:38:12 AM10/20/16
to Dataverse Users Community
In searching for an appropriate data/research repository system for my research institution, I came across the Dataverse Project. Sounds like just the thing we need!
I have a few questions, though, so I hope you can bear with me – any advice anyone can provide would be welcome.
(Note: Sorry for intruding if this is not the correct place to post such a question...if you could point me in the right direction, that would be great.)
1. If we were to choose the Harvard Dataverse, our files would be on the Harvard servers.
a) I understand that Harvard makes backups. But could we make/get our own backups of what is in the Dataverse? Or would we have to do that before we send files to the Dataverse?
b) Is there a space limit on what any given institution or user can place in the Dataverse? How much can we put in there?!
c) What about files such as video, audio, PowerPoint, images? Can those go in the Dataverse?
2. If we installed Dataverse on our own server:
a) In your opinion, how skilled would an IT person need to be to install/manage a Dataverse?
b) How frequent are updates?
c) Once Dataverse is installed, is there some kind of a back end dashboard or other tool to use for uploading files/managing the system?
d) We use Wordpress for our website. Would we be able to integrate Dataverse into our site?
e) Do you know of any companies that do Dataverse installations?
Many thanks in advance for the information.
Maureen
Philip Durbin
Oct 20, 2016, 6:52:20 AM10/20/16
to dataverse...@googlegroups.com
Hi Maureen,
You've found exactly the right place to ask these questions. I'm just a developer so I don't have all the answers but I'm sure others from the community will jump in as needed to help.
First, you've laid out two options but "host with Harvard" should actually say something like "host with Harvard, UNC, or Scholars Portal" because all three institutions offer hosting:
- "The Harvard Dataverse is open to all researchers worldwide in all disciplines" according to http://dataverse.org/researchers
- UNC Dataverse is "open for all researchers worldwide from all disciplines to deposit data" according to the map at http://dataverse.org
- "The Scholars Portal Dataverse is a repository for research data collected by researchers and organizations primarily affiliated with Ontario universities. It is open to anyone in the world to deposit, share, and archive data." http://guides.scholarsportal.info/dataverse
And who knows! May additional Datavese installations will offer hosting to the world. There really should be a list. I just made a reminder for us to make one: https://trello.com/c/YSwtYMUi/30-make-a-list-of-dataverse-installations-that-offer-hosting-to-the-world
But let's say you host with Harvard. Yes, there are backups. Please see the "Harvard Dataverse Preservation Policy" at http://dataverse.org/best-practices/harvard-dataverse-preservation-policy . Sure, you can download all your files at any time but you'll want to hire a programmer to use the "Data Access API" at http://guides.dataverse.org/en/4.5.1/api/dataaccess.html . As for the metadata, it can be exported via the GUI as described at http://guides.dataverse.org/en/4.5.1/user/dataset-management.html#supported-metadata but you'll probably want your programmer to write a script to download metadata using what we affectionately call the "native" API: http://guides.dataverse.org/en/4.5.1/api/native-api.html . This is getting a bit in the weeds already, something I'm prone to, but the point is that Harvard does back up the data but you can also retrieve it yourself at any time.
The storage limit is 1 TB when you host with Harvard the last time I checked at https://groups.google.com/d/msg/dataverse-community/zn8fik2GgXI/jpjs4DFZBwAJ but this should be written down somewhere beyond the archives of this mailing list so I added a reminder for that too: https://trello.com/c/IIwuoMmb/31-document-1-tb-storage-size-limit-for-harvard-dataverse
Sure, when you host with Harvard or any Dataverse installation you can upload any file. Harvard has all the extra components installed to do special handling of a variety of formats. For details, please see http://guides.dataverse.org/en/4.5.1/user/dataset-management.html
Now on to your option 2: installing Dataverse yourself. Yes, you need skills. Your sysadmin should not be intimidated by upgrade instructions like https://github.com/IQSS/dataverse/releases/tag/v4.5 but should know that emailing this list and other community support options (IRC, private ticket to IQSS) are available: http://guides.dataverse.org/en/4.5.1/installation/intro.html#getting-help
I haven't crunched the numbers at https://github.com/IQSS/dataverse/releases but I feel like we release every month or two. There's a mini data science and visualization opportunity for someone here. :)
Generally speaking, the software tries to give the end user all the power to upload files and such. There *are* plans for an Administrative Dashboard in the works. http://dataverse.org/goals-roadmap-and-releases says, "The Dashboard will allow administrators to manage their Dataverse installation's settings, permissions, and integrations, and also perform other administrative tasks. This will include additional Dataverse metrics for users. Restoring administrative functionality will help complete Dataverse 4." That's the plan anyway. Some of this stuff such as managing permissions is already possible in the regular GUI using a "superuser" account. A lot of backend stuff has to be done at the command line right now. Your sysadmin should know what curl is. :)
No, I don't know of any companies that do Dataverse installations. There's another opportunity!
Thanks for writing in! I hope this helps!
Phil
You've found exactly the right place to ask these questions. I'm just a developer so I don't have all the answers but I'm sure others from the community will jump in as needed to help.
First, you've laid out two options but "host with Harvard" should actually say something like "host with Harvard, UNC, or Scholars Portal" because all three institutions offer hosting:
- "The Harvard Dataverse is open to all researchers worldwide in all disciplines" according to http://dataverse.org/researchers
- UNC Dataverse is "open for all researchers worldwide from all disciplines to deposit data" according to the map at http://dataverse.org
- "The Scholars Portal Dataverse is a repository for research data collected by researchers and organizations primarily affiliated with Ontario universities. It is open to anyone in the world to deposit, share, and archive data." http://guides.scholarsportal.info/dataverse
And who knows! May additional Datavese installations will offer hosting to the world. There really should be a list. I just made a reminder for us to make one: https://trello.com/c/YSwtYMUi/30-make-a-list-of-dataverse-installations-that-offer-hosting-to-the-world
But let's say you host with Harvard. Yes, there are backups. Please see the "Harvard Dataverse Preservation Policy" at http://dataverse.org/best-practices/harvard-dataverse-preservation-policy . Sure, you can download all your files at any time but you'll want to hire a programmer to use the "Data Access API" at http://guides.dataverse.org/en/4.5.1/api/dataaccess.html . As for the metadata, it can be exported via the GUI as described at http://guides.dataverse.org/en/4.5.1/user/dataset-management.html#supported-metadata but you'll probably want your programmer to write a script to download metadata using what we affectionately call the "native" API: http://guides.dataverse.org/en/4.5.1/api/native-api.html . This is getting a bit in the weeds already, something I'm prone to, but the point is that Harvard does back up the data but you can also retrieve it yourself at any time.
The storage limit is 1 TB when you host with Harvard the last time I checked at https://groups.google.com/d/msg/dataverse-community/zn8fik2GgXI/jpjs4DFZBwAJ but this should be written down somewhere beyond the archives of this mailing list so I added a reminder for that too: https://trello.com/c/IIwuoMmb/31-document-1-tb-storage-size-limit-for-harvard-dataverse
Sure, when you host with Harvard or any Dataverse installation you can upload any file. Harvard has all the extra components installed to do special handling of a variety of formats. For details, please see http://guides.dataverse.org/en/4.5.1/user/dataset-management.html
Now on to your option 2: installing Dataverse yourself. Yes, you need skills. Your sysadmin should not be intimidated by upgrade instructions like https://github.com/IQSS/dataverse/releases/tag/v4.5 but should know that emailing this list and other community support options (IRC, private ticket to IQSS) are available: http://guides.dataverse.org/en/4.5.1/installation/intro.html#getting-help
I haven't crunched the numbers at https://github.com/IQSS/dataverse/releases but I feel like we release every month or two. There's a mini data science and visualization opportunity for someone here. :)
Generally speaking, the software tries to give the end user all the power to upload files and such. There *are* plans for an Administrative Dashboard in the works. http://dataverse.org/goals-roadmap-and-releases says, "The Dashboard will allow administrators to manage their Dataverse installation's settings, permissions, and integrations, and also perform other administrative tasks. This will include additional Dataverse metrics for users. Restoring administrative functionality will help complete Dataverse 4." That's the plan anyway. Some of this stuff such as managing permissions is already possible in the regular GUI using a "superuser" account. A lot of backend stuff has to be done at the command line right now. Your sysadmin should know what curl is. :)
No, I don't know of any companies that do Dataverse installations. There's another opportunity!
Thanks for writing in! I hope this helps!
Phil
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8ea3408b-fdcd-462c-ba65-f84e40ccbf41%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin
julian...@g.harvard.edu
Oct 20, 2016, 10:47:34 AM10/20/16
to Dataverse Users Community
Hi Maureen,
2. If we installed Dataverse on our own server:
...
d) We use Wordpress for our website. Would we be able to integrate Dataverse into our site?
One solution for this could be using the Dataverse listing widget to display your dataverse on your website. This section of the user guide has more details: http://guides.dataverse.org/en/latest/user/dataverse-management.html
Best,
Julian
Julian Gautier
Product Research Specialist, IQSS
maureen...@worldveg.org
Oct 20, 2016, 7:57:43 PM10/20/16
to Dataverse Users Community, philip...@harvard.edu
Thanks a million for your advice, Phil!
This information will help me decide how to go forward with Dataverse.
All the best,
Maureen
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/8ea3408b-fdcd-462c-ba65-f84e40ccbf41%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
maureen...@worldveg.org
Oct 20, 2016, 7:58:09 PM10/20/16
to Dataverse Users Community
Thanks for the tip, Julian!
danny...@g.harvard.edu
Oct 21, 2016, 11:55:43 AM10/21/16
to Dataverse Users Community, philip...@harvard.edu
Hey Maureen - welcome and thanks for considering Dataverse! Big thanks to Phil and Julian for the helpful responses here.
Sherry Lake
Oct 22, 2016, 7:59:03 AM10/22/16
to Dataverse Users Community
Hi Maureen,
I can answer some of the questions in your 2nd part "installing dataverse on our own server" - See my responses inline below:
In searching for an appropriate data/research repository system for my research institution, I came across the Dataverse Project. Sounds like just the thing we need!
2. If we installed Dataverse on our own server:
a) In your opinion, how skilled would an IT person need to be to install/manage a Dataverse?
At UVa Library we have two "groups", system administrators that installed the server and set up all the network/storage backends and another repository "group" (me) that manages the software on the servers (once installed). Not sure about what skills are needed for an IT person to install, but here at UVa our system admins (who administer our other Library services for our catalog, web page, subscription databases, etc.), installed dataverse, set up the network (including shibboleth) on our server. I, the Library Repository manager, customized and manage Dataverse (as much as I could) and is in charge of supporting the user-side.
b) How frequent are updates?
Looks like updates come out every six months or so. We wait about two weeks after the update is put on Harvard's site before we update our local dataverse. Instructions (I've been told) for updating have been easy for our system admins to complete in about two hours. We upgrade our test server first. I test the upgrade on test and then we upgrade our production server (http://dataverse.lib.virginia.edu).
At the University of Virginia, http://dataverse.lib.virginia.edu, we only allow UVa affiliated researchers to upload and deposit. We have turned off local account creation and only UVa authenticated users (from UVa Shibboleth) can log on and create accounts. Anyone can go to the site and search/discover and download.
Our installation for Dataverse, is pretty much out of the box. We do not have the developers to go into the code and modify things, so it pretty much works like Harvard's (but with local servers and local storage/backups).
Maureen, if you have questions about installing and managing locally, I'll be glad to help.
--
Sherry
Sherry Lake | Scholarly Repository Librarian | University of Virginia Library | shL...@virginia.edu | 434.924.6730 | @shLakeUVA | Alderman Library, 160 N. McCormick Road, Charlottesville, VA 22903 | Alderman 563 | LinkedIn Profile | “Keeper of the Dataverse"
Mecozzi, Maureen
Oct 23, 2016, 8:01:44 PM10/23/16
to dataverse...@googlegroups.com
Thanks for sharing UVa's experience with Dataverse, Sherry. Learning about the ways different organizations use the platform helps a great deal!
Best,
Maureen
--
You received this message because you are subscribed to a topic in the Google Groups "Dataverse Users Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dataverse-community/HqnMk1RzNzo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/f154ceb8-ac89-456c-a12d-fc3f64ca4d72%40googlegroups.com.
Maureen Mecozzi
Head, Communications and Information
World Vegetable Center
P.O. Box 42, Shanhua, Tainan 74199 Taiwan
T +886-6-583-7801
E maureen...@worldveg.org
Head, Communications and Information
World Vegetable Center
P.O. Box 42, Shanhua, Tainan 74199 Taiwan
T +886-6-583-7801
E maureen...@worldveg.org
Skype: maureen.mecozzi1
YouTube: http://www.youtube.com/WorldVegetableCenterAlleviating poverty and malnutrition in the developing world through increased production and consumption of nutritious, health-promoting vegetables
Reply all
Reply to author
Forward
0 new messages