Dataverse or a DVN?

[Shubham Sharma]

Hi everyone,

I am doing a project for data migration for a social science research group which currently uses NESSTAR for there data and they want to migrate there data to Dataverse. The data is around 700 MB with around 50 files which are generally surveys and have files like SPSS, txt, pdf and SAS. I was trying to figure out whether to create a DVN i.e. a whole new dataverse network for it or just create a Dataverse in the IQSS website of Harvard DVN?

Moreover, if i create a DVN, how to do it? ( i saw the installer's guide but its a little confusing) and what are the requirements needed for it? and can i use a virtual server to host it?

Any help would be much appreciated, thanks in advance!

[Condon, Kevin]

Hi,

You should try out the software on the demo site first: http://dvn-demo.iq.harvard.edu . Create a dataverse, add some studies and files. If it's what you're looking for then you can create a dataverse for free on the production site: http://dvn.iq.harvard.edu . There is no charge for less than 1 TB of data.

If you don't want to bother with system administration and the associated costs, I think creating a dataverse is the way to go, especially with the amount and type of data you've described. You can restrict the data to specific users if desired. If you want complete control of the environment or expect to scale up a great deal, then go with an installation. What was it about the installer's guide you found confusing? We'll try to answer any questions and it would help us to improve our documentation.

[Shubham Sharma]

Hi Kevin,

First of all, thanks a lot for the prompt reply. My supervisor has decided to go with the DVN way as they want it get implemented for the whole university wide level (University of Auckland) and starting from the research group center to test its functionality. Moreover as per the requirements, is it necessary to have both redhat linux and Mac OS for the installation? or can i install it on windows as well? Also, the research center doesn't have any Mac OS so the most i can do is to install redhat linux and go forward with the installation. Moreover, instead of using a physical server to store the data, i was wondering if i can use a VMware/Virtual server to try it out and save costs initially. for that matter, which VMware will work best on DVN if there is any best option. After installing it, how exactly can i link my website to the DVN network which i just created? apologies for some noob questions, but any help is much appreciated. Thanks a lot again. Looking forward for your reply.

Shubham

[Condon, Kevin]

Shubham,

I think I understand some of the confusion: we need only one OS and our installation script supports both Redhat Linux and Mac OS. There are notes for both OS in the installation doc on where they might differ. So, Redhat alone would be perfectly OK and is what we use in production. CentOS is also a popular choice and should work with the installer. You can install DVN on Windows or any system that runs a JVM but our installation script wasn't written for that so you would need to configure it manually in that case.

We have had people install DVN on a VM but we do not have specific recommendations. You may have decreased performance, depending on how you size it.

If you want to try an experimental, as yet unsupported, DVN on VM configuration, try this: https://github.com/pdurbin/dvn-vagrant . It uses Virtual Box and CentOS 6 and a Vagrant script that installs in 1/2 hour. If you are more comfortable using VMWare and running our installation script there, then go ahead and do that.

Once installed, you can share your data with other DVNs and harvest study metadata from other DVNs using the OAI protocol. In this scenario, study metadata, eg. Cataloging information, is exported and made available to harvesting clients. When a client harvests the metadata, it appears as a local dataverse with a list of studies. Files remain on the original server but that is transparent to the user downloading the data. You can read more here:

http://guides.thedata.org/book/create-new-harvesting-dataverse , http://guides.thedata.org/book/manage-harvesting , http://guides.thedata.org/book/manage-oai-harvesting-sets .

Let me know if you have further questions.

Kevin

[gurudatt kumar]

Hi,

           I want to install Dataverse in my system, is red hat linux only enough or i have to manually istall any other softwares to work with it. please tell me the steps for that, waiting for ur response.

thanks in advance.

[Philip Durbin]

Hi! In addition to Linux you will need to install Java, Glassfish, and Postgresql. For details, please see http://thedata.harvard.edu/guides/dataverse-installer-main.html

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[gurudatt kumar]

hi! these softwares i have to install manually to work with dataverse? how to configure these softwares and also integrating R to Dataverse is needed for me. How? 

On Friday, 29 November 2013 20:57:33 UTC+5:30, Philip Durbin wrote:

Hi! In addition to Linux you will need to install Java, Glassfish, and Postgresql. For details, please see http://thedata.harvard.edu/guides/dataverse-installer-main.html

Phil

On Nov 28, 2013 11:35 PM, "gurudatt kumar" <gurudatt...@gmail.com> wrote:

Hi,

           I want to install Dataverse in my system, is red hat linux only enough or i have to manually istall any other softwares to work with it. please tell me the steps for that, waiting for ur response.

thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

[Philip Durbin]

If you'd like you can try our "install demo" which automates much of
the software installation within a virtual machine:
https://github.com/dvn/dvn-install-demo

It's not for production but to get an idea of how to get started. It
includes a "Installing R and Rserve" section.

Phil

>>> email to dataverse-commu...@googlegroups.com.

>>> For more options, visit https://groups.google.com/groups/opt_out.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Dataverse Users Community" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to dataverse-commu...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.

--
Philip Durbin
Software Developer for http://thedata.org
http://www.iq.harvard.edu/people/philip-durbin

[gurudatt kumar]

hi! i have fedora. DVN works in this or i want any other requirements for this also. i dont have redhat thats why. please inform me for this.

>>> email to dataverse-community+unsub...@googlegroups.com.

>>> For more options, visit https://groups.google.com/groups/opt_out.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Dataverse Users Community" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to dataverse-community+unsub...@googlegroups.com.

[gurudatt kumar]

hi! i have fedora. DVN works in this or i want any other requirements for this also. i dont have redhat thats why. please inform me for this.

thanks.. 

[Philip Durbin]

DVN should work on Fedora but I would recommend trying it on CentOS
first. Like Fedora, CentOS is free (gratis) but it's almost the same
as Red Hat Enterprise Linux (RHEL):
http://en.wikipedia.org/wiki/CentOS

The "install demo" I mentioned previously uses CentOS.

On Wed, Dec 18, 2013 at 12:47 AM, gurudatt kumar

>> >>> email to dataverse-commu...@googlegroups.com.

>> >>> For more options, visit https://groups.google.com/groups/opt_out.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Dataverse Users Community" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to dataverse-commu...@googlegroups.com.

>> > For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>> --
>> Philip Durbin
>> Software Developer for http://thedata.org
>> http://www.iq.harvard.edu/people/philip-durbin
>

> --
> You received this message because you are subscribed to the Google Groups
> "Dataverse Users Community" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to dataverse-commu...@googlegroups.com.

[gurudatt kumar]

hi! 

         i have installed redhat and when installing glassfish it is showing an error message like this..

./glassfish-3.1.2-unix.sh

Extracting the installer archive...

Could not locate a suitable Java runtime.

Please ensure that you have Java 6 or newer installed on your system

and accessible in your PATH or by setting JAVA_HOME

please send me the link for java runtime to download and run the glassfish in my system to work....

thanks 

[Philip Durbin]

Here it is: http://java.com/en/download/manual.jsp

Or you could use yum since you're on Red Hat.

Happy Holidays!

Phil

--

[gurudatt kumar]

hi!

 

  i installed glassfish,postgresql in my system. but when install dataverse it is showing like this....

Internet Address of your host: localhost.localdomain

Glassfish Directory: /usr/local/glassfish3

SMTP (mail) server to relay notification messages: localhost

Postgres Server: localhost

Postgres Server Port: 5432

Name of the Postgres Database: dvnDb

Name of the Postgres User: dvnApp

Postgres user password: secret

Rserve Server: localhost

Rserve Server Port: 6311

Rserve User Name: rserve

Rserve User Password: rserve

Is this correct? [y/n] y

Checking system user "postgres"... OK.

ERROR: I haven't been able to find the psql command in your PATH!

Please make sure PostgresQL is properly installed and try again.

[root@localhost dvninstall]# ./install

Welcome to the DVN installer.

You will be guided through the process of setting up a NEW

instance of the DVN application

This appears to be a RedHat system; good.

Please enter the following configuration values:

(hit [RETURN] to accept the default value)

Internet Address of your host: [localhost.localdomain] 

Glassfish Directory: [/usr/local/glassfish3] /root/glassfish3

SMTP (mail) server to relay notification messages: [localhost] 

Postgres Server: [localhost] 

Postgres Server Port: [5432] 5432

Name of the Postgres Database: [dvnDb] postgres

Name of the Postgres User: [dvnApp] postgres

Postgres user password: [secret] icrisat

Rserve Server: [localhost] 

Rserve Server Port: [6311] 

Rserve User Name: [rserve] 

Rserve User Password: [rserve] 

OK, please confirm what you've entered:

Internet Address of your host: localhost.localdomain

Glassfish Directory: /root/glassfish3

SMTP (mail) server to relay notification messages: localhost

Postgres Server: localhost

Postgres Server Port: 5432

Name of the Postgres Database: postgres

Name of the Postgres User: postgres

Postgres user password: icrisat

Rserve Server: localhost

Rserve Server Port: 6311

Rserve User Name: rserve

Rserve User Password: rserve

Is this correct? [y/n] y

Checking system user "postgres"... OK.

ERROR: I haven't been able to find the psql command in your PATH!

Please make sure PostgresQL is properly installed and try again.

[root@localhost dvninstall]# ./install

Welcome to the DVN installer.

You will be guided through the process of setting up a NEW

instance of the DVN application

This appears to be a RedHat system; good.

Please enter the following configuration values:

(hit [RETURN] to accept the default value)

Internet Address of your host: [localhost.localdomain] 

Glassfish Directory: [/usr/local/glassfish3] /root/glassfish3

SMTP (mail) server to relay notification messages: [localhost] 

Postgres Server: [localhost] 127.0.0.1

Postgres Server Port: [5432] 5432

Name of the Postgres Database: [dvnDb] postgres

Name of the Postgres User: [dvnApp] postgres

Postgres user password: [secret] icrisat

Rserve Server: [localhost] 

Rserve Server Port: [6311] 

Rserve User Name: [rserve] 

Rserve User Password: [rserve] 

OK, please confirm what you've entered:

Internet Address of your host: localhost.localdomain

Glassfish Directory: /root/glassfish3

SMTP (mail) server to relay notification messages: localhost

Postgres Server: 127.0.0.1

Postgres Server Port: 5432

Name of the Postgres Database: postgres

Name of the Postgres User: postgres

Postgres user password: icrisat

Rserve Server: localhost

Rserve Server Port: 6311

Rserve User Name: rserve

Rserve User Password: rserve

Is this correct? [y/n] Y

Please enter 'y' or 'n'!

(or ctrl-C to exit the installer)

y

In order to use a PostgresQL database running on a remote server,

Please run this installer on that host with the "--pg_only" option:

./install --pg_only

Press any key to continue the installation process once that has been

done. Or press ctrl-C to exit the installer.

.

What version of PostgresQL is installed on the remote server?

 [9.1] 8.4

Proceeding with the Glassfish setup.

Checking your Glassfish installation...

Invalid Glassfish directory /root/glassfish3!

Enter the root directory of your Glassfish installation:

(Or ctrl-C to exit the installer): /root/glassfish3

Invalid Glassfish directory /root/glassfish3!

Enter the root directory of your Glassfish installation:

(Or ctrl-C to exit the installer):

how to setup the connections please help me. waitng for ur reply

thanks ,

gurudatt

[Philip Durbin]

On Thu, Jan 9, 2014 at 12:33 AM, gurudatt kumar
<gurudatt...@gmail.com> wrote:
>
> ERROR: I haven't been able to find the psql command in your PATH!
> Please make sure PostgresQL is properly installed and try again.

Can you please try...

yum install postgresql-server

... and then try again?

Thanks,

Phil

p.s. Since you're on Red Hat.

p.p.s. Oh, speaking of Red Hat, I found this interesting (CentOS is
basically a free version of Red Hat Enterprise Linux or RHEL):

[CentOS-announce] CentOS Project joins forces with Red Hat -
http://lists.centos.org/pipermail/centos-announce/2014-January/020100.html

[gurudatt kumar]

Hi philip! 

when installing dvn everything is going right but problem with postgres database connection its showing this error..

In order to use a PostgresQL database running on a remote server,

Please run this installer on that host with the "--pg_only" option:

./install --pg_only..

what to do with this?

tell me please..

 thanks..

[gurudatt kumar]

what is the smtp mail messaging server address to give.

[Philip Durbin]

Hi! At this stage you probably don't need --pg_only...

It's for more advanced configurations, with postgresql on a separate server.

For now, please try just using postgresql on "localhost".

The same goes for the SMTP server... You can use localhost for now. I
use the equivalent of `yum install postfix` to install an SMTP server
on localhost in the install demo I put together:
https://github.com/dvn/dvn-install-demo/blob/master/modules/packages/manifests/init.pp

I hope this helps. It sounds like you're making progress.

Phil

On Fri, Jan 24, 2014 at 1:06 AM, gurudatt kumar

[gurudatt kumar]

HI! 

 its showing the same error to install --pg_only, where i have to use postgresql. database is created but how to connect is not getting when deployig the application. 

what is the network admin name and password i have to give here at last its showing this error.

Starting glassfish.

Waiting for domain1 to start ...................

Successfully started the domain : domain1

domain  Location: /usr/local/glassfish3/glassfish/domains/domain1

Log File: /usr/local/glassfish3/glassfish/domains/domain1/logs/server.log

Admin Port: 4848

Debugging is enabled.  The debugging port is: 9009

Command start-domain executed successfully.

Attempting to deploy the application:

Enter admin user name>  Gurudatt

Enter admin password for user "Gurudatt"> 

Authentication failed for user: Gurudatt

(Usually, this means invalid user name and/or password)

Command deploy failed.

Could not deploy DVN application!

(exit code: )..

thanks and regards,

gurudatt

[gurudatt kumar]

hi!

 given a token number and when clicking on link its going to login page.. how to login?(no authenticatoin details)

thanks,

gurudatt

[gurudatt kumar]

hi philip,

It is asking login credentials to go to that page. what should i do?

thanks,

gurudatt

[gurudatt kumar]

hi sir!

 postgres is also working now only thing showing error is..
Enter admin user name>  gurudatt123
Enter admin password for user "gurudatt123">
Authentication failed for user: gurudatt123

(Usually, this means invalid user name and/or password)
Command deploy failed.
Could not deploy DVN application!

thanks,

gurudatt

[Philip Durbin]

Hi Gurudatt,

I saw some internal emails between you and sup...@thedata.org

It seems like you now have a working DVN installation. Great!

Phil

On Mon, Jan 27, 2014 at 1:00 AM, gurudatt kumar

> --
> You received this message because you are subscribed to the Google Groups
> "Dataverse Users Community" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dataverse-commu...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[gurudatt kumar]

Hi philip,

Its working in my virtual machine but it cannot be accessed in windows machine(localhost). How to make accession in our intranet connection. Can you please help about this.

Gurudatt

[Philip Durbin]

Sorry, the Vagrant demo is only for practicing an installation. To make DVN accessible on your intranet, please perform a regular installation by following the Installers Guide: http://thedata.harvard.edu/guides/dataverse-installer-main.html

If you need any assistance, please feel free to open a ticket by emailing sup...@thedata.org

Phil

[Tanooj ch]

HI Philip,

i am tanooj and new to dataverse

Hope you are fond of below scenario(Highlighted in red font)...please guide me for successful installation

Starting glassfish.

Waiting for domain1 to start ..............

Successfully started the domain : domain1

domain  Location: /home/icrisat/glassfish3/glassfish/domains/domain1

Log File: /home/icrisat/glassfish3/glassfish/domains/domain1/logs/server.log

Admin Port: 4848

Debugging is enabled.  The debugging port is: 9009

Command start-domain executed successfully.

Attempting to deploy the application:

Enter admin user name>  root

Enter admin password for user "root"> 

Authentication failed for user: root

(Usually, this means invalid user name and/or password)

Command deploy failed.

Could not deploy DVN application!

(exit code: )

Thanks and Regards,

Tanooj ch.