dvn-vagrant, was Re: Dataverse or a DVN?

[Philip Durbin]

On Wed, Dec 19, 2012 at 6:58 PM, Condon, Kevin <kco...@hmdc.harvard.edu> wrote:
> If you want to try an experimental, as yet unsupported, DVN on VM
> configuration, try this: https://github.com/pdurbin/dvn-vagrant

I'm the one who came up with that experimental demo of DVN on Vagrant.

For now I mostly use it for quick tests, for logging in to a pristine
DVN installation as networkAdmin to poke around at something. I have
some other uses in mind for the future.

The nice thing about it is that the set up is fully automated. All I
have to do is type `vagrant up` and I get a working DVN at
http://localhost:8081/dvn

The bad thing about it is that it takes a while to run (about 30
minutes) but I have some ideas for speeding it up.

Today I ran `vagrant up` before lunch and had a working DVN by the
time I got back. :)

When I'd done with it I'll type `vagrant destroy` to get rid of it.

Anyway, I just thought I'd mention that I'm very interested on
feedback on dvn-vagrant if anyone would like to try it out. :)

Phil

--
Philip Durbin
Software Developer for http://thedata.org
http://www.iq.harvard.edu/people/philip-durbin

[Shubham Sharma]

Hi Philip,

Thanks for the post. I was trying to setup DVN on VM for my project but i am not able to install it successfully. I think there is some problem with the script as its not able to install glassfish properly in the VM. I have addd the snapshot of the problem, do take a look at it. Thanks in advance. :)

Shubh

[Philip Durbin]

Hmm...

It looks like the script to install glassfish failed.

Can you please try...

vagrant provision

... to re-run it?

If that doesn't work...

vagrant destroy

.... followed by...

vagrant up

... will make it try again.

Thanks,

Phil

[Manfredo Hopp]

The other bad thing about it is that the set up is fully unusable if you have a proxy behind!

Even if you manually load that box you get:

C:\Users\mhopp\dvn-vagrant>vagrant box add centos CentOS-6.3-x86_64-minimal.box
[vagrant] Downloading with Vagrant::Downloaders::File...
[vagrant] Copying box to temporary location...
[vagrant] Extracting box...
[vagrant] Cleaning up downloaded box...
C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib/archive/tar/
minitar.rb:493:in `read': undefined method `size' for nil:NilClass (NoMethodErro
r)
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:740:in `block (2 levels) in extract_entry'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:739:in `loop'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:739:in `block in extract_entry'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:738:in `open'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:738:in `extract_entry'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:973:in `block (2 levels) in unpack'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:679:in `block in each'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:614:in `block in each_entry'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:605:in `loop'
        from C:/vagrant/vagrant/embedded/gems/gems/archive-tar-minitar-0.5.2/lib
/archive/tar/minitar.rb:605:in `each_entry'
...........................................................etc.

Manfredo

[Philip Durbin]

Hi Shubham,

I searched around a bit but I still don't know why you saw an exit code of 102 from Puppet executing my little script that downloads the glassfish installer with wget and then runs it: https://github.com/pdurbin/dvn-vagrant/blob/master/modules/bucket/files/root/glassfish-install.sh

Very strange.

That's why I'm hoping you'll try it again. You can run `vagrant provision` over and over if it gets stuck anywhere. I'm using stages to ensure that configuration is done in the proper order: https://github.com/pdurbin/dvn-vagrant/blob/master/manifests/init.pp

To *really* try again, `vagrant destroy` will delete the VM to give you a clean slate. Then by running `vagrant up` again, a new VM will be created and configured (provisioned, to use the term from Puppet and Vagrant).

I think I mentioned before that I've never tried https://github.com/pdurbin/dvn-vagrant on Windows before. If anyone out there wants to give it a try, please let me know.

I *did* have a new member of our team try out dvn-vagrant yesterday and it worked fine for her. This was on Mac, though. And I've used it myself on Linux and Mac.

I hope this helps,

Phil

[Philip Durbin]

Hi Manfredo,

I've never tried dvn-vagrant from behind a proxy before... I'm not
sure what's going on here but I reported it in a couple places to the
Vagrant people:

https://github.com/mitchellh/vagrant/issues/498#issuecomment-12107950

http://irclogger.com/.vagrant/2013-01-10#1357844156

Thanks for letting me know about this!

Phil

[Manfredo Hopp]

Phil, thank you for your reply.
I should mention that my installation is on Windows7.

On the first place, I should say that when running "vagrant up" behind a Proxy I get timeout, which leads to think that one of the problems is vagrant (or wathever underling package) connection configuration.

On the other hand, downloding the box manually leads to second (and reported) problem of minitar's nil:nilClass.

So what I did next, is downloading the box again, since I could read form Mitchell's comments that this error is due to downloaded file beeing corrupted

and found a difference of 309MG-189MB beetwen both boxes! Therefore a cheksum as Mitchell says would be very appreciated!

After second download I got:

[vagrant] Cleaning up downloaded box...

The box file you're attempting to add is invalid. This can be
commonly attributed to typos in the path given to the box add
command. Another common case of this is invalid packaging of the
box itself.

So moved file to a dir with a simple path name and finally could add that box!

Now with vagrant up I get:

Guest Additions Version: 4.1.18
VirtualBox Version: 4.2.6
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 8080 => 8081 (adapter 1)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Running any VM customizations...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.

After wainting half an hour I give up!

Thanks Manfredo

2013/1/10 Philip Durbin <philip...@harvard.edu>

[Philip Durbin]

Hi Manfredo,

Right, it wasn't until after I hit "send" that I realized you were
were adding the box with `vagrant box add` on a local file, which is a
great test and entirely Vagrant territory (nothing to do with my
dvn-vagrant repo).

I feel like I've seen "Waiting for VM to boot" before... that it
happens occasionally with Vagrant...

Would you be willing to try again? Now that the base box is in Vagrant
that part should be all set. No more big download. To re-create the VM
you would type:

vagrant destroy
vagrant up

I'm sorry it didn't "just work" for you. :(

Phil

[Manfredo Hopp]

End of story:

[default] Failed to connect to VM!
Failed to connect to VM via SSH. Please verify the VM successfully booted
by looking at the VirtualBox GUI.

Looking at the GUI, the virtual box is up and running!

Manfredo

[default] Failed to connect to VM!
Failed to connect to VM via SSH. Please verify the VM successfully booted
by looking at the VirtualBox GUI.

2013/1/10 Philip Durbin <philip...@harvard.edu>

[Manfredo Hopp]

Phil: don´t worry about that straight installation, it's difficult to foresee every situation, but I think that with a little more effort things can run properly!

[Manfredo Hopp]

Phil: 

Looking at VM when booting I find:

 "kernel requires an X86-64 CPU but only detected an i686 CPU. Unable to boot - please use a kernel appropiate for your cpu"

Manfredo

[Philip Durbin]

On Thu, Jan 10, 2013 at 5:13 PM, Manfredo Hopp <hoppm...@gmail.com> wrote:
> Looking at VM when booting I find:
>
> "kernel requires an X86-64 CPU but only detected an i686 CPU. Unable to
> boot - please use a kernel appropiate for your cpu"

Uh oh! Hmm, and all the Vagrant base boxes at
http://www.vagrantbox.es for CentOS 6 are x86-64. :(

The CentOS project is working on official Vagrant base boxes:
http://lists.centos.org/pipermail/centos-virt/2012-October/003049.html

I assume they'll release a 32 bit one...

In other news, I rebooted my Linux box at home into Windows and it's
chugging along at its first `vagrant up` (screenshot attached).

[Manfredo Hopp]

Phil, I think that this experience was helpful to make a warning on documentation regarding potential user's hardware!

Thank you

Manfredo

[Philip Durbin]

Hi Manfredo,

On Thu, Jan 10, 2013 at 6:08 PM, Manfredo Hopp <hoppm...@gmail.com> wrote:
>
> Phil, I think that this experience was helpful to make a warning on
> documentation regarding potential user's hardware!

Absolutely it has been. :)

Good news! dvn-vagrant works on Windows. I now have a fresh install of
DVN 3.2 running on http://localhost:8081/dvn/ per the attached
screenshot.

Let me ask around about a 32 bit CentOS 6 base box and I'll update the
readme. Thanks for all the feedback, everyone!

[Manfredo Hopp]

The answer is http://yum.mnxsolutions.com/vagrant/centos_56_32.box

The bad news is, it is 800MG!

Manfredo

2013/1/10 Philip Durbin <philip...@harvard.edu>

Hi Manfredo,

[Manfredo Hopp]

And on this one I get:

[default] Waiting for VM to boot. This can take a few minutes.

[default] VM booted and ready for use!
[default] Configuring and enabling network interfaces...
[default] Mounting shared folders...
[default] -- v-root: /vagrant
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` v-root /vagrant

Not so easy!

Manfredo

2013/1/10 Philip Durbin <philip...@harvard.edu>

Hi Manfredo,

[Shubham Sharma]

Hi Phil,

I actually tried "vagrant provision" and "vagrant destroy" and then "vagrant up" a couple of times and it gives me back the same error --

########################################
ssha614@ARTS271086 ~/dvn-vagrant (master)
$ vagrant provision
[default] Running provisioner: Vagrant::Provisioners::Puppet...
[default] Running Puppet with /tmp/vagrant-puppet/manifests/init.pp...
←[1;35merr: /Stage[postgres]/Postgres/Exec[install_glassfish]/returns: change from notrun to 0 failed: /bin/bash /root/glassfish-i
nstall.sh returned 102 instead of one of [0] at /tmp/vagrant-puppet/modules-0/postgres/manifests/init.pp:21←[0m
←[0;36mnotice: /Stage[last]/Last/Service[iptables]: Dependency Exec[install_glassfish] has failures: true←[0m

←[0;33mwarning: /Stage[last]/Last/Service[iptables]: Skipping because of failed dependencies←[0m

←[0;36mnotice: /Stage[last]/Last/Exec[dvn-install]: Dependency Exec[install_glassfish] has failures: true←[0m

←[0;33mwarning: /Stage[last]/Last/Exec[dvn-install]: Skipping because of failed dependencies←[0m

←[0;36mnotice: Finished catalog run in 17.64 seconds←[0m
######################################

I am using windows 7 64 bit enterprise edition though and 'git bash' to run the terminal.

[Philip Durbin]

Hi Shubham,

I don't know if your "git bash" shell has an ssh client but the next thing I'd try is sshing to your VM and running the Glassfish installer manually. With putty, this works for me:

putty.exe vagrant@localhost -pw vagrant -P 2222

-pw is password, -P is the port

That *should* let you ssh right into the CentOS VM.

From here you can become root with this: sudo su -

Once you are root you could try to run the Glassfish installer manually:

/bin/bash /root/glassfish-install.sh

You can download putty from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

I hope this helps,

Phil

[Philip Durbin]

On Thu, Jan 10, 2013 at 7:07 PM, Manfredo Hopp <hoppm...@gmail.com> wrote:
> And on this one I get:
>
> [default] Waiting for VM to boot. This can take a few minutes.
> [default] VM booted and ready for use!
> [default] Configuring and enabling network interfaces...
> [default] Mounting shared folders...
> [default] -- v-root: /vagrant
> The following SSH command responded with a non-zero exit status.
> Vagrant assumes that this means the command failed!
>
> mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` v-root /vagrant

You're talking about the 32 bit CentOS image you found, right? This
one: http://yum.mnxsolutions.com/vagrant/centos_56_32.box

That must be CentOS 5. I've only tested dvn-vagrant on CentOS 6, I'm afraid.

[Philip Durbin]

On Mon, Jan 14, 2013 at 8:02 AM, Caesar Olima <caesar...@gmail.com> wrote:
> I was looking institutions that have implemented Dataverse

Please see "Dataverse Networks Around the World" at http://thedata.org
for a list of some installations at various institutions.

If anyone reading this is thinking, "Hey! I run a Dataverse Network
and I'm not on the list!" ... Let's talk! Please send an email to
dvn_s...@help.hmdc.harvard.edu and we'll see if we can add you. :)

Thanks,

[Philip Durbin]

On Mon, Jan 14, 2013 at 6:32 PM, Shubham Sharma
<letters...@gmail.com> wrote:
> Hi Caeser,
>
> Thanks for the mail and asking me regarding my project. I am doing my summer
> internship and my project is to migrate NZZSSDS from Nesstar to dataverse. I
> am in the middle of the migration and after the initial scoping, the
> research team finalized of going ahead with the installation of dataverse
> for NZSSDS. you can take a look here regarding details of my project-"
> http://www.eresearch.org.nz/content/data-migration-nesstar-dataverse-and-progress-so-far

Great post! I just wanted to mention that you shouldn't actually need
to get familiar with NetBeans. Also, for learning about creating
dataverses, you are welcome to create some on our demo site:
http://dvn-demo.iq.harvard.edu/dvn/

> Currently, i am trying to install 'DVN on vagrant" and i am stuck on it.

I sad dvn-vagrant didn't "just work" for you, but thanks for trying
it! I still need to update the readme with details Manfredo and I were
talking about.

> Later this week, i will try to install DVN Network on CentOS and will start
> uploading the data sets from there on. As i am new to linux and dataverse,
> it is taking time for me to get used to it.

Yes, installing on CentOS is a good idea. Hopefully the DVN
Installer's Guide will be helpful. If anything is unclear, please let
us know! http://guides.thedata.org/book/installers-guides

Linux does take some time to get used to but it's worth it. :)

[Shubham Sharma]

Hi Philip,

Thanks for the reply and taking time to read the whole thing. appreciate it :)

I tried to install glassfish manually as you mentioned in your previous post but still unsuccessful, i have attached the screenshot of the error.

###############
: command not foundh: line 3:
This program requires DISPLAY environment variable to be set.
Please re-run after assigning an appropriate value to DISPLAY.
###############

[Philip Durbin]

On Mon, Jan 14, 2013 at 9:48 PM, Shubham Sharma
<letters...@gmail.com> wrote:
> I tried to install glassfish manually as you mentioned in your previous post but still unsuccessful, i have attached the screenshot of the error.
>
> ###############
> : command not foundh: line 3:
> This program requires DISPLAY environment variable to be set.
> Please re-run after assigning an appropriate value to DISPLAY.
> ###############

Strange... I'm not sure why you're seeing this...

It seems like the glassfish-3.1.2.2-unix.sh installer downloaded ok...

The -s flag should mean "silent" ... fully automated because we
provide an "answer file" with the -a flag.

Can you please tell me if you get the same error when you run the last
part of the command manually:

sh glassfish-3.1.2.2-unix.sh -a /root/glassfish-answerfile -s

?

Also, while dvn-vagrant is intended to give you a fresh install of DVN
to play with in a sandbox, for a real install you should definitely
follow our DVN Installer's Guide at http://guides.thedata.org . And
you are welcome to open a ticket by emailing
dvn_s...@help.hmdc.harvard.edu .

Thanks,

[Shubham Sharma]

Hi Philip,

Sorry for the late reply, I finally managed to manually install glassfish when i ran that command you mentioned in your previous post. I am wondering what to do next?

[Philip Durbin]

Great!

In theory, another "vagrant provision" should finish the job.

It'll take a while, maybe 20 minutes.

[Shubham Sharma]

I did the same and got this error -

#################################

$ vagrant provision

[default] Running provisioner: Vagrant::Provisioners::Puppet...

[default] Running Puppet with /tmp/vagrant-puppet/manifests/init.pp...

←[1;35merr: /Stage[last]/Last/Service[iptables]/ensure: change from stopped to running failed: Could not start Service[iptables]:

Execution of '/sbin/service iptables start' returned 1:  at /tmp/vagrant-puppet/modules-0/last/manifests/init.pp:7←[0m

←[1;35merr: /Stage[last]/Last/Exec[dvn-install]/returns: change from notrun to 0 failed: /bin/bash /root/dvn-install.sh returned 1

 instead of one of [0] at /tmp/vagrant-puppet/modules-0/last/manifests/init.pp:13←[0m

←[0;36mnotice: Finished catalog run in 1099.91 seconds←[0m

#################################

then i tried "vagrant reload" to make sure the problem of possible forwarded ports but it returns the same error. It is very weird, i think i will go forward with the complete installation now and hopefully it should work. Thanks. 

[Philip Durbin]

I guess you could try to run it manually but at this point you are better off following our official installation guide at http://guides.thedata.org

I wonder how far the script got but it's hard to tell from here.

So close! Oh well. Thanks for trying! :)

[Shubham Sharma]

Thanks for the reply. Yep, will start with the official installation guide, will let you know if i get stuck somewhere. Thanks a lot though, you have been very very helpful and this whole group in general. I can't imagine troubleshooting these problems by myself! cheers! :)

[Shubham Sharma]

Hi Philip,

I was just installing dataverse through the official installation, all the dependencies were installed correctly.

When i ran the installer file, the initial installation went fine but i unfortunately i got stuck with this error -

################
-
-
-
-
 Checking your Glassfish installation...
Invalid Glassfish directory /usr/local/glassfish3!
Enter the root directory of your Glassfish installation:
(Or ctrl-C to exit the installer): /root/glassfish3
OK!

Setting the heap limit for Glassfish to 562MB.
You may need to adjust this setting to better suit
your system.


Press any key to continue...




Writing glassfish configuration file (domain.xml)... done.
Insecure dependency in system while running with -T switch at ./install line 720
###########################################

I don't understand the problem, i checked the installer code as well, everything seems fine, no idea why its giving me this error. maybe my glassfish directly location needs to be changed?

ps: I'm installing it on Centos 6.3 on a VM.

Thanks.

Shubham

On Thursday, 17 January 2013 15:47:41 UTC+13, Philip Durbin wrote:

[Philip Durbin]

Hi Shubham,

It was a holiday yesterday (Martin Luther King Day) but one of us will be able to take a look at this sometime soon.

Can you please tell us if you're trying to install version 3.2 or 3.3?

Thanks,

Phil

[Shubham Sharma]

Hi Phil,

Sorry for the late reply. I was trying to install the newer version 3.3 although I was finally able to install DVN application on my virtual machine successfully. Turns out my glassfish directory path was wrong and i un-installed it and installed it again in the right directory path and it worked. Right now i am trying out the optional components of DVN and then will finally web host the DVN application and link it to "http://www.nzssds.org.nz/" New Zealand Social Science Data Service (NZSSDS) i.e. the organization for which the DVN app is being developed and possibly will extend it for the whole University of Auckland and U.O.A library if this one goes fine. I'll let you know about the progress.

Moreover, i faced some bugs while trying to install the DVN application and that's why i was planning to develop an installation guide including those bugs which might help and be useful to new users like me to install it. I will post it here once i develop it as it might help to improve the official installation guide as well. :)

cheers!

Shubham

University of Auckland

[Philip Durbin]

Hi Shubham,

On Thu, Jan 24, 2013 at 8:23 PM, Shubham Sharma <letters...@gmail.com> wrote:

Sorry for the late reply. I was trying to install the newer version 3.3 although I was finally able to install DVN application on my virtual machine successfully. Turns out my glassfish directory path was wrong and i un-installed it and installed it again in the right directory path and it worked.

Great. I was wondering if the problem had something to do with your Glassfish installation. I'm glad you figured it out.

It's also great that you're using the latest and greatest release: 3.3
 

Right now i am trying out the optional components of DVN and then will finally web host the DVN application and link it to "http://www.nzssds.org.nz/" New Zealand Social Science Data Service (NZSSDS) i.e. the organization for which the DVN app is being developed and possibly will extend it for the whole University of Auckland and U.O.A library if this one goes fine. I'll let you know about the progress.

Perfect. You're ahead of me. I've only ever installed the base part of DVN. I should try to install Rserve, at least, and put it into my new https://github.com/dvn/dvn-install-demo

 

Moreover, i faced some bugs while trying to install the DVN application and that's why i was planning to develop an installation guide including those bugs which might help and be useful to new users like me to install it. I will post it here once i develop it as it might help to improve the official installation guide as well. :)

Great! Now you've got me wondering if we should start a wiki that users like you can contribute to...

Here, I just created https://github.com/IQSS/dvn/wiki ... if you sign up for GitHub you are welcome to start adding some notes there. We can always move the content into our official guides.

Phil

[Shubham Sharma]

Hi Phil,

Thanks a lot for the feedback though! and its great that you made the wiki page as well, i will definitely add my installation notes once i finish the complete installation and uploading of the data. :) Cheers!

Shubham

University of Auckland

[Shubham Sharma]

Hi Philp,

I am finally able to install the DVN network for the NZSSDS (New Zealand Social Science Data Service) Its up and running internally in University of Auckland network and will be accessible to everyone once i finish uploading the data.

I had a query though, it seems that port 6311 which is meant for Rserve was not open and its not running. I opened the 6311 port AFTER the installation however it still is not running. I was wondering how to make Rserve functional as well, do i have to run the installation script again or is there any other option? Thanks.

Shubham

University of Auckland

On Saturday, 26 January 2013 02:58:48 UTC+13, Philip Durbin wrote:

[Terry Rowland]

Shubham,

Phil can correct me if I'm wrong, but I believe all you'd need to to is check that the following JVM options are set in glassfish, and then restart the rserve daemon.  Unless you changed the rserve username/password during the install, it should simply be username:rserve, password:rserve.

-Dvdc.dsb.host=<RServe server hostname>
-Dvdc.dsb.rserve.user=<account>
-Dvdc.dsb.rserve.pwrd=<password>
-Dvdc.dsb.rserve.port=<port number>

Terry Rowland

The University of North Carolina at Chapel Hill

Odum Institute

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[esotiri]

Hi Shubham,

Here are the direction on how to setup Rserve.  http://www.rforge.net/Rserve/doc.html

I set this up on my local dvn installation following the above directions and after run the dvn install.

I hope this helps.

thanks

elda

[Shubham Sharma]

Hi Terry and Elda.

Thanks for the reply. :) Although i installed Rserve before i ran the D.V.N installation script, i forgot to open 6311 port and as it turns out, i just had to restart the reserve daemon after opening the port as Terry mentioned. Thanks for the help. Have a good day. :)

Regards,

Shubham