Dataverse 4.1 Stata file ingest failure

169 views
Skip to first unread message

Leonhard Maylein

unread,
Oct 1, 2015, 5:29:17 AM10/1/15
to dataverse...@googlegroups.com
Hi,

we've installed Dataverse 4.1 on a test system.
As we plan to use shibboleth, the glassfish is placed behind an
apache webserver.
We've also installed R, rApache and TwoRavens as described
at
http://guides.dataverse.org/en/latest/installation/r-rapache-tworavens.html
(we use Ubuntu 14.04, therefore we had to
adapt some of the steps).

TwoRavens configuration:

********************************************
Directory where TwoRavens is installed: /var/www/html/dataexplore
Apache config directory: /etc/apache2
Apache Web Root directory: /var/www/html
Internet address of the rApache host: xxx.ub.uni-heidelberg.de
rApache port number: 443
http or https?: https
URL address of the Dataverdse application, to access files and metadata:
https://xxx.ub.uni-heidelberg.de/
********************************************

The two ravens url is registered in DVN via
curl -X PUT -d https://xxx.ub.uni-heidelberg.de/dataexplore/gui.html
http://localhost:8080/api/admin/settings/:TwoRavensUrl


When trying to add a stata file (which has been successfully
publicated via DVN 3.6.2) we get an error message:


********************************************
18|{"data":"failure"}18|{"data":"failure"}<?xml version='1.0'
encoding='UTF-8' ?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en"><head>
<title>Test - Test Dataverse Maylein Dataverse</title>
...
********************************************

After reload there is a warning sign next to the download button
which says:
"Ingest produced tabular data, but failed to save it in the database; 1
No further information is available.


In the files directory there are three new files:

-rw-r--r-- 1 root root 75969 Okt 1 10:45 150229322da-40bfd2a7e399
-rw-r--r-- 1 root root 76953 Okt 1 10:45 150229322da-40bfd2a7e399.90d
-rw-r--r-- 1 root root 70193 Okt 1 10:44 150229322da-40bfd2a7e399.orig


The logfile shows:

********************************************
[2015-10-01T10:45:03.699+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataver
se.ingest.tabulardata.impl.plugins.dta] [tid: _ThreadID=62
_ThreadName=p: thread
-pool-1; w: 5] [timeMillis: 1443689103699] [levelValue: 800] [[
***** DTAFileReader: read() end *****]]

[2015-10-01T10:45:03.705+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataver
se.ingest.IngestServiceBean] [tid: _ThreadID=62 _ThreadName=p:
thread-pool-1; w:
5] [timeMillis: 1443689103705] [levelValue: 800] [[
Tabular data successfully ingested; DataTable with 123 variables
produced.]]

[2015-10-01T10:45:03.710+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=62
_ThreadName=p: thread-pool-1; w: 5] [timeMillis: 1443689103710]
[levelValue: 800] [[
Tab-delimited file produced: /tmp/tempTabfile.4282625164438807627.tab]]

[2015-10-01T10:45:04.712+0200] [glassfish 4.1] [SEVERE] []
[org.dataverse.unf.RoundRoutines] [tid: _ThreadID=62 _ThreadName=p:
thread-pool-1; w: 5] [timeMillis: 1443689104712] [levelValue: 1000] [[
RoundRoutines:decimal separator no in right place]]

[2015-10-01T10:45:04.728+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=62
_ThreadName=p: thread-pool-1; w: 5] [timeMillis: 1443689104728]
[levelValue: 800] [[
Ingest failure: Failed to save tabular data (datatable,
datavariables, etc.) in the database. Clearing the datafile object.]]

[2015-10-01T10:45:04.789+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=62
_ThreadName=p: thread-pool-1; w: 5] [timeMillis: 1443689104789]
[levelValue: 800] [[
Unknown excepton saving ingested file; Sent push notification to the
page.]]

[2015-10-01T10:45:04.814+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestMessageBean] [tid: _ThreadID=62
_ThreadName=p: thread-pool-1; w: 5] [timeMillis: 1443689104814]
[levelValue: 800] [[
Error occurred during ingest job!]]
********************************************

Same problem when we try to add an Excel file from
the Harvard DVN:


********************************************
[2015-10-01T11:03:01.311+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest] [tid: _ThreadID=49
_ThreadName=jk-connector(2)] [timeMillis: 1443690181311] [levelValue:
800] [[
buffer_size: 500]]

[2015-10-01T11:03:01.424+0200] [glassfish 4.1] [SEVERE] [] [] [tid:
_ThreadID=49 _ThreadName=Thread-9] [timeMillis: 1443690181424]
[levelValue: 1000] [[
[Error] jhove.conf:2:14: cvc-elt.1: Deklaration des Elements
"jhoveConfig" kann nicht gefunden werden.]]

[2015-10-01T11:03:01.429+0200] [glassfish 4.1] [SEVERE] []
[edu.harvard.hul.ois.jhove] [tid: _ThreadID=49
_ThreadName=jk-connector(2)] [timeMillis: 1443690181429] [levelValue:
1000] [[
Testing SEVERE level]]

[2015-10-01T11:03:01.935+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.util.FileUtil] [tid: _ThreadID=49
_ThreadName=jk-connector(2)] [timeMillis: 1443690181935] [levelValue:
800] [[
Type by extension, for Detection of anaemia using conjunctival
images.xlsx:
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet]]

[2015-10-01T11:03:04.579+0200] [glassfish 4.1] [INFO] [] [] [tid:
_ThreadID=50 _ThreadName=Thread-8] [timeMillis: 1443690184579]
[levelValue: 800] [[
ADDING FILE: Detection of anaemia using conjunctival images.xlsx; for
dataset: doi:10.5072/FK2/PBY9MP]]

[2015-10-01T11:03:05.984+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=50
_ThreadName=jk-connector(3)] [timeMillis: 1443690185984] [levelValue:
800] [[
Attempting to ingest 1 tabular data file(s).]]

[2015-10-01T11:03:06.081+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestMessageBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690186081]
[levelValue: 800] [[
Start ingest job;]]

[2015-10-01T11:03:06.162+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.tabulardata.impl.plugins.xlsx] [tid:
_ThreadID=61 _ThreadName=p: thread-pool-1; w: 4] [timeMillis:
1443690186162] [levelValue: 800] [[
entering processSheet]]

[2015-10-01T11:03:08.515+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Servlet] [tid: _ThreadID=50 _ThreadName=jk-connector(3)]
[timeMillis: 1443690188515] [levelValue: 900] [[
WELD-000714: HttpContextLifecycle guard leak detected. The Servlet
container is not fully compliant. The value was 1]]

[2015-10-01T11:03:08.516+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Context] [tid: _ThreadID=50 _ThreadName=jk-connector(3)]
[timeMillis: 1443690188516] [levelValue: 900] [[
WELD-000225: Bean store leak was detected during
org.jboss.weld.context.http.HttpRequestContextImpl association:
com.sun.enterprise.web.pwc.connector.coyote.PwcCoyoteRequest@51caf55]]

[2015-10-01T11:03:08.518+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Context] [tid: _ThreadID=50 _ThreadName=jk-connector(3)]
[timeMillis: 1443690188518] [levelValue: 900] [[
WELD-000225: Bean store leak was detected during
org.jboss.weld.context.http.HttpSessionContextImpl association:
com.sun.enterprise.web.pwc.connector.coyote.PwcCoyoteRequest@51caf55]]

[2015-10-01T11:03:08.521+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Servlet] [tid: _ThreadID=50 _ThreadName=jk-connector(3)]
[timeMillis: 1443690188521] [levelValue: 900] [[
WELD-000715: HttpContextLifecycle guard not set. The Servlet
container is not fully compliant.]]

[2015-10-01T11:03:08.533+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Servlet] [tid: _ThreadID=51 _ThreadName=jk-connector(4)]
[timeMillis: 1443690188533] [levelValue: 900] [[
WELD-000714: HttpContextLifecycle guard leak detected. The Servlet
container is not fully compliant. The value was 1]]

[2015-10-01T11:03:08.534+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Context] [tid: _ThreadID=51 _ThreadName=jk-connector(4)]
[timeMillis: 1443690188534] [levelValue: 900] [[
WELD-000225: Bean store leak was detected during
org.jboss.weld.context.http.HttpRequestContextImpl association:
com.sun.enterprise.web.pwc.connector.coyote.PwcCoyoteRequest@245ba6c3]]

[2015-10-01T11:03:08.535+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Context] [tid: _ThreadID=51 _ThreadName=jk-connector(4)]
[timeMillis: 1443690188535] [levelValue: 900] [[
WELD-000225: Bean store leak was detected during
org.jboss.weld.context.http.HttpSessionContextImpl association:
com.sun.enterprise.web.pwc.connector.coyote.PwcCoyoteRequest@245ba6c3]]

[2015-10-01T11:03:08.547+0200] [glassfish 4.1] [WARN] []
[org.jboss.weld.Servlet] [tid: _ThreadID=51 _ThreadName=jk-connector(4)]
[timeMillis: 1443690188547] [levelValue: 900] [[
WELD-000715: HttpContextLifecycle guard not set. The Servlet
container is not fully compliant.]]

[2015-10-01T11:03:09.211+0200] [glassfish 4.1] [WARNING] []
[edu.harvard.iq.dataverse.ingest.tabulardata.impl.plugins.xlsx] [tid:
_ThreadID=61 _ThreadName=p: thread-pool-1; w: 4] [timeMillis:
1443690189211] [levelValue: 900] [[
Null r attribute in the first row element!]]

[2015-10-01T11:03:09.212+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.tabulardata.impl.plugins.xlsx] [tid:
_ThreadID=61 _ThreadName=p: thread-pool-1; w: 4] [timeMillis:
1443690189212] [levelValue: 800] [[
Established variable (column) count: 5]]

[2015-10-01T11:03:09.389+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690189389]
[levelValue: 800] [[
Tabular data successfully ingested; DataTable with 5 variables
produced.]]

[2015-10-01T11:03:09.390+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690189390]
[levelValue: 800] [[
Tab-delimited file produced: /tmp/data-5090698695537394135.tab]]

[2015-10-01T11:03:09.416+0200] [glassfish 4.1] [SEVERE] []
[org.dataverse.unf.RoundRoutines] [tid: _ThreadID=61 _ThreadName=p:
thread-pool-1; w: 4] [timeMillis: 1443690189416] [levelValue: 1000] [[
RoundRoutines:decimal separator no in right place]]

[2015-10-01T11:03:09.418+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690189418]
[levelValue: 800] [[
Ingest failure: Failed to save tabular data (datatable,
datavariables, etc.) in the database. Clearing the datafile object.]]

[2015-10-01T11:03:09.430+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690189430]
[levelValue: 800] [[
Unknown excepton saving ingested file; Sent push notification to the
page.]]

[2015-10-01T11:03:09.437+0200] [glassfish 4.1] [INFO] []
[edu.harvard.iq.dataverse.ingest.IngestMessageBean] [tid: _ThreadID=61
_ThreadName=p: thread-pool-1; w: 4] [timeMillis: 1443690189437]
[levelValue: 800] [[
Error occurred during ingest job!]]
********************************************


Could you please give us a hint on how we could solve
this problem?


BTW:

The german message

[Error] jhove.conf:2:14: cvc-elt.1: Deklaration des Elements
"jhoveConfig" kann nicht gefunden werden.]]

means

'declaration of the element "jhoveConfig" could not be found'

Regarding the message

"RoundRoutines:decimal separator no in right place"

In German the decimal separator is a comma. Could this
cause the problem. Do we have to change a glassfish configuration
parameter?

Leonhard Maylein

University library of Heidelberg


Philip Durbin

unread,
Oct 1, 2015, 9:34:34 AM10/1/15
to dataverse...@googlegroups.com
Hi Leonhard,

Thank you for all the detail!

As a quick sanity check, could you please try uploading a very simple "50by1000.dta" Stata file which you can find at https://github.com/IQSS/dataverse/blob/v4.1/scripts/api/data-deposit/data/example.zip ? It should be "ingested" as a tabular file.

It sounds like the Excel xlsx file you're testing with is "Detection of anaemia using conjunctival images.xlsx" from http://dx.doi.org/10.7910/DVN/L4MDKC . It appears to have ingested properly there ( https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/L4MDKC ) at least.

It sounds like the Stata file you're having trouble with is somewhere at https://heidata.uni-heidelberg.de/dvn/ but can you please let us know where we can find it?

The RoundRoutines errors are coming from https://github.com/IQSS/UNF/blob/9b51c107743fbd92086184186eb1eed4a33abd4b/src/main/java/org/dataverse/unf/RoundRoutines.java#L394 but like you I don't know if this is really causing the problem. The ingest code attempts to calculate the UNF but I'm not sure what the behavior is supposed to be if there's a problem calculating it ( https://github.com/IQSS/dataverse/blob/v4.2/src/main/java/edu/harvard/iq/dataverse/ingest/IngestServiceBean.java#L2096 ). This may or may not be your actual problem.

Rather than Ubuntu we run RHEL/CentOS at https://dataverse.harvard.edu so our ability to help will be limited. However, Dataverse community members such as Lucien van Wouw have apparently figured out how to get everything including TwoRavens, rApache and Rserve working on Ubuntu and have even captured the config in Puppet at https://github.com/IQSS/dataverse-puppet . So one thing you could try is cloning that repo and running `vagrant up` and (assuming you get a working Dataverse installation) you could upload your Stata file there and see if you can reproduce the problem.

Since you mentioned Shibboleth, I should add that I'm not sure that Shibboleth is well supported on Ubuntu. Or rather, https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLinuxInstall says it's only "officially supported" on RHEL, CentOS, SUSE, and OpenSUSE, which are all RPM-based distributions. Certainly http://guides.dataverse.org/en/4.2/installation/shibboleth.html is oriented toward RHEL/CentOS. I don't mean to discourage you from trying however! I'd love to have more people trying out the Shibboleth feature of Dataverse.

Again, if you could please tell us where we can download the Stata file, we'd really appreciate it. (Sorry if I missed this.) You could try it yourself at https://dataverse-demo.iq.harvard.edu and if it doesn't ingest you're welcome to open an issue at https://github.com/IQSS/dataverse/issues .

I hope this helps!

Phil




--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/560CFCEC.3040901%40ub.uni-heidelberg.de.
For more options, visit https://groups.google.com/d/optout.



--

Lucien van Wouw

unread,
Oct 1, 2015, 10:13:04 AM10/1/15
to Dataverse Users Community, philip...@harvard.edu
Hi,

> [Error] jhove.conf:2:14: cvc-elt.1: Deklaration des Elements "jhoveConfig" kann nicht gefunden werden.]] 

I also got this error on Ubuntu 12.04 lts with a simple csv on a clean installation.... Yet it occurred quite out of the blue... that is, previous installations on both 12 and 14 had no issues with it. I intended to do some remote debugging at JhoveFileType to see what happens.

To work around it, an asadmin undeploy [your package] followed by a asadmin deploy [your war] made the conversion routine work for me thereafter.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/560CFCEC.3040901%40ub.uni-heidelberg.de.
For more options, visit https://groups.google.com/d/optout.

Leonhard Maylein

unread,
Oct 1, 2015, 11:19:01 AM10/1/15
to dataverse...@googlegroups.com
Hi Philip,

Am 01.10.2015 um 15:34 schrieb Philip Durbin:
> Hi Leonhard,
>
> Thank you for all the detail!
>
> As a quick sanity check, could you please try uploading a very simple
> "50by1000.dta" Stata file which you can find at
> https://github.com/IQSS/dataverse/blob/v4.1/scripts/api/data-deposit/data/example.zip
> ? It should be "ingested" as a tabular file.


This file was ingested without any failure.
The Explore button works.



>
> It sounds like the Excel xlsx file you're testing with is "Detection of
> anaemia using conjunctival images.xlsx" from
> http://dx.doi.org/10.7910/DVN/L4MDKC . It appears to have ingested
> properly there (
> https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/L4MDKC
> ) at least.


Yes, that's correct :-)


>
> It sounds like the Stata file you're having trouble with is somewhere at
> https://heidata.uni-heidelberg.de/dvn/ but can you please let us know
> where we can find it?


http://dx.doi.org/10.11588/data/10060
File 0_data.tab (downloaded as "saved original")

To my surprise, I could ingest this file now.
The difference is that I've unset the LANG variable before
starting the glassfish domain.
And vice versa: When starting glassfish with LANG=de_DE.UTF-8
the problem reappears.




>
> The RoundRoutines errors are coming from
> https://github.com/IQSS/UNF/blob/9b51c107743fbd92086184186eb1eed4a33abd4b/src/main/java/org/dataverse/unf/RoundRoutines.java#L394
> but like you I don't know if this is really causing the problem. The
> ingest code attempts to calculate the UNF but I'm not sure what the
> behavior is supposed to be if there's a problem calculating it (
> https://github.com/IQSS/dataverse/blob/v4.2/src/main/java/edu/harvard/iq/dataverse/ingest/IngestServiceBean.java#L2096
> ). This may or may not be your actual problem.
>
> Rather than Ubuntu we run RHEL/CentOS at https://dataverse.harvard.edu
> so our ability to help will be limited. However, Dataverse community


Yes, I know :-)


> members such as Lucien van Wouw have apparently figured out how to get
> everything including TwoRavens, rApache and Rserve working on Ubuntu and
> have even captured the config in Puppet at
> https://github.com/IQSS/dataverse-puppet . So one thing you could try is
> cloning that repo and running `vagrant up` and (assuming you get a
> working Dataverse installation) you could upload your Stata file there
> and see if you can reproduce the problem.


Good to know, thanks.


>
> Since you mentioned Shibboleth, I should add that I'm not sure that
> Shibboleth is well supported on Ubuntu. Or rather,
> https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLinuxInstall says
> it's only "officially supported" on RHEL, CentOS, SUSE, and OpenSUSE,
> which are all RPM-based distributions. Certainly
> http://guides.dataverse.org/en/4.2/installation/shibboleth.html is
> oriented toward RHEL/CentOS. I don't mean to discourage you from trying
> however! I'd love to have more people trying out the Shibboleth feature
> of Dataverse.


For our production systems we use self-compiled Shibboleth service
providers. For our test dataverse I've planned to use the Ubuntu
packages. Let's see ...


>
> Again, if you could please tell us where we can download the Stata file,
> we'd really appreciate it. (Sorry if I missed this.) You could try it
> yourself at https://dataverse-demo.iq.harvard.edu and if it doesn't
> ingest you're welcome to open an issue at
> https://github.com/IQSS/dataverse/issues .


Stata 13 files (Version 117) couldn't be ingested (just as this
was the case for DVN 3.x). Is this correct?
I've tried to ingest such a file at dataverse-demo.iq.harvard.edu.
The warning sign says "DataReader.readBytes called to read zero or
negative number of bytes."


BTW: When trying to upload a file which already exists, the whole
system becomes inoperable (at least for test system with version 4.1). I
have to delete the session cookies to keep on working with the Dataverse
frontend.


Leonhard Maylein



>
> I hope this helps!
>
> Phil
>
> On Thu, Oct 1, 2015 at 5:29 AM, Leonhard Maylein
> <May...@ub.uni-heidelberg.de <mailto:May...@ub.uni-heidelberg.de>> wrote:
>
> Hi,
>
> we've installed Dataverse 4.1 on a test system.
> As we plan to use shibboleth, the glassfish is placed behind an
> apache webserver.
> We've also installed R, rApache and TwoRavens as described
> at
> http://guides.dataverse.org/en/latest/installation/r-rapache-tworavens.html
> (we use Ubuntu 14.04, therefore we had to
> adapt some of the steps).
>
> TwoRavens configuration:
>
> ********************************************
> Directory where TwoRavens is installed: /var/www/html/dataexplore
> Apache config directory: /etc/apache2
> Apache Web Root directory: /var/www/html
> Internet address of the rApache host: xxx.ub.uni-heidelberg.de
> <http://xxx.ub.uni-heidelberg.de>
> <mailto:dataverse-community%2Bunsu...@googlegroups.com>.
> To post to this group, send email to
> dataverse...@googlegroups.com
> <mailto:dataverse...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dataverse-community/560CFCEC.3040901%40ub.uni-heidelberg.de.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Philip Durbin
> Software Developer for http://dataverse.org
> http://www.iq.harvard.edu/people/philip-durbin
>
> --
> You received this message because you are subscribed to the Google
> Groups "Dataverse Users Community" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dataverse-commu...@googlegroups.com
> <mailto:dataverse-commu...@googlegroups.com>.
> To post to this group, send email to
> dataverse...@googlegroups.com
> <mailto:dataverse...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dataverse-community/CABbxx8EeCo040Mx5_cqPQ-mScZdsQdmc-XY8fOrWVtCzOnuU8w%40mail.gmail.com
> <https://groups.google.com/d/msgid/dataverse-community/CABbxx8EeCo040Mx5_cqPQ-mScZdsQdmc-XY8fOrWVtCzOnuU8w%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Philip Durbin

unread,
Oct 1, 2015, 11:45:34 AM10/1/15
to dataverse...@googlegroups.com
Hmm, it sounds like there are three potential issues.

According to http://guides.dataverse.org/en/4.2/user/tabulardataingest/supportedformats.html Dataverse 4.2 does support Stata 13 so can you please create an issue at https://github.com/IQSS/dataverse/issues about the file (and where to download it) that isn't working?

Can you also please create an issue about LANG=de_DE.UTF-8 and Glassfish. This feels like something we should at least document in the Installation Guide. Oh, and please to check out and consider contributing to the Installation Guide improvement and reorganisation proposal at https://groups.google.com/forum/#!topic/dataverse-community/qujUFJDYXG0

I don't know what to say about the "When trying to upload a file which already exists, the whole system becomes inoperable (at least for test system with version 4.1)" issue. If you can reproduce this issue on https://dataverse-demo.iq.harvard.edu which was recently upgraded to 4.2, please open an issue for that too!

Good to know about Shibboleth and Ubuntu. I'm seeing some stuff about osfamily=Debian at https://github.com/IQSS/dataverse-puppet/blob/4.0.1/manifests/apache2/shibboleth.pp so maybe it works? Dunno. :)

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/560D4EE2.3040405%40ub.uni-heidelberg.de.

For more options, visit https://groups.google.com/d/optout.

Lucien van Wouw

unread,
Oct 1, 2015, 6:38:26 PM10/1/15
to Dataverse Users Community, philip...@harvard.edu
Hello all,

The 'declaration of the element "jhoveConfig" could not be found' kept biting on my test box, so I placed this observation here as an issue:

Leonhard Maylein

unread,
Oct 2, 2015, 3:50:32 AM10/2/15
to dataverse...@googlegroups.com
Hi Philip,

regarding the Stata 13 support:

Maybe my test file ("male testing.dta") is corrupt.
It was not ingested correctly at dataverse-demo.iq.harvard.edu

https://dataverse-demo.iq.harvard.edu/dataset.xhtml?persistentId=doi%3A10.5072%2FFK2%2FBLAA3W&version=DRAFT

Another stata 13 file did not cause any problems (file "data_orign.dta"
at the same test dataset).

Similarly, my test installation (Version 4.1) also ingests
"data_orign.dta" but not "male testing.dta".

There is another problem with the file ingest which I could
observe at my test installation

When saving file additions, my browser (firefox) complains about
incorrect xml code. The source code of the page looks like this:

33|{"data":"Success data_orign.tab"}33|{"data":"Success
data_orign.tab"}<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en"><head>
<title>Test - Test Dataverse Maylein Dataverse</title>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8" />
...

This happens frequently but is not reproducible every single time.

Leonhard Maylein


Am 01.10.2015 um 17:45 schrieb Philip Durbin:
> Hmm, it sounds like there are three potential issues.
>
> According to
> http://guides.dataverse.org/en/4.2/user/tabulardataingest/supportedformats.html
> Dataverse 4.2 does support Stata 13 so can you please create an issue at
> https://github.com/IQSS/dataverse/issues about the file (and where to
> download it) that isn't working?
>
> Can you also please create an issue about LANG=de_DE.UTF-8 and
> Glassfish. This feels like something we should at least document in the
> Installation Guide. Oh, and please to check out and consider
> contributing to the Installation Guide improvement and reorganisation
> proposal at
> https://groups.google.com/forum/#!topic/dataverse-community/qujUFJDYXG0
>
> I don't know what to say about the "When trying to upload a file which
> already exists, the whole system becomes inoperable (at least for test
> system with version 4.1)" issue. If you can reproduce this issue on
> https://dataverse-demo.iq.harvard.edu which was recently upgraded to
> 4.2, please open an issue for that too!
>
> Good to know about Shibboleth and Ubuntu. I'm seeing some stuff about
> osfamily=Debian
> athttps://github.com/IQSS/dataverse-puppet/blob/4.0.1/manifests/apache2/shibboleth.pp
Reply all
Reply to author
Forward
0 new messages