Upload a (citation) metadata XML file to Dataverse

[Philipp at UiT]

I've been asked by a colleague at another university in Norway whether it is possible to upload a Dublin Core metadata xml file to Dataverse instead of filling out the citation metadata schema in Dataverse. I guess this is possible via API, but not through the web interface.

I don't really know what use case this should cover. I think they are looking for a way to teach researchers to create metadata without using a specific repository software. But I think if you want to push your metadata into a repository you should do it through an integration with a metadata tool like CEDAR (https://metadatacenter.org) which allows you to manager your metadata through the lifecycle of your data. I don't really see the point in just creating an XML file instead of filling out a schema in a repository software.

Any thoughts on this?

Best, Philipp

[Philip Durbin]

Hi Philipp,

It's a good idea and Dataverse does something similar with a file type from the world of Astronomy called FITS.

At https://dataverse.org/software-features the feature is described in this way:

"Pull header metadata from Astronomy (FITS) files

Dataset metadata prepopulated from FITS file metadata."

From that "features" page, if you click "More information" you be taken to http://guides.dataverse.org/en/latest/user/dataset-management.html#astronomy-fits where you get a longer description of how it works:

"Metadata found in the header section of Flexible Image Transport System (FITS) files are automatically extracted by Dataverse, aggregated and displayed in the Astronomy Domain-Specific Metadata of the Dataset that the file belongs to. This FITS file metadata, is therefore searchable and browsable (facets) at the Dataset-level."

So it's similar to what you're talking about. The researcher uploads a file (again, this is all in the GUI) and certain metadata fields are filled in. But this only works for FITS files, not a metadata XML file.

I can imagine someone creating a "configure" tool at the dataset level that fills in metadata fields based on a Dublin Core XML file or JSON (or whatever) file.

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Philipp at UiT]

Thanks, Phil. I wasn't aware of the FITS feature. As for citation metadata, I still think I'd prefer to handle this through an integration with a metadata management tool or a DMP tool. I'll let my colleague know. Best, Philipp

onsdag 6. november 2019 22.25.50 UTC+1 skrev Philip Durbin følgende:

Hi Philipp,

It's a good idea and Dataverse does something similar with a file type from the world of Astronomy called FITS.

At https://dataverse.org/software-features the feature is described in this way:

"Pull header metadata from Astronomy (FITS) files

Dataset metadata prepopulated from FITS file metadata."

From that "features" page, if you click "More information" you be taken to http://guides.dataverse.org/en/latest/user/dataset-management.html#astronomy-fits where you get a longer description of how it works:

"Metadata found in the header section of Flexible Image Transport System (FITS) files are automatically extracted by Dataverse, aggregated and displayed in the Astronomy Domain-Specific Metadata of the Dataset that the file belongs to. This FITS file metadata, is therefore searchable and browsable (facets) at the Dataset-level."

So it's similar to what you're talking about. The researcher uploads a file (again, this is all in the GUI) and certain metadata fields are filled in. But this only works for FITS files, not a metadata XML file.

I can imagine someone creating a "configure" tool at the dataset level that fills in metadata fields based on a Dublin Core XML file or JSON (or whatever) file.

Phil

On Wed, Nov 6, 2019 at 2:14 PM Philipp at UiT <uit.p...@gmail.com> wrote:

I've been asked by a colleague at another university in Norway whether it is possible to upload a Dublin Core metadata xml file to Dataverse instead of filling out the citation metadata schema in Dataverse. I guess this is possible via API, but not through the web interface.

I don't really know what use case this should cover. I think they are looking for a way to teach researchers to create metadata without using a specific repository software. But I think if you want to push your metadata into a repository you should do it through an integration with a metadata tool like CEDAR (https://metadatacenter.org) which allows you to manager your metadata through the lifecycle of your data. I don't really see the point in just creating an XML file instead of filling out a schema in a repository software.

Any thoughts on this?

Best, Philipp

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

[Philip Durbin]

Sure, we're happy to help facilitate new integrations. This sounds like a "getting data in" integration: http://guides.dataverse.org/en/4.17/admin/integrations.html#getting-data-in

Here are the formats (all XML or JSON) used by the integrations listed on that page to create a dataset:

- OJS: Dublin Core dcterms XML (SWORD)

- OSF: same as OJS

- RSpace: Dataverse's native JSON format

There's also a new tool from Slava at https://github.com/IQSS/dataverse-ddi-converter-tool but I'm not sure if there's a GUI for it:

- DDI to Dataverse converter tool: DDI

With regard to DMP tools, there was a lot of chatter last week starting here: https://github.com/oblassers/dmap/issues/1#issuecomment-547553329

I hope this helps,

Phil

On Thu, Nov 7, 2019 at 2:38 AM Philipp at UiT <uit.p...@gmail.com> wrote:

Thanks, Phil. I wasn't aware of the FITS feature. As for citation metadata, I still think I'd prefer to handle this through an integration with a metadata management tool or a DMP tool. I'll let my colleague know. Best, Philipp

onsdag 6. november 2019 22.25.50 UTC+1 skrev Philip Durbin følgende:

Hi Philipp,

It's a good idea and Dataverse does something similar with a file type from the world of Astronomy called FITS.

At https://dataverse.org/software-features the feature is described in this way:

"Pull header metadata from Astronomy (FITS) files

Dataset metadata prepopulated from FITS file metadata."

From that "features" page, if you click "More information" you be taken to http://guides.dataverse.org/en/latest/user/dataset-management.html#astronomy-fits where you get a longer description of how it works:

"Metadata found in the header section of Flexible Image Transport System (FITS) files are automatically extracted by Dataverse, aggregated and displayed in the Astronomy Domain-Specific Metadata of the Dataset that the file belongs to. This FITS file metadata, is therefore searchable and browsable (facets) at the Dataset-level."

So it's similar to what you're talking about. The researcher uploads a file (again, this is all in the GUI) and certain metadata fields are filled in. But this only works for FITS files, not a metadata XML file.

I can imagine someone creating a "configure" tool at the dataset level that fills in metadata fields based on a Dublin Core XML file or JSON (or whatever) file.

Phil

On Wed, Nov 6, 2019 at 2:14 PM Philipp at UiT <uit.p...@gmail.com> wrote:

I've been asked by a colleague at another university in Norway whether it is possible to upload a Dublin Core metadata xml file to Dataverse instead of filling out the citation metadata schema in Dataverse. I guess this is possible via API, but not through the web interface.

I don't really know what use case this should cover. I think they are looking for a way to teach researchers to create metadata without using a specific repository software. But I think if you want to push your metadata into a repository you should do it through an integration with a metadata tool like CEDAR (https://metadatacenter.org) which allows you to manager your metadata through the lifecycle of your data. I don't really see the point in just creating an XML file instead of filling out a schema in a repository software.

Any thoughts on this?

Best, Philipp

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/4618e2a7-5727-4afe-b4b6-52f73a79fd99%40googlegroups.com.

[Meghan Goodchild]

Hello Philip,

We are trying to help a user who is attempting to upload FITS files to use the extraction feature. Currently, we aren't seeing anything happening upon ingest. I found this ticket (https://github.com/IQSS/dataverse/issues/5919). Does this issue only affect version 4.14? We are on version 4.17.

Thanks,

Meghan

Scholars Portal Dataverse

On Wednesday, November 6, 2019 at 4:25:50 PM UTC-5, Philip Durbin wrote:

Hi Philipp,

It's a good idea and Dataverse does something similar with a file type from the world of Astronomy called FITS.

At https://dataverse.org/software-features the feature is described in this way:

"Pull header metadata from Astronomy (FITS) files

Dataset metadata prepopulated from FITS file metadata."

From that "features" page, if you click "More information" you be taken to http://guides.dataverse.org/en/latest/user/dataset-management.html#astronomy-fits where you get a longer description of how it works:

"Metadata found in the header section of Flexible Image Transport System (FITS) files are automatically extracted by Dataverse, aggregated and displayed in the Astronomy Domain-Specific Metadata of the Dataset that the file belongs to. This FITS file metadata, is therefore searchable and browsable (facets) at the Dataset-level."

So it's similar to what you're talking about. The researcher uploads a file (again, this is all in the GUI) and certain metadata fields are filled in. But this only works for FITS files, not a metadata XML file.

I can imagine someone creating a "configure" tool at the dataset level that fills in metadata fields based on a Dublin Core XML file or JSON (or whatever) file.

Phil

On Wed, Nov 6, 2019 at 2:14 PM Philipp at UiT <uit.p...@gmail.com> wrote:

I've been asked by a colleague at another university in Norway whether it is possible to upload a Dublin Core metadata xml file to Dataverse instead of filling out the citation metadata schema in Dataverse. I guess this is possible via API, but not through the web interface.

I don't really know what use case this should cover. I think they are looking for a way to teach researchers to create metadata without using a specific repository software. But I think if you want to push your metadata into a repository you should do it through an integration with a metadata tool like CEDAR (https://metadatacenter.org) which allows you to manager your metadata through the lifecycle of your data. I don't really see the point in just creating an XML file instead of filling out a schema in a repository software.

Any thoughts on this?

Best, Philipp

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

[Philip Durbin]

Hi Meghan,

Unfortunately, once a bug creeps in (this one is a regression), it tends to stay broken unless someone goes in and fixes it. :)

If you have the time to re-test on https://demo.dataverse.org (which is running 4.18.1 right now) and then leave a comment on that issue you found ( https://github.com/IQSS/dataverse/issues/5919 ) about that FITS feature is working or not, it would be much appreciated!

Thanks,

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--

You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/b816d03c-54bf-416e-bb9d-472513574f23%40googlegroups.com.

[Meghan Goodchild]

Hi Phil,

Thanks for the response. I did a test on demo and left a comment on the issue. 

Thanks,

Meghan

On Monday, December 2, 2019 at 8:33:02 AM UTC-5, Philip Durbin wrote:

Hi Meghan,

Unfortunately, once a bug creeps in (this one is a regression), it tends to stay broken unless someone goes in and fixes it. :)

If you have the time to re-test on https://demo.dataverse.org (which is running 4.18.1 right now) and then leave a comment on that issue you found ( https://github.com/IQSS/dataverse/issues/5919 ) about that FITS feature is working or not, it would be much appreciated!

Thanks,

Phil

On Fri, Nov 29, 2019 at 10:32 AM Meghan Goodchild <meghan.goo...@gmail.com> wrote:

Hello Philip,

We are trying to help a user who is attempting to upload FITS files to use the extraction feature. Currently, we aren't seeing anything happening upon ingest. I found this ticket (https://github.com/IQSS/dataverse/issues/5919). Does this issue only affect version 4.14? We are on version 4.17.

Thanks,

Meghan

Scholars Portal Dataverse

On Wednesday, November 6, 2019 at 4:25:50 PM UTC-5, Philip Durbin wrote:

Hi Philipp,

It's a good idea and Dataverse does something similar with a file type from the world of Astronomy called FITS.

At https://dataverse.org/software-features the feature is described in this way:

"Pull header metadata from Astronomy (FITS) files

Dataset metadata prepopulated from FITS file metadata."

From that "features" page, if you click "More information" you be taken to http://guides.dataverse.org/en/latest/user/dataset-management.html#astronomy-fits where you get a longer description of how it works:

"Metadata found in the header section of Flexible Image Transport System (FITS) files are automatically extracted by Dataverse, aggregated and displayed in the Astronomy Domain-Specific Metadata of the Dataset that the file belongs to. This FITS file metadata, is therefore searchable and browsable (facets) at the Dataset-level."

So it's similar to what you're talking about. The researcher uploads a file (again, this is all in the GUI) and certain metadata fields are filled in. But this only works for FITS files, not a metadata XML file.

I can imagine someone creating a "configure" tool at the dataset level that fills in metadata fields based on a Dublin Core XML file or JSON (or whatever) file.

Phil

On Wed, Nov 6, 2019 at 2:14 PM Philipp at UiT <uit.p...@gmail.com> wrote:

I've been asked by a colleague at another university in Norway whether it is possible to upload a Dublin Core metadata xml file to Dataverse instead of filling out the citation metadata schema in Dataverse. I guess this is possible via API, but not through the web interface.

I don't really know what use case this should cover. I think they are looking for a way to teach researchers to create metadata without using a specific repository software. But I think if you want to push your metadata into a repository you should do it through an integration with a metadata tool like CEDAR (https://metadatacenter.org) which allows you to manager your metadata through the lifecycle of your data. I don't really see the point in just creating an XML file instead of filling out a schema in a repository software.

Any thoughts on this?

Best, Philipp

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/97d3d361-3e25-4dd9-9c57-923fbb5f70ec%40googlegroups.com.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/b816d03c-54bf-416e-bb9d-472513574f23%40googlegroups.com.