customize intra-dataverse metadata and search?

[gja...@ucsb.edu]

Hello, I've poked around a bit but haven't quite been able to find an answer to my question: Is it possible to customize the metadata and search facets within a particular dataverse?

The background to this question is that I'm encountering datasets that might be appropriate to put in a dataverse within Dataverse, but the metadata and desired search criteria are quite specific to that dataset.  E.g., search criteria might include things like experiment type (drawn from a dataset-specific vocabulary), laser setting, is the laser focused or not, timing, and many other such quantities that don't generalize and wouldn't be found in any garden variety metadata standard.  And, while simple tagging can be used as a way to implement some of these criteria, tags don't suffice in all cases, and one also loses the virtues of faceted search.  Ergo...

1. Is it possible to create a custom metadata block that can then be used by a dataverse?  If so, is this something an end user can do, or does it require an administrator or installation admin to do?

2. Assuming a dataverse uses a custom metadata block, is it possible to specify search facets for that dataverse (only), based on the metadata fields in the block?

3. If a metadata field is numeric (is it possible to specify the types of metadata fields?), is it possible to do numeric range searching over that metadata?

Thanks in advance,

-Greg

[Philip Durbin]

Hi Greg,

It's fine to poke around, of course, but I'm glad you asked. None of this is obvious. I'm glad you're back since we haven't heard[1] from you in a while! :)

People definitely want to add metadata that doesn't fit into any garden variety standard. It sounds like you're already familiar with the Dataverse term "custom metadata block", which is great. Speaking of your laser example, there's a group at Harvard Medical School that's using Dataverse to record metadata fields such as "Beamline and Collection Facility" and "PDB ID". I'll attach a screenshot. These custom fields are so important that we added a feature to show them just below the citation on the dataset page (currently this affects the entire installation and the config option is called ":CustomDatasetSummaryFields"). I'll add a screenshot of that too.

As to your specific questions, yes, it's possible to create your own custom metadata block but the process is not well documented. Please see the Google doc linked from https://github.com/IQSS/dataverse/issues/3168 for details and do know that end users can't load them into an installation of Dataverse. A tab separated values file must be created and loaded up using curl by a sysadmin with access to "admin" APIs available only to localhost.

Once the custom metadata block is available to the installation of Dataverse, it is fully supported for an end user to make use of the custom metadata block including making search facets show up. For details, please see http://guides.dataverse.org/en/4.9/user/dataverse-management.html#general-information

Sadly, no real progress has been made on search features since the last time you wrote. While Solr, the search engine we use, is fully capable of handling range searches across numbers or dates, this feature is not exposed via the Dataverse GUI. That is to say, range searches will work fine if you hack on the URL or use the Search API but you won't have a friendly GUI for this. For more on the topic of range searches, please see https://github.com/IQSS/dataverse/issues/2291 but that issue is about dates and a new issue about numeric range searches would be welcome.

I hope this helps. Please keep the questions coming!

Phil

1. questions from potential adopter: https://groups.google.com/d/msg/dataverse-community/d2HUkD6S-jA/EcKrxc0RAgAJ

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1b444fbf-1ca6-4fc1-87f9-18a671e847a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Philip Durbin
Software Developer for http://dataverse.org
http://www.iq.harvard.edu/people/philip-durbin

[Pete Meyer]

Hi Greg,

On Tuesday, June 19, 2018 at 4:00:24 PM UTC-4, gja...@ucsb.edu wrote:

Hello, I've poked around a bit but haven't quite been able to find an answer to my question: Is it possible to customize the metadata and search facets within a particular dataverse?

The background to this question is that I'm encountering datasets that might be appropriate to put in a dataverse within Dataverse, but the metadata and desired search criteria are quite specific to that dataset.  E.g., search criteria might include things like experiment type (drawn from a dataset-specific vocabulary), laser setting, is the laser focused or not, timing, and many other such quantities that don't generalize and wouldn't be found in any garden variety metadata standard.  And, while simple tagging can be used as a way to implement some of these criteria, tags don't suffice in all cases, and one also loses the virtues of faceted search.  Ergo...

1. Is it possible to create a custom metadata block that can then be used by a dataverse?  If so, is this something an end user can do, or does it require an administrator or installation admin to do?

To add to Phil's comments / suggestions, I'd recommend experimenting with custom metadata blocks using a development installation; in particular one where the database can be dropped and reset to a fresh configuration easily.  This helps avoiding issues of creating a dataset using one iteration of a custom metadata block, changing it, and trying to delete the dataset.

 

2. Assuming a dataverse uses a custom metadata block, is it possible to specify search facets for that dataverse (only), based on the metadata fields in the block?

In addition to the UI, this can be done by API as well.

Best,

Pete