Fw: filename.nc.nc

[James Gallagher]

On March 8, 2016 at 4:55:35 PM, Peter Cornillon (pcorn...@me.com) wrote:

Hi James,

I just saw Calloway’s comment re the addition of an extra .nc on the file request. I have found this a bit odd as well but I also understand your argument. Why not change the first ‘.’ to an underscore ‘_’:

filename.nc ==> filename_nc.nc or
filename.h4 ==> filename_h4.nc

That would satisfy both your argument and I think that it would satisfy Calloway’s concern although it is not clear filename.nc.nc breaks their system. He doesn’t have to have a filename that is exactly the same does he?

Peter
—
Peter Cornillon
215 South Ferry Road Telephone: (401) 874-6283
Graduate School of Oceanography
University of Rhode Island
Narragansett, RI 02882 USA

-- 
James Gallagher
jgall...@opendap.org

[James Gallagher]

On March 8, 2016 at 4:55:35 PM, Peter Cornillon (pcorn...@me.com) wrote:

Hi James, 

I just saw Calloway’s comment re the addition of an extra .nc on the file request. I have found this a bit odd as well but I also understand your argument. Why not change the first ‘.’ to an underscore ‘_’: 

filename.nc ==> filename_nc.nc or 
filename.h4 ==> filename_h4.nc 

This seems like one approach that would keep some notion of lineage in the name. 

James

[Rich Signell]

I like this solution also.

---
Rich Signell
81 Queen St
Falmouth, MA 02540

--
You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

[James Gallagher]

On March 9, 2016 at 10:36:19 AM, Peter Cornillon (pcorn...@me.com) wrote:

Another way to handle the provenance without adding another .nc would be to add a global attribute to the file giving the story. In fact, a global variable showing the OPeNDAP request that generated the netCDF fileout would be VERY useful. That way, you could make it so that the filename corresponds to the THREDDS filename while still providing the information and the OPeNDAP request would contain a lot more information than the additional .nc. I would prefer this to the way that it is done now and to not having any information about the request.

Peter

On Mar 9, 2016, at 10:22 AM, James Gallagher <jgall...@opendap.org> wrote:

On March 9, 2016 at 10:08:00 AM, Peter Cornillon (pcorn...@me.com) wrote:

Oh, I see that you did post it to the site. 

Yes - I should have asked first, but it seemed to be a very apropos comment.

BTW, Dan is now leaning toward a meeting with the Rutgers OOI folks, but I need to look into the details. Travel during the rest of March will be problem for me, given that I’ve been in OGC-land for since the first of the month… That said, I’ll sort something out.

James

Peter

On Mar 8, 2016, at 11:33 PM, James Gallagher <jgall...@opendap.org> wrote:

On March 8, 2016 at 4:55:35 PM, Peter Cornillon (pcorn...@me.com) wrote:

Hi James, 

I just saw Calloway’s comment re the addition of an extra .nc on the file request. I have found this a bit odd as well but I also understand your argument. Why not change the first ‘.’ to an underscore ‘_’: 

filename.nc ==> filename_nc.nc or  
filename.h4 ==> filename_h4.nc 

That would satisfy both your argument and I think that it would satisfy Calloway’s concern although it is not clear filename.nc.nc breaks their system. He doesn’t have to have a filename that is exactly the same does he? 

Peter 
— 
Peter Cornillon 
215 South Ferry Road Telephone: (401) 874-6283 
Graduate School of Oceanography 
University of Rhode Island 
Narragansett, RI 02882 USA 

-- 
James Gallagher
jgall...@opendap.org

-- 
You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

—

Peter Cornillon
 215 South Ferry Road               Telephone: (401) 874-6283
  Graduate School of Oceanography
    University of Rhode Island
      Narragansett, RI 02882 USA

-- 
James Gallagher
jgall...@opendap.org

—

Peter Cornillon
 215 South Ferry Road               Telephone: (401) 874-6283
  Graduate School of Oceanography
    University of Rhode Island
      Narragansett, RI 02882 USA

-- 
James Gallagher
jgall...@opendap.org

[Lynnes, Christopher S. (GSFC-5860)]

On Mar 9, 2016, at 11:23 AM, James Gallagher <jgall...@opendap.org> wrote:

On March 9, 2016 at 10:36:19 AM, Peter Cornillon (pcorn...@me.com) wrote:

Another way to handle the provenance without adding another .nc would be to add a global attribute to the file giving the story. In fact, a global variable showing the OPeNDAP request that generated the netCDF fileout would be VERY useful. That way, you could make it so that the filename corresponds to the THREDDS filename while still providing the information and the OPeNDAP request would contain a lot more information than the additional .nc. I would prefer this to the way that it is done now and to not having any information about the request.

I think the :history global attribute is the netCDF convention for just this sort of thing.  I know nco always appends its last runtime command to the :history attribute (it's an array of strings).

--

Christopher Lynnes    NASA/GSFC  301-614-5185

"Innovation is not about saying yes to everything. It's about saying NO to all but the most crucial features." - S. Jobs

[Chris Calloway]

To answer the question regarding "He doesn’t have to have a filename that is exactly the same does he?," the answer is yes, I already have a filename.nc as the original file Hyrax is hosting.

[Chris Calloway]

I was originally prompted to file the issue on Github by a researcher using the Hyrax server with Hydroshare. She was thrown off by the extra file extension because it's not what TDS had done. After filing the issue I reported it to the project PI who thought this is only a cosmetic issue and not worth bothering with.

But just now I had a talk with both the researcher and the project PI about the suggestions here. This is our take:

Hacking the filename is ugly. If we need to change the file name, let the user specify the output filename when the user clicks the "Get as NetCDF 3" or "Get as NetCDF 4" button. Take that as a feature request from the project PI.

That said, we very much like the idea of using the :history global attribute. This is what the history attribute is made for. And there is a history of using the history attribute already. The history attribute should indicate where the original was, what processed the output, and what processing constraints were applied to the output. That would correctly indicate the provenance of the file, and is append-able to the previous history. As provenance is becoming an increasingly important aspect of metadata, as automated tools are harvesting metadata, and as the presence of provenance metadata is increasingly being validated as part of the evaluation of data quality, the argument for a truthful construction of history attributes is most appealing, whether or not if the filename is hacked or selected by the user.

Thanks for the feedback.

--
Sincerely,

Chris Calloway, Applications Analyst
UNC Renaissance Computing Institute
100 Europa Drive, Suite 540, Chapel Hill, NC 27517
(919) 599-3530

On Tuesday, March 8, 2016 at 11:34:43 PM UTC-5, James Gallagher wrote:

[James Gallagher]

Chris, et al.,

Thanks very much for the discussion so far. I don’t want this to be left without any kind of resolution.

Here’s a draft plan I‘d like to put forth for discussion., and if it’s OK, we’ll figure out how to work it into our schedule for releases:

1. For the web interface, add a pop-up that offers a way to choose whatever name the user wants, with the current name as the default. For machine access, we’ll leave the name as it is - hacking the filename is just to problematic and programatic access tools can rename files as they see fit.

2. For the provenance information in the global history attribute, we’ll treat that as separate from the naming issue and figure out how best to do that - recording as much of the information about the source of the data as we can.

#1 and #2 don’t need to be linked.

#1 does need to address both the person using a web interface and a machine running code.

Comments?

James

--

You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

-- 
James Gallagher
jgall...@opendap.org

[Chris Calloway]

James, thank you for keeping this on your radar. Your proposals sound spot on to me.

Cheers, Chris

[James Gallagher]

Chris,

We’re nearly ready for a new Hyrax release but I’m afraid changes to the web form interface are not going to make it in. There is too little time. However, we may be able to come up with a compromise solution:

Add the history attribute information

Add an option to the server that selects between:

1. Appending ‘.nc’ (or ‘.nc4’) to the end of the dataset name (e.g., file.hdf —> file.hdf.nc; file2.nc —> file2.nc.nc). This is the ‘old behavior’

2. Appending ‘.nc’ (or ‘.nc4’) to the end of the dataset base name (e.g., file.hdf —> file.nc; file2.nc —> file2.nc).

Until we can rework the form UI, is this acceptable? 

One question we had was if you wanted an option where the server would return the dataset name completely unchanged. That is: file.hdf —> file.hdf (but the return file is really a netCDF3 file) and file2.nc —> file2.nc

Sorry to disappoint you WRT the form interface, but hopefully these options will serve as a stop-gap. Also, please let me know about the third option.

Thanks,

James

--
You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

-- 
James Gallagher
jgall...@opendap.org