Re: [opendap-tech] Hyrax 1.15.1

9 views
Skip to first unread message

Joe Lee

unread,
Feb 26, 2019, 10:16:56 PM2/26/19
to James Gallagher, OPeNDAP Inc., ope...@opendap.org
Hi, James!

  Do you have any detailed instruction / documentation  on running dmrpp_module with S3?

  I could run Hyrax on AWS EC2 free tier but space is very limited after instalation so
  I need to run dmrpp_module to handle 80G HDF5 file on S3.

  Sincerely,

--
Akadio: Let AI solve your business problem using Bigdata in real-time with Spark ML + HDF Streaming.


On Mon, Nov 26, 2018 at 1:32 PM James Gallagher <jgall...@opendap.org> wrote:
Greetings!

We are happy to announce that Hyrax 1.15.1 is released. This version is (mostly) a bug-fix release for Hyrax 1.15.0. See https://www.opendap.org/software/hyrax/1.15.1.

The MDS (Metadata Store) was installed incorrectly from the CentOS rpm packages; fixed.

This version of the server also includes the first developer (i.e., source) release of the BALTO brokering code and an alpha version of JSON-LD support for BALTO/Hyrax.

Included, in addition to the RPM packages for CentOS 6 and 7 and source tar balls, are Docker images for the server both with and without the ncWMS service. See https://hub.docker.com/u/opendap/ or the link on the Hyrax release page.

Specific tickets fixed for 1.15.1

HK-277 hyrax-dependencies on the web site are out of date
HK-269 The current BES RPM package installs with MDS unwritable
HK-258 Modify roi() server-side function so that it is more flexible in the kinds of arguments it can use
HK-215, HK-246 Support subsetting a 3D+ variable using the roi() function
HK-268, HK-108, HK-257, HK-136, HK-224, HK-249, HK-152, HK-222 Implement  Continuous Delivery for Hyrax binary packages
HK-214, Add Debian package build to libdap and BES
HK-103, Build an initial version of the BALTO broker for two different data source types: Hyrax and Files accessible using HTTP
HK-22 The max_response_size limit is not working.
HK-1, HK-2, HK-3, HK-10 Data Transfer Errors in DMR++

The full release notes follow.

--
You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

--
James Gallagher
jgall...@opendap.org




--
You received this message because you are subscribed to the Google Groups "OPeNDAP Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendap-tech...@opendap.org.
To post to this group, send email to openda...@opendap.org.
Visit this group at https://groups.google.com/a/opendap.org/group/opendap-tech/.
For more options, visit https://groups.google.com/a/opendap.org/d/optout.

Joe Lee

unread,
Mar 6, 2019, 1:15:32 AM3/6/19
to James Gallagher, Tech OPeNDAP, ope...@opendap.org

Hi, James!

 

Thank you for tips on build_dmrpp.

I made a tiny patch for help message of build_dmrpp tool on GitHub.

 

However, I could not generate DMR++ map with -r option.

I think the -r option is supposed to write an XML from HDF5 file

 but the tool failed with an obscure error:

 

[ec2-user@ip-172-31-18-183 hyrax]$ ./build/bin/build_dmrpp -f CER_SYN1deg-1Hour_Terra-Aqua-MODIS_Edition4A_407406.20180601.nc4  -r test.dmrpp

Error: ^{

 

I built Hyrax from the source for EC2.

 I think my Hyrax is working with http_d_int.h5.dmrpp

after I whitelist cloudydap s3 in gateway module.

 

  Do you have any clue on how to debug / make -r option work?

 

 

 

 

From: James Gallagher <jgall...@opendap.org>
Sent: Tuesday, March 05, 2019 12:23 PM
To: Joe Lee <hyo...@hdfgroup.org>
Cc: James Gallagher <jgall...@opendap.org>; Tech OPeNDAP <openda...@opendap.org>; ope...@opendap.org
Subject: Re: [opendap-tech] Hyrax 1.15.1

 

 



On Feb 26, 2019, at 20:16, Joe Lee <hyo...@hdfgroup.org> wrote:

 

Hi, James!

 

  Do you have any detailed instruction / documentation  on running dmrpp_module with S3?

 

No. If you just have one file, use EBS. But you raise a good point!! We should have docs for this.

 

Here’s an outline - it’s really not hard. 

 

In the DMR++ code there’s a tool called ./build_dmrpp that is used to seed the Metadata Store with the DMR++ needed by the handler to read from the granule when it’s in S3. The DMR++ is added to the MDS using a certain URL pathname and that URL path is used to access the DMR++ by the server. The DMR++ has the S3 URL of the granule embedded in it.

 

It’s pretty much the wild west at this point. But I think you can get it working. Let me know how it goes; we can use your experiences and ideas to make it simpler for less technical users.

 

Thanks,

James

--

James Gallagher

 

 

 

 

Reply all
Reply to author
Forward
0 new messages