Considering a significant (backward incompatible) refactoring of ulmo

[Dharhas Pothina]

Hi All,

I'm not sure how much uptake of ulmo there is outside of folks at TWDB and my own agency (the USACE). 

Anyway, I'm planning a complete refactor of ulmo with the following (major) target features:

• Move to a plugin system for the services (using stevedore)
  • This will allow enforcement of a more consistent api (i.e. get_sites, get_stations etc would be harmonized, while still retaining flexibilty for individual plugins).
  • Allow including closed plugins that are for non-public datasets
• Consistently use Pandas DataFrames internally with options to serialize to Python Dicts and GeoJSON
• Move duplicate/common functionality into a common place
• Create a common caching system. i.e. expand the hdf5 cache ability that ulmo.usgs.nwis.hdf has to all services
• Python 3 support (While retaining Python 2.7 compatibility) using the 'six' module

I will probably tag the release 1.0 to indicate that it will break backwards compatibility.

If folks are interested in contributing to the refactor or wish to discuss these changes in more detail. Please comment on this issue.

If anyone else is interested in helping or wants to give input on direction. Please comment on the github issue: 

https://github.com/ulmo-dev/ulmo/issues/109

- dharhas

[Emilio Mayorga]

Dharhas asked this:

I'm not sure how much uptake of ulmo there is outside of folks at TWDB and my own agency (the USACE).

Independent of the refactoring work he announced, it seems to me that getting some input on ulmo uptake/usage  from the community would be helpful to Dharhas and ulmo overall.

So, here's my own reporting: I use ulmo regularly, including in "operational" code (automated data harvesting at regular intervals). Currently I use mostly the cuahsi and usgs/nwis stuff, but I expect to start using some of the climate readers later this year.

I'll follow up on the github issue itself, regarding the refactoring work. Thanks for the heads-up, Dharhas. Sounds like a very nice refactoring!

Cheers,

-Emilio

--
You received this message because you are subscribed to the Google Groups "ulmo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ulmo+uns...@googlegroups.com.
To post to this group, send email to ul...@googlegroups.com.
Visit this group at http://groups.google.com/group/ulmo.
For more options, visit https://groups.google.com/d/optout.

[Cameron Bracken]

My 2 cents. I work for Bureau of Reclamation and I use ulmo frequently as do a few coworkers I know of. I suggest it to anyone I can for new data grabbing tasks.  I am fully in support of a refactor if it increases the longevity of ulmo. 

I am also the author of the cdec module so would be happy to help refactoring that.

Thanks,

Cameron

[Dharhas Pothina]

@Cameron : Refactoring the cdec module would be great. I'll ping you when there is a clear path showing what to do. 

Hopefully the refactor will make it easier to add datasets to ulmo without reinventing the wheel in each plugin.

- dharhas