sunpy
to store timeseries data in going forward. This comes with the context ofpandas.DataFrame
having a number of drawbacksAlthough I'm funded to do this for sunpy, I really want some input from the PyHC community on any advice or recommendations for choosing a suitable container. Perhaps a common data container for Python is something we could start standardising on as a community eventually?
I have made some notes here: https://hackmd.io/@5tUXI9ejSAmY27nKY7SvAA/HJsh8qMPs. I would be greatful for any feedback anyone has on:
astropy.timeseries.TimeSeries
pandas.DataFrame
xarray.DataArray
(or xarray.DataSet
)numpy.ndarray
ndcube
Any other comments or suggestions outside those prompts very welcome too!
All the best,
David
--
You received this message because you are subscribed to the Google Groups "pyhc-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyhc-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CAGm9cHDS-4211BAnMOFYwqWWGrTFRr%3Dz%3D4AUNLuB-4oBjWz1Mg%40mail.gmail.com.
-- Jonathan Niehof, Ph.D. UNH Space Science Center 106 Morse Hall, 8 College Rd., Durham, NH 03824 (603) 862-0649
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/91d2f6b1-77f7-23e9-f9b4-bda3dd2da84c%40unh.edu.
<nextgen.png>
--
You received this message because you are subscribed to the Google Groups "pyhc-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyhc-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CALgKVXZb33Q8zVWZkcc1mNDdZR93NCu8bfAnbyqih5jRvdU9Yw%40mail.gmail.com.
<nextgen.png>
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CA99E425-68DC-4C68-BEBA-A0F497EAD185%40igpp.ucla.edu.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CAGm9cHB1W_8VJacbDY-hXBQ4xJd_KWdtUEs8XLeYO063pqbWPg%40mail.gmail.com.
> Attempting to push the packages to a uniform solution will be a dividing stepI defintely disagree with this - while there are certainly some downsides to developing something uniform, I wouldn't say it's dividing (I certainly wouldn't want to force anyone to use anything!). I don't doubt the work involved in creating something that's widely applicable would be large, but I think the payoff would be even larger. Maintaining one thing is always easier than maintaining n > 1 things (n**2 things if converters between each structure have to be maintained!), and a common interface to data for users who come from historically separate 'domains' would be huge.
a common interface to data for users who come from historically separate 'domains' would be huge.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CAGm9cHB1W_8VJacbDY-hXBQ4xJd_KWdtUEs8XLeYO063pqbWPg%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/351588D6-1ABF-4D0B-B0F3-F6F5E6358F71%40stoneris.com.
Hi all,
Coming back to this discussion, I was wondering if David (or someone else) has followed the changes in pandas with the upcoming version 2.0, which among other things introduce pyarrow as the backend instead of numpy (e.g. https://datapythonista.me/blog/pandas-20-and-the-arrow-revolution-part-i)? I'm not sure if this will address some of the constraints that pandas.dataframes had so far (like memory usage)...
Cheers,
Jan
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/79ba08ab-2f68-cce8-cef6-c14dd372b13e%40utu.fi.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyhc-list/CAFkXHDN%2BGh4%2BvHC1oPJMer1W35F0iOQu5U_Srk1ha_hRUC7qgw%40mail.gmail.com.