Help with sunpy helioviewer problem
peter berrett
Can someone please help me with a downloading problem I am having?
I need to bulk download good quality PNG files for the C2 telescope on the SOHO spacecraft. For example I might want to download all the C2 files for March 2000. The reason I want to download so many files is because I look through these images using python tools I have created in the hope of finding comets.To date I have discovered 45 comets this way.
There are images downloadable from the Sungrazer Nascom site in fits or jpg format. The jpg images are acceptable quality though only for 2010 and later. For 1996-2009 they are quite poor quality. I have tried to process fits images but they are very large files to download and i have not been able to process these to good quality png files, at least not as good quality as are available from Helioviewer.org
Sunpy has a python api for Helioviewer but I have been unable to find a way of downloading a day or a month's worth of c2 images. The few images I have been able to download in png format have been cropped at the edges to 768 x 768. I need full 1024 x 1024 images.
If I could download good quality full c2 png images for dates, weeks and months I could trawl through the 1996-2009 data and hopefully find a lot of missed comets.
The api for helioviewer seems to only permit single downloads and even then I haven't been able to get it to download a full image. Also the code requries you to specify a a particular time and the nearest image (in time) to that is downloaded which is an awkward way to download files.
Can anyone please help me with a python script that will download a day's worth of full c2 PNG files? I can then modify this for longer periods.
Ireland, Jack (GSFC-6710)
Peter,
I was able to download a 1024 x 1024 PNG file of LASCO images using the following commands. The image is attached.
In [1]: from sunpy.net import helioviewer
In [2]: hv = helioviewer.HelioviewerClient()
In [3]: f = hv.download_png('2012/07/16 10:08:00', 11.9, "[SOHO,LASCO,C2,white-light,1,100]",x1=-512*11.9, x2=512*11.9, y1=-512*11.9, y2=512*11
...: .9)
Changing the image time would allow you to get a day’s worth of C2 images.
Hope this helps,
Jack
--
You received this message because you are subscribed to the Google Groups "SunPy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
sunpy+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sunpy/65297a3a-c05a-409c-a5f7-db0a121478f6n%40googlegroups.com.
peter berrett
I have been able to adapt the code and can now download a years worth of data at a time <evil laugh>.
One thing I am curious about though is the construction of your code at line
f = hv.download_png('2012/07/16 10:08:00', 11.9, "[SOHO,LASCO,C2,white-light,1,100]",x1=-512*11.9, x2=512*11.9, y1=-512*11.9, y2=512*11
Ireland, Jack (GSFC-6710)
Peter,
Construction of the script requires a bit more knowledge about the original data. The script I sent prepares a PNG of the Helioviewer JPEG2000 file using the scale size and the extent of the original FITS data. In the original data, the length of one side of a pixel is 11.9 arcseconds. The original image data size is 1024 x 1024 pixels. Also, in these images the center of the Sun is at (0,0) in helioprojective Cartesian coordinates. So the full x-extent of the image in arcseconds is between x1=-512*11.9 and x2=512*11.9. The same applies to the y-extent of the image.
You can get the information on the pixel scale size and the number of pixels in the image through Helioviewer.org. If you select a SOHO LASCO C2 image, press the small “H” button in the image down. This shows the FITS header information. The size of each pixel is given by the CDELT1 and CDELT2 values. These value of CDELT1 and CDELT2 are the same for LASCO C2 data. Similarly, the NAXIS1 and NAXIS2 values give you the number of pixels on each side of the image. This information is available for all images provided by helioviewer.org.
Are you familiar with https://sungrazer.nrl.navy.mil/ ? It’s a good resource for comet reports in LASCO data.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sunpy/0e598c77-3029-461f-a027-f700380f1a95n%40googlegroups.com.
peter berrett
I am well acquainted with the Sungrazer website and its administrator Karl Battams. In fact that website has helped me discover 45 comets, mostly in archive images!
I do have one further followup question due to a fly in the ointment I have encountered. The original fits images for soho are not rotated so the stars travel through the images at weird angles. When making the png files helioviewer doesn't do a straight pixel for pixel conversion from the fits images but instead rotates the image so that solar north is at the top. As you would expect when rotating a square image, the image corners get rotated out of view and replaced with blank space.
Do you know of a way to code the command so that the image is not rotated i.e its a straight conversion from the fits image?
Ireland, Jack (GSFC-6710)
Peter,
All images from helioviewer.org have been rotated compared to their source FITS files so that solar north is in the same direction. The reason for this is that applying rotation on-the-fly is an expensive operation, but doing it once is reasonable for the main purpose of helioviewer, namely the visualization of solar image data.
To get the highest fidelity images you have to use the original FITS data. The FITS data and the image processing software are available online. The software is written in IDL and is available at https://www.lmsal.com/solarsoft/ . The FITS data are also available via the Virtual Solar Observatory.
I wrote the helioviewer LASCO data processing pipeline a few years ago, and if it might help, it is available here:
https://github.com/Helioviewer-Project/jp2gen/tree/master/idl/soho/lasco
This code also relies on the Solarsoft/IDL software. One note - I am by no means an expert on LASCO data processing. I would recommend contacting Karl Battams for more detailed information about LASCO data processing if you wish to pursue that route.
Thanks,
To view this discussion on the web visit
https://groups.google.com/d/msgid/sunpy/87481bec-6cab-45c8-acc3-a1ad0afd0335n%40googlegroups.com.
peter berrett
My analysis software (written in python) is not rotation dependent so I think I can get by by simply masking out the areas that are more or less blank and searching for comets in the remainder. If I find anything moving I can then revert back to the fits images (albeit a poor quality processing of them) and get coordinates to give to Karl from there. Not ideal but its a start.
If anyone from the helioviewer project is reading this I would ask them to seriously consider making available processed versions of fits files for download (with easier downloading of a days images). I read that it was proposed to revisit the archive and reprocess the images at some point to improve their quality. When this is done can you please save a copy of the processed fits image before the rotation and make that available for download? It would also be helpful if the python api could have a simple command to download all the c2 images for a given day.
My justification is that I have written a python program that can search processed c2 images for comets. I have been able to apply this to good quality images for 2009 and later and achieved significant success however the lack of a archive of good quality processed c2 images is stopping me from finding many more missed comets.
One of those missed comets could well be headed for a collision with Earth so I think its time and money well spent. :)
regards