K9LA propagation tutorial - dead links

[Dave Typinski]

Does anyone know of an updated version of K9LA's tutorial? All the NOAA links
in this document appear to be dead.

http://k9la.us/Propagation_101_23Sep07.pdf

Thanks!
--
Dave

[Phil Erickson]

Hi Dave,

  The first link in Carl's document should be

https://www.swpc.noaa.gov/products-and-data

  I'm sure Carl can update the others; I tried a couple things but I'm sure that "www.swpc.noaa.gov" contains the information, somewhere.  You might find it by poking around.

73

Phil W1PJE

--
Please follow the HamSCI Community Participation Guidelines at http://hamsci.org/hamsci-community-participation-guidelines.
---
You received this message because you are subscribed to the Google Groups "HamSCI" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hamsci+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/5FFBCAF4.10506%40typnet.net.

--

----
Phil Erickson
phil.e...@gmail.com

[Dave Typinski]

Thanks Phil. Yes, the SWPC completely reworked their web site a few years ago.
Spent hours hunting for the new links when that happened. Found most of 'em,
but there are some things that aren't available anymore (e.g., historical 7-day
K index plots).
--
Dave

On 1/10/21 23:33, Phil Erickson wrote:
> Hi Dave,
>
> The first link in Carl's document should be
>
> https://www.swpc.noaa.gov/products-and-data
>
> I'm sure Carl can update the others; I tried a couple things but I'm sure

> that "www.swpc.noaa.gov <http://www.swpc.noaa.gov>" contains the information,

> somewhere. You might find it by poking around.
>
> 73
> Phil W1PJE
>
> On Sun, Jan 10, 2021 at 10:50 PM Dave Typinski <dav...@typnet.net
> <mailto:dav...@typnet.net>> wrote:
>
> Does anyone know of an updated version of K9LA's tutorial? All the NOAA links
> in this document appear to be dead.
>
> http://k9la.us/Propagation_101_23Sep07.pdf
>
> Thanks!
> --
> Dave
>
> --
> Please follow the HamSCI Community Participation Guidelines at
> http://hamsci.org/hamsci-community-participation-guidelines.
> ---
> You received this message because you are subscribed to the Google Groups
> "HamSCI" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hamsci+un...@googlegroups.com

> <mailto:hamsci%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hamsci/5FFBCAF4.10506%40typnet.net.
>
>
>
> --
> ----
> Phil Erickson

> phil.e...@gmail.com <mailto:phil.e...@gmail.com>

>
> --
> Please follow the HamSCI Community Participation Guidelines at
> http://hamsci.org/hamsci-community-participation-guidelines.
> ---
> You received this message because you are subscribed to the Google Groups
> "HamSCI" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to hamsci+un...@googlegroups.com <mailto:hamsci+un...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hamsci/CAAZaqEsqdm6N1YB_PsKAUboyrU_EWSF1OvdVPDL%2BhHC5APwchg%40mail.gmail.com
> <https://groups.google.com/d/msgid/hamsci/CAAZaqEsqdm6N1YB_PsKAUboyrU_EWSF1OvdVPDL%2BhHC5APwchg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[James Secan]

I’ve got a six-day plot of Kp along with other pertinent information at:
<https://spawx.nwra.com/spawx/kpsw.html>.

Jim
3222 NE 89th St
Seattle, WA 98115
(206) 430-0109

> To unsubscribe from this group and stop receiving emails from it, send an email to hamsci+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/5FFBDD3B.90905%40typnet.net.

[Dave Typinski]

Hi Jim,

That's a very nice group of plots. Are historical plots available beyond the
last one or two rotations?

The one in mind from SWPC:
https://services.swpc.noaa.gov/images/station-k-index.png

There doesn't seem to be an FTP archive for these 7-day plots like there is for
the 3-day K index plots.
ftp://ftp.swpc.noaa.gov/pub/warehouse/

I have yet to find the location of the actual K index /data/ used by the SWPC,
current or historical. All I can find are the plots.

I ended up writing a wget script that fires once a day, downloads the 7-day
plot, and saves it with the date in the file name. Those are nice to look at,
but not so useful when trying to get long term data into a CSV file for analysis.
--
Dave

[James Secan]

Dave,

SWPC isn’t in the archival business, that’s the job of what’s now called the National Centers for Environmental Information (used to be NGDC) in Boulder. That said, NCEI does not archive many, if any, SWPC products.

I stopped using the SWPC Kp estimates some time back when the real thing, the Kp index (provisional) calculated by the folks at the Helmholtz Center at the GFZ German Research Center for Geosciences in Potsdam, Germany, became available on a real-time basis. I download the data from their ftp server (URLs at the end) which is what I plot. You can get the full history of the official Kp, plots and data, from them at their website.

GFZ Potsdam URLs:
ftp site: ftp://ftp.gfz-potsdam.de/pub/home/obs/kp-ap/
web site: https://www.gfz-potsdam.de/en/kp-index/

Jim
3222 NE 89th St
Seattle, WA 98115
(206) 430-0109

> To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/5FFC8EB2.6000009%40typnet.net.

[David Themens]

Hey,

Just an FYI, the near-real-time Kp values are reported in the links below (more information available by scrolling to the bottom of Jim's link to their website: https://www.gfz-potsdam.de/en/kp-index/) and are not available in their traditional ftp landing page.

Previous Month: http://www-app3.gfz-potsdam.de/kp_index/qlyymm.wdc
Current Month: http://www-app3.gfz-potsdam.de/kp_index/pqlyymm.wdc (updated every 3 hours on a provisional basis)

If you use Matlab, IDL, or SQLite, we provide databases of Kp and many other geomagnetic (Kp, AE, PC, Dst, etc...) and solar (F10.7, SSN, IG, etc...) indices in the database files used by E-CHAIM, organized in a simple structure format and available here:

SQLite: https://chain-new.chain-project.net/echaim_downloads/DBFiles/CHAIM_DB.db

Matlab: https://chain-new.chain-project.net/echaim_downloads/DBFiles/MATLAB_DB.mat

IDL: https://chain-new.chain-project.net/echaim_downloads/DBFiles/IDL_DB.sav

New versions are updated and pushed every morning at 8:00am Eastern Time. 

Hope this helps.

Cheers,

David

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/EB6367FD-37D5-4DF3-AF7B-374708118F2E%40gmail.com.

[Dave Typinski]

Hi Jim,

Thanks a bunch, very helpful! That's an outstanding resource.
--
Dave

[James Secan]

I have two old pieces of network equipment that might be of interest to some of you DIY types: a SonicWall TZ180 firewall appliance, and a Linksys WRT54GL router. Both have original power bricks and cables, and the router has the two antenna sticks. They worked when last they were used (over a decade ago for the SonicWall, about seven years for the router).

If anyone is interested let me know. These are freebies. I rather they found a new home than be added to our global garbage midden.

[Bob Gerzoff]

Can someone point me to a document, link, or reference that would get me started accessing the geomagnetic and solar data referenced below.  I’m looking for data for the past five years.    I have MATLAB, I have used SQL,  and have no idea what IDL is. I can click the links and save some files, but then I’m lost.

 

Thanks and 73,

 

Bob, WK2Y

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CACyAW57QTAw%3Dt-Nr9Q43_CchOZNkEWDNHJudbCZEsJWB_%2B6OXg%40mail.gmail.com.

[Ethan Miller K8GU]

Bob,

IDL is "Interactive Data Language"...it's sort of a predecessor to the way we use MATLAB and Python today as a data analysis tool.  It is predominantly used in medical imaging and space science, especially for space-based sensor data.  If you know your way around MATLAB or Python, learning IDL is pointless and painful.

I'll let others who may have cracked the datasets below comment on them specifically...

73,

--Ethan, K8GU.

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/SA1PR13MB50716808D503B195A167121ED0FD9%40SA1PR13MB5071.namprd13.prod.outlook.com.

--

http://www.k8gu.com/
Repair.  Re-use.  Re-purpose.  Recycle.   

[Bob Gerzoff]

Thank you.  I’m always willing to avoid pain.

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAFkosZK0JcK4iAV9vkqyoKrLAWx-q3zQkWZ0t%3DoEX6PdkD4QwA%40mail.gmail.com.

[David Themens]

Hi,

I manage that website, so I can probably help you out. 

Those files all contain the same data, just for use in different programming languages (trying to be helpful in providing data in a bunch of different formats for people, hahaha). The Matlab_DB.mat file, for example, is just a Matlab save file with a structure inside. You can open the file using load(Matlab_DB.mat) (or double clicking it if Matlab is the designated default opener for that file format on your system). 

The data in the structure (under the variable name "data") is organized relatively straightforwardly, where there are time variables in the structure in Julian Date format for each time resolution of index.

To convert any of these times into the Matlab datetime object format (Matlab's preferred time variable format) you can use:

times = datetime(data.hourly_jd,'convertfrom','juliandate');

That hourly time variable is used with the AE, PC, and DST indices. 

The threehour_jd time variable is used with the Kp index and ap index (mistakenly labelled as capitalized in the structure).

The daily_jd time variable is used with the F107, F10_27 (27-day smoothed F10.7), and F107_81 (81-day smoothed F10.7 flux).

The monthly_jd time variable is used with the IG index.

Please note, there are indices called API, AP8, DSTI, and PCI. Please ignore these indices, as they are time-integrated versions that are used for my model but aren't useful for most users.

Also, please note that the F10.7 flux variables include the NOAA F10.7 flux forecast for any date after the date on which the database was updated.

If you do use these data for something public facing, please acknowledge the original data providers, if you can. For Kp/ap its GFZ Potsdam, for AE and Dst it's the Kyoto World Data Center, for F10.7 it is Natural Resources Canada, and for PC index it is the Technical University of Denmark (DTU).

I hope this helps. Please let me know if you need any further assistance.

Cheers,

David

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/SA1PR13MB50716808D503B195A167121ED0FD9%40SA1PR13MB5071.namprd13.prod.outlook.com.

[Bob Gerzoff]

That definitely gets  me started.  Thank you!

 

From: David Themens <david....@gmail.com>
Sent: Monday, August 16, 2021 1:36 PM
To: ham...@googlegroups.com; Bob Gerzoff <b...@bobgerzoff.com>
Subject: Re: [HamSCI] K9LA propagation tutorial - dead links

 

 

Hi,

 

I manage that website, so I can probably help you out. 

 

Those files all contain the same data, just for use in different programming languages (trying to be helpful in providing data in a bunch of different formats for people, hahaha). The Matlab_DB.mat file, for example, is just a Matlab save file with a structure inside. You can open the file using load(Matlab_DB.mat) (or double clicking it if Matlab is the designated default opener for that file format on your system). 

 

The data in the structure (under the variable name "data") is organized relatively straightforwardly, where there are time variables in the structure in Julian Date format for each time resolution of index.

 

To convert any of these times into the Matlab datetime object format (Matlab's preferred time variable format) you can use:

 

times = datetime(data.hourly_jd,'convertfrom','juliandate');

 

That hourly time variable is used with the AE, PC, and DST indices. 

 

The threehour_jd time variable is used with the Kp index and ap index (mistakenly labelled as capitalized in the structure).

 

The daily_jd time variable is used with the F107, F10_27 (27-day smoothed F10.7), and F107_81 (81-day smoothed F10.7 flux).

 

The monthly_jd time variable is used with the IG index.

 

Please note, there are indices called API, AP8, DSTI, and PCI. Please ignore these indices, as they are time-integrated versions that are used for my model but aren't useful for most users.

 

Also, please note that the F10.7 flux variables include the NOAA F10.7 flux forecast for any date after the date on which the database was updated.

 

[Michael Fene]

IDL is the language underlying the signal processing package entitled: ENVI.  L3/Harris are now the vendor for ENVI; its pretty expensive.

M

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/SA1PR13MB50716808D503B195A167121ED0FD9%40SA1PR13MB5071.namprd13.prod.outlook.com.

[Gerald Creager]

That still doesn't make IDL... or ENVI easy to master!

gerry

Capt Gerry Creager, CAP

SWR Health Services Officer

Weather and Environmental Support Officer -- Incident Management Team

OKWG Asst Dir Communications Planning

(C) 979.229.5301
Civil Air Patrol, U.S. Air Force Auxiliary
gocivilairpatrol.com

Southwest Region -- Civil Air Patrolhttps://swrcap.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAPioHAbpBQi5iZE6_2Hp%2BHx7O3HNaMu3rYGrB2JsxZDVPoZ65g%40mail.gmail.com.

[Michael Fene]

It Is indeed a heavy lift to master!

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAG3zga5H%3DxY60uMX2jy5Z_KP%2BhwtfLLBUHcpxpeVomRJZXtcrw%40mail.gmail.com.

[Gerald Creager]

I was an early adopter of ENVI. Then Python came along and I went there. At the time, it was stellar.

gerry

Capt Gerry Creager, CAP

SWR Health Services Officer

Weather and Environmental Support Officer -- Incident Management Team

OKWG Asst Dir Communications Planning

(C) 979.229.5301
Civil Air Patrol, U.S. Air Force Auxiliary
gocivilairpatrol.com

Southwest Region -- Civil Air Patrolhttps://swrcap.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAPioHAZcLRmcN4yhh-RUy9a9hj4_VHvCAQxadBS7o4PQ5vAd5w%40mail.gmail.com.

[Frank Howell]

Gerry and others,

ENVI was a top-flight remote sensing platform back in the mid-1990s. I was a $50K annual customer when I was the software coordinator for the college board in Mississippi back then. I coordinated a $60M commercial remote sensing project for NASA as a faculty member at Mississippi State University. ERDAS and ESRI was my two other major licenses. IDL was the major reason many faculty and scientists wanted ENVI.

73,

Frank

K4FMH

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAG3zga7fvULMq%3Ddn7uq7_zu60A9ZWhMCN%2BoVsozhXRkA8TTqXA%40mail.gmail.com.

[Gerald Creager]

Frank

I first met ENVI when it was first released and used it at Texas A&M in our remote sensing lab. Until that time we'd been using Intergraph tools. Unfortunately, I was the only one who learned to use it. Everyone else ran screaming from the room and we started using ESRI products (as an aside, Jack Dangerman kicked me out of an ESRI user's conference for asking when they'd support Linux in an open Q&A). Somewhere in all that, I developed a system that used a TNC to audio-encode a video record of nadir-pointing imagery taken from a Cessna 182, flying pipeline patrol. The data came from an Ashtech GPS that had roll, pitch and yaw. I post-processed against differential receivers and precise ephemeris, and registered and rectified the data for image height and angular issues. I really should have published on that project but it was a sidelight of getting the image data we needed... never seemed like a big deal.

gerry

Capt Gerry Creager, CAP

SWR Health Services Officer

Weather and Environmental Support Officer -- Incident Management Team

OKWG Asst Dir Communications Planning

(C) 979.229.5301
Civil Air Patrol, U.S. Air Force Auxiliary
gocivilairpatrol.com

Southwest Region -- Civil Air Patrolhttps://swrcap.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/hamsci/CAFz2rFidHzsV%2BGR8qSfK36tOox-w8vAqF6bQf54L0Be9t%2B4-hA%40mail.gmail.com.

[Frank Howell]

Gerry,

I guess we had very different experiences with ESRI’s President, Jack Dangermond. He threw you out of an ESRI conference for asking about Linux support (which I, too, wish they would as they did earlier). Jack gave my Lab a crystal award for designing their state-wide site license program, which is still in use today.

Frank

K4FMH