Converting GOFS 3.1 meanstd .ab files to netCDF
Jon Cusick
Hello,
I’m trying to convert the monthly mean u and v components for the GOFS 3.1 .ab HYCOM files [1] into netCDF, using the archv2ncdf3z utility. However, I’m running into indexing errors with idm and jdm, and I’m not sure how to resolve them.
I call the utility as follows, with the file regional.grid.b downloaded from [2] and placed in the current directory. See attached for the input file passed to the utility and the output returned.
$ ./archv2ncdf3z < archv2ncdf3z.in
I’m set idm and jdm to 4500 and 4263, respectively, in the input file based on the contents of regional.grid.b, though something seems to be misaligned with those parameters.
My two asks are:
- Can you provide input on how to resolve the idm, jdm sizing issues seen in the .out file?
- Can you provide guidance on how to use this same utility to subset the resulting netCDF file to a given lat/lon bounding box? Is the workflow to update {i, j}orign and {i, j}dmp in the .in file, where I divide the total idm and jdm by the min/max lat/lon listed in regional.grid.b to get the index offsets?
Thanks,
Jon
[1] https://data.hycom.org/datasets/GLBb0.08/expt_93.0/data/meanstd/
[2] https://data.hycom.org/datasets/GLBy0.08/expt_93.0/topo/
Alan Wallcraft
3298 'jdm ' = latitudinal array size
Jon Cusick
Thanks Alan, I seemed to have just grabbed the wrong topo files by mistake. Using the correct GLBb0.08 files seemed to do the trick. There was a message about archive depth mismatch range, but it was on the order of millimeters, so I don’t expect it to be a major deal.
When reading in a few of the converted netCDF files, it seemed like ~55% of the u component values were filled in with NaNs (at least for 930_archMN.2022_11.a and 930_archMN.2022_12.a so far).
I saw a few older threads on the forum mention a “hycom_NAN” tool [0, 1]. I wanted to try and run that to determine if the issue is in the original data, with the archv2ncdf3z tool, or in the way I’m reading in the resulting netCDF data, but wasn’t sure where/what that NAN tool is.
Where can I find the hycom_NAN tool and do you have any advice about how to go about tracking the issue down?
Thanks,
Jon
[0] https://groups.google.com/a/hycom.org/g/forum/c/JOySdRDmGN0
[1] https://groups.google.com/a/hycom.org/g/forum/c/a3aEmRjN_GY/m/9mYOjs2uBQAJ
From: Alan Wallcraft <alan.wa...@hycom.org>
Sent: Friday, June 27, 2025 17:15
To: HYCOM.org Forum <fo...@hycom.org>
Cc: Jon Cusick <JCu...@aphysci.com>
Subject: Re: Converting GOFS 3.1 meanstd .ab files to netCDF [Non-APS]
GD-APS Security Warning: This email originated from outside of our organization. Please proceed with caution when clicking links or opening attachments.
Alan Wallcraft
306 FIELDS PROCESSED, 0 CONTAINED NaNs and/or Infs
## expt idm jdm kdm iloc jloc lonloc latloc yrflag
## 930 4500 3298 41 3010 24 314.9 -77.9 3
## model-day srfhgt surflx dpbl dpmixl tmix smix thmix umix vmix ubavg vbavg steric nsterc oneta E-P
# 44530.0000 -177.25 -16.9 1.010 1.206 -1.8184 34.2002 36.9702 0.98 0.44 0.10 -0.20 -144.20 -33.05 1.0000 0.000
## model-day covice thkice temice
# 44530.0000 94.75 0.47 -4.19
# k utot vtot p.temp saln p.dens thkns dpth
1 0.98 0.44 -1.8184 34.2002 36.9702 1.000 0.500
2 0.43 0.24 -1.7016 34.2769 37.0205 1.000 1.500
3 -0.19 -0.42 -1.6565 34.3186 37.0495 1.000 2.500
4 -0.41 -0.86 -1.6290 34.3463 37.0689 1.000 3.500
5 -0.42 -0.86 -1.6286 34.3461 37.0687 1.000 4.500
6 -0.42 -0.86 -1.6286 34.3461 37.0687 0.000 5.000
7 -0.42 -0.86 -1.6286 34.3461 37.0687 0.000 5.000
Jon Cusick
I was able to find the hycom_NaN tool within the bin/ dir from the HYCOM tools repo and can confirm that the original .a files do not contain any NaNs nor Infs.
Continuing to investigate the issue…
Jon Cusick
Gotcha, thanks. When running hycom_NaN, I get the same results as you do for 930_archMN.2022_12.a which is good.
It turns out that I was incorrectly indexing the generated netCDF files in post-processing on my end.
The “Latitude” and “Longitude” values in the netCDF files seem to be 2-d arrays, each indexed by both X and Y (which I still don’t fully understand – I would have expected each one to be a 1-d array).
I found some XY and ij to lonlat conversion scripts in same bin/ directory as the hycom_NaN too and those were very helpful. Using those to work backwards and then interact with the netCDF XY values instead of lat/lon enabled me to get things looking good on my end.
I appreciate the quick responses the last few days, thanks Alan!
From: Alan Wallcraft <alan.wa...@hycom.org>
Sent: Monday, June 30, 2025 15:25
To: HYCOM.org Forum <fo...@hycom.org>
Cc: Jon Cusick <JCu...@aphysci.com>; Alan Wallcraft <alan.wa...@hycom.org>
Subject: Re: Converting GOFS 3.1 meanstd .ab files to netCDF
The archive depth mismatch range message is normal and millimeters is OK.
The NaN checker is hycom_NaN
There are no NaN's in the archive file:
narwhal07 204> hycom_NaN 930_archMN.2022_12.a 4500 3298
306 FIELDS PROCESSED, 0 CONTAINED NaNs and/or Infs
What i,j locations are exhibiting this? You can use hycom_profile to see what the archive file has at such a location. At a shallow location there will only be five non-zero layers:
narwhal07 209> hycom_profile 930_archMN.2022_12.a 3010 24
## expt idm jdm kdm iloc jloc lonloc latloc yrflag
## 930 4500 3298 41 3010 24 314.9 -77.9 3
## model-day srfhgt surflx dpbl dpmixl tmix smix thmix umix vmix ubavg vbavg steric nsterc oneta E-P
# 44530.0000 -177.25 -16.9 1.010 1.206 -1.8184 34.2002 36.9702 0.98 0.44 0.10 -0.20 -144.20 -33.05 1.0000 0.000
## model-day covice thkice temice
# 44530.0000 94.75 0.47 -4.19
# k utot vtot p.temp saln p.dens thkns dpth
1 0.98 0.44 -1.8184 34.2002 36.9702 1.000 0.500
2 0.43 0.24 -1.7016 34.2769 37.0205 1.000 1.500
3 -0.19 -0.42 -1.6565 34.3186 37.0495 1.000 2.500
4 -0.41 -0.86 -1.6290 34.3463 37.0689 1.000 3.500
5 -0.42 -0.86 -1.6286 34.3461 37.0687 1.000 4.500
6 -0.42 -0.86 -1.6286 34.3461 37.0687 0.000 5.000
7 -0.42 -0.86 -1.6286 34.3461 37.0687 0.000 5.000
The utot and vtot columns are usually interpolated to the p-grid, which is what the 3z program does as well.
I think the "3z" version would have data void for levels below 5 m.
Alan.
On Monday, June 30, 2025 at 2:22:22 PM UTC-4 Jon Cusick wrote:
Thanks Alan, I seemed to have just grabbed the wrong topo files by mistake. Using the correct GLBb0.08 files seemed to do the trick. There was a message about archive depth mismatch range, but it was on the order of millimeters, so I don’t expect it to be a major deal.
When reading in a few of the converted netCDF files, it seemed like ~55% of the u component values were filled in with NaNs (at least for 930_archMN.2022_11.a and 930_archMN.2022_12.a so far).
I saw a few older threads on the forum mention a “hycom_NAN” tool [0, 1]. I wanted to try and run that to determine if the issue is in the original data, with the archv2ncdf3z tool, or in the way I’m reading in the resulting netCDF data, but wasn’t sure where/what that NAN tool is.
Where can I find the hycom_NAN tool and do you have any advice about how to go about tracking the issue down?
Thanks,
Jon
[0] https://groups.google.com/a/hycom.org/g/forum/c/JOySdRDmGN0
[1] https://groups.google.com/a/hycom.org/g/forum/c/a3aEmRjN_GY/m/9mYOjs2uBQAJ
From: Alan Wallcraft <alan.wa...@hycom.org>
Sent: Friday, June 27, 2025 17:15
To: HYCOM.org Forum <fo...@hycom.org>
Cc: Jon Cusick <JCu...@aphysci.com>
Subject: Re: Converting GOFS 3.1 meanstd .ab files to netCDF
The native 1/12 degree global grid is GLBb0.08, so use the regional grid and depth from https://data.hycom.org/datasets/GLBb0.08/expt_93.0/topo/
Alan Wallcraft
