SAVEFILE_CLEANUP error

[Bernat]

Hi folks,

I have an ORDEREDHASH saved in SAV file with IDL 8.7. When I try to restore  the file using IDL 9.0 I get the following error:

% Wrong number of tags defined for structure: ORDEREDHASH.
% RESTORE: Structure not restored due to conflict with existing definition: ORDEREDHASH.

[...]

% Wrong number of tags defined for structure: ORDEREDHASH.
% RESTORE: Structure not restored due to conflict with existing definition: ORDEREDHASH.
% Wrong number of tags defined for structure: ORDEREDHASH.
% RESTORE: Structure not restored due to conflict with existing definition: ORDEREDHASH.
% Execution halted at: $MAIN$

After reading the documentation, I realized that this error occurs because some routines have been ported from PRO to C/C++ code, and ORDEREDHASH is one of these.

it seems that I have to use the routine SAVEFILE_CLEANUP. However, this throws an error:
SAVEFILE_CLEANUP,  '20240622.sav'  
% Loaded DLM: IDL_IDLBRIDGE.
% Compiled module: SAVEFILE_CLEANUP.
% SAVEFILE_CLEANUP: File is a data save file: 20240622.sav.
% Execution halted at: SAVEFILE_CLEANUP  127 /usr/local/nv5/idl90/lib/utilities/savefile_cleanup.pro

I guess that this is a bug?

Bernat

[Chris Torrence]

Hi Bernat,

SAVEFILE_CLEANUP is only supposed to be used for "routine" save files, that contain function and procedure routines. For example, if you had a routine save file that contained the ordered hash code, it would strip that code out of the file so it didn't cause problems.

However, since you have a data save file (containing actual ordered hash objects), then you probably need to use the RELAXED_STRUCTURE_ASSIGNMENT keyword to RESTORE:

RESTORE, '20240622.sav', /RELAXED

I hope this helps.

Cheers,

Chris
NV5 Geospatial

[Bernat]

It worked! Thank you,

Bernat

[Bernat]

Hi everyone, 

I will re-open this thread since I have the same problem as I reported before before but between 2 different IDL versions:
SAV file created with IDL 9.0
restore, sav_file
help
% At $MAIN$          
RES             ORDEREDHASH  <ID=3  NELEMENTS=578>

print, !version.release
9.0.0

The same SAV file cannot be restored with IDL 8.7
restore, sav_file, /relaxed

...

% RESTORE: Recovering incompatible definition of structure ORDEREDHASH using relaxed structure assignment rules.
% RESTORE: Recovering incompatible definition of structure HASH using relaxed structure assignment rules.
% RESTORE: Restored variable: RES.
% At $MAIN$          
RES             ORDEREDHASH  <ID=3  NELEMENTS=0>

print, !version.release
8.7.0

I've found the following comment on the IDL documentation page:
Restoring a save file from previous versions of IDL will replace those definitions, destabilizing IDL. Use the SAVEFILE_CLEANUP routine to purge the ported routines from your save file to avoid this. Calling RESTORE with the /VERBOSE keyword will list all the symbols being restored, to help identify which save file needs to be cleaned up. 

Is there a way to make this work?
Thanks,
B

[Chris]

hey Bernat,

I did some testing. First, that "savefile_cleanup" is only useful for save files with programs in them. It doesn't do anything for data save files.

Second, I think you are out of luck when restoring a newer save file that contains lists or hashes in the older IDL 8.7. Using /relaxed will prevent errors, but it doesn't actually help restore the data. If you create the save file in IDL 9.0, you can successfully restore lists and hashes going back to IDL 8.8, but not any earlier.

The only way to get the data into IDL 8.7 would be to convert it to a different IDL data type, such as an IDL structure or a regular array. Or just save the data out to a plain text file and read it into IDL 8.7 that way.

Finally, since IDL 8.7 was already released a long time ago, there's nothing that we can do to fix the problem in a future IDL release. Sorry about that.

Cheers,

Chris

NV5 Geospatial Software

[Bernat]

Hi Chris, 

Thanks for your answer. This is what I ended up doing: converting the hash into a structure. I then wondered whether it would have been easier to use the IDL-Python bridge and read the file with scipy.io.readsav, but I’m not sure whether hashes and lists are supported.

Cheers,

B