PDFgetX3 bug with file names

[Tedi-Marie Usher]

Hi all,

I ran into a strange error today when trying to batch process some .chi files from 11-ID-B using PDFgetX3. See the attached image for the error.

I could process the files individually without problem, but got this bug when trying to run them together using the --find command. The files names were "SC_B_0-00000.chi", "SC_B_1-00000.chi", ... "SC_B_6-00000.chi". I figured out that if I changed the file names to remove "00000", the bug would not appear and I could batch process the files together. Even though I can use PDFgetX3 normally again by changing the files names, I wanted to let the developers know.

I was running Python 2.7 in Anaconda and I can provide more version numbers or anything if you'd like.

Regards,
Tedi-Marie

[Pavol Juhas]

On Thu, Apr 28, 2016 at 10:22:55AM -0700, Tedi-Marie Usher wrote:
> I ran into a strange error today when trying to batch process some
> .chi files from 11-ID-B using PDFgetX3. See the attached image for
> the error.
>
> I could process the files individually without problem, but got this
> bug when trying to run them together using the --find command. The
> files names were "SC_B_0-00000.chi", "SC_B_1-00000.chi", ...
> "SC_B_6-00000.chi". I figured out that if I changed the file names
> to remove "00000", the bug would not appear and I could batch
> process the files together. Even though I can use PDFgetX3 normally
> again by changing the files names, I wanted to let the developers
> know.

Hi Tedi-Marie,

The crash is due to some numerical error in polynomial fit and so
it should not depend on the naming of chi files. Perhaps the
"--find" pattern matched some file with invalid entries
(NaN or Inf) or it attempted to process the background scan.

Can you try to run it as

pdfgetx3 --list --find SC "chi"

to check what files get selected for processing? If this is correct
please run pdfgetx3 with the "--verbose=all" option to find for which
chi file does the problem occur. Does that file have any invalid
values such as NaN or Inf and does it look OK when plotted?

If there is still no obvious cause, can you please send me that chi
file and all other inputs (background scan, pdfgetx3.cfg) so I can
try to reproduce the error?

Thank you. BTW, to match the chi files SC_B_0 to SC_B_6 in your
series you can also use

pdfgetx3 --list --find "SC_B_<0-6>" "00000.chi$"

Here the "$" anchors at the end of the filename, i.e., only files
ending with "00000.chi" would be found rather than those that
have it in their names.

Thank you again for reporting the issue.
Best wishes,

Pavol

--
Dr. Pavol Juhas
Condensed Matter Physics and Materials Science Department
Brookhaven National Laboratory
P.O. Box 5000
Upton, NY 11973-5000

[Tedi-Marie Usher]

Hi Pavol,

I was having trouble reproducing the bug but I finally figured it out. 

I have files SC_B_0-00000 through SC_B_6-00000, but the first file, SC_B_0-00000 is the background file. If I leave that file in the folder with the same name as well as having it listed as the background file in pdfgetx3.cfg, and the background scale set to 1.0, I get the error. If the background scale is something less than zero, the error does not occur. That makes sense to me - if you try to subtract 100% of the background from the background then the result will be 0 - hence the error. Later in the day yesterday when I removed the extra "00000" from the file names I also changed the background file name to bkgd.chi. I hadn't run into this before because in measurements I've done previously I set the background scale to something small like 0.2 because of my experimental setup. 

Looks like this one was mostly user error! Thank you anyway for responding and for the note on another way to identify the files I want to run.

Best regards,

Tedi-Marie

[Simon Billinge]

Thanks for posting this though Tedi-Marie.

By conversations like this taking place on the forum it helps us to make better software, and also these are problems that others may run into in the future and this could really help them!

S

--
You received this message because you are subscribed to the Google Groups "diffpy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diffpy-dev+...@googlegroups.com.
To post to this group, send email to diffp...@googlegroups.com.
Visit this group at https://groups.google.com/group/diffpy-dev.
For more options, visit https://groups.google.com/d/optout.

--

----
Prof. Simon Billinge

Applied Physics & Applied Mathematics
Columbia University
500 West 120th Street
Room 200 Mudd, MC 4701
New York, NY 10027
Tel: (212)-854-2918 (o) 851-7428 (lab)

Condensed Matter Physics and Materials Science Dept.

Brookhaven National Laboratory
P.O. Box 5000
Upton, NY 11973-5000

(631)-344-5661

email: sb2896 at columbia dot edu
home: http://thebillingegroup.com

[Tedi-Marie Usher]

Found a typo - I meant " If the background scale is something less than 1, the error does not occur." instead of " If the background scale is something less than zero, the error does not occur."

[Pavol Juhas]

On Fri, Apr 29, 2016 at 09:36:21AM -0400, Simon Billinge wrote:
> Thanks for posting this though Tedi-Marie.
>
> By conversations like this taking place on the forum it helps us to make
> better software, and also these are problems that others may run into in
> the future and this could really help them!

Indeed. Now that I think of it again the behavior is actually
a bug. If background-subtracted intensities are all zero they
should give zero PDF rather than produce a crash.

I found where this happens in the code and will have
it fixed for the next release.

Thanks again for reporting. Cheers,