PDFgui R-value?
Helen Playford
I am wondering how the Rw value that is quoted at the end of PDF
refinements is calculated? I am assuming it uses a standard equation
as found for other least-squares refinements (e.g. Rietveld) but are
there PDF-specific weighting factors?
As an addendum to the above, is the Rw value directly comparable to
those for Rietveld refinements?
Many thanks,
Helen Playford
Pavol Juhas
> I am wondering how the Rw value that is quoted at the end of PDF
> refinements is calculated? I am assuming it uses a standard equation
> as found for other least-squares refinements (e.g. Rietveld) but are
> there PDF-specific weighting factors?
Hi Helen,
The Rw is calculated according to the equation (A.49), page 64 of the
pdffit manual at http://www.diffpy.org/doc/pdffit/pdf_man.pdf
The PDFgui reports one more value, "Reduced chi squared", equal to
sum(wi * Gdiff**2) / (N - p)
where wi equals inverted squared standard deviation of the observed
data point, N is the number of data points and p number of fitted
parameters.
> As an addendum to the above, is the Rw value directly comparable to
> those for Rietveld refinements?
The formulas for the Rw and Reduced-chi-squared in PDFgui/PDFfit
are equivalent to the GSAS definitions for Rwp and chi**2.
However, they cannot be directly compared, because the standard
deviations in measured PDFs tend to be unreliable or completely
missing, especially for x-ray diffraction data. There are several
reasons for that - deviations in the measured x-ray intensities
might be missing or strongly correlated, especially for area
detectors. The propagation of standard deviations is probably
not correct in the Fourier transformation step that converts to PDF.
Another problem is that PDF is often fitted on oversampled r-grid,
which introduces strong correlations between the neighboring points.
This can be avoided by choosing Nyquist data sampling in the PDFgui
(rstep = pi/qmax), but that still does not help with the other
problems.
I hope this clarifies the matter. Cheers,
Pavol
--
Dr. Pavol Juhas
Applied Physics & Applied Mathematics
Columbia University
Room 200 Mudd Building, MC 4701
500 West 120th Street
New York, NY 10027
Simon Billinge
PDF data being correct:
On Fri, Sep 16, 2011 at 5:52 AM, Pavol Juhas <pj2...@columbia.edu> wrote:
> The formulas for the Rw and Reduced-chi-squared in PDFgui/PDFfit
> are equivalent to the GSAS definitions for Rwp and chi**2.
> However, they cannot be directly compared, because the standard
> deviations in measured PDFs tend to be unreliable or completely
> missing, especially for x-ray diffraction data. There are several
> reasons for that - deviations in the measured x-ray intensities
> might be missing or strongly correlated, especially for area
> detectors. The propagation of standard deviations is probably
> not correct in the Fourier transformation step that converts to PDF.
> Another problem is that PDF is often fitted on oversampled r-grid,
> which introduces strong correlations between the neighboring points.
> This can be avoided by choosing Nyquist data sampling in the PDFgui
> (rstep = pi/qmax), but that still does not help with the other
> problems.
However, his statement "The propagation of standard deviations is probably
not correct in the Fourier transformation step that converts to PDF."
is incorrect. If PDFgetX2 receives correct estimated standard
deviations on the intensity data, then the errors are propagated
correctly all the way to G(r). There is often confusion about how
errors get correlated in the Fourier Transform, and how this somehow
messes things up, but it is a myth. The Fourier transform is just a
double integral, which is done in PDFgetX2 as a double sum through
which it is possible to correctly propagate the errors. This means
that the estimated error on each point in the computed PDF is correct.
This has been discussed in detail in the paper I wrote with Brian
Toby in Acta Crystallographica some time ago (B. H. Toby and S. J. L.
Billinge, Determination of standard uncertainties in fits to pair
distribution functions, Acta Crystallogr. A 60, 315-317 (2004).).
However, the errors on neighboring points in the PDF are not
independent, they are correlated. It is also discussed in that Acta A
paper how to propagate the full error correlations but this is not
implemented anywhere. As Pavol points out, the effects of error
correlations are minimized when you calculate the PDF on the Nyquist
grid, as discussed in a paper that will soon appear in PRB, but is on
arXiv:
Christopher L. Farrow, Margaret Shaw, Hyun-Jeong Kim, Pavol Juhás and
Simon J. L. Billinge, The Nyquist-Shannon sampling theorem and the
atomic pair distribution function, arXiv , arXiv:1104.0874 (2011).
These things together make many of the error estimates and Rw's
reported in PDF refinements around the world statistically
unjustifiable, or just plain wrong for most area detector rapid
acquisition PDF work. We are working to sort this out so that it is
more straightforward to get correct error estimates on things like
refined parameters in PDF work but currently it is caveat emptor.
The measures such as Rw are still useful. It is still a measure of
goodness of fit and a lower Rw means a better fit, but they cannot be
directly compared to Rietveld values in most cases.
S
--
Prof. Simon Billinge
Applied Physics & Applied Mathematics
Columbia University
500 West 120th Street
Room 200 Mudd, MC 4701
New York, NY 10027
Tel: (212)-854-2918 (o) 851-7428 (lab)
Condensed Matter Physics and Materials Science Dept.
Brookhaven National Laboratory
P.O. Box 5000
Upton, NY 11973-5000
(631)-344-5661
email: sb2896 at columbia dot edu
home: http://nirt.pa.msu.edu/