Here's the issue in an even simpler system:
------------------------------------------------------------------------------------------------
from diffpy.structure import loadStructure
from diffpy.srreal.pdfcalculator import DebyePDFCalculator
# Create Pt dimer embedded in Cr matrix
embedded = Structure()
embedded.addNewAtom(atype='Pt', xyz=[0, 0, 1])
embedded.addNewAtom(atype='Pt', xyz=[0, 0, -1])
embedded.addNewAtom(atype='Cr', xyz=[3, 0, 0])
embedded.addNewAtom(atype='Cr', xyz=[-3, 0, 0])
embedded.addNewAtom(atype='Cr', xyz=[0, 3, 0])
embedded.addNewAtom(atype='Cr', xyz=[0, -3, 0])
# Replicate just the Pt dimer
molecule = Structure()
molecule.addNewAtom(atype='Pt', xyz=[0, 0, 1])
molecule.addNewAtom(atype='Pt', xyz=[0, 0, -1])
calc = DebyePDFCalculator(qmin=1, qmax=20)
# Large system calculation
r, g_tot = calc(embedded)
# Isolated system calculation
r, g_mol = calc(molecule)
# Calculate all the pairs summing to the large system
calc.setPairMask([0, 1], [0, 1], True, False)
r, g_PtPt = calc(embedded)
calc.setPairMask([0, 1], [2, 3, 4, 5], True, False)
r, g_PtCr = calc(embedded)
calc.setPairMask([2, 3, 4, 5], [2, 3, 4, 5], True, False)
r, g_CrCr = calc(embedded)
------------------------------------------------------------------------------------------------
Plotting the various contributions to the total PDF of the large system, it appears as though setPairMask is doing exactly what I would expect. However, plotting the (normalized) Pt-Pt contribution calculated from the large system or from the isolated system, it appears as though the relative magnitudes of the Fourier truncation errors are different.
In this dummy example, it makes practically no difference. My concern is that, in systems with more complex internal structures and a mixture of light atom and heavy atom scatterers, this discrepancy makes using setPairMask an unreliable tool for dissecting the total scattering in terms of individual contributions (i.e., these errors may be of a similar magnitude as the real pair correlations in the system).