Estimating test retest and inter rater reliability

[s.w...@sgul.ac.uk]

Dear All

Is it possible to obtain estimates of test retest reliability and
inter rater reliability using the following design?

Time 1 Time 2
S1 rated by R1 S1 rated by R1
S2 rated by R1 S2 rated by R2
S3 rated by R2 S3 rated by R2

S1, 2 and 3 indicating 3 independent samples of as yet undetermined
size.

R1 and 2 indicating two different raters.

The nature of the tool being tested means that it is not practical or
appropriate for simultaneous ratings by R1 and R2 to be carried out.

I had envisaged that by obtaining estimates of test retest reliability
using sample 1 and 3, this could be incorporated in to the calculation
of IRR in Sample 2.

Any advice would be greatly appreciated.

Sarah