Interpretation of model index
Rayfo...@aol.com
Hello Christopher,
I wonder if you can give me some guidance on interpretation.
I have 5 RDates A, B, C, D, E. (E being later) Testing them as a single phase, (R-Combine), they are statistically consistent with a relatively short period, T’=4.1; v=4; T’ (5%) 9.5.
However, other interpretations are possible, e.g. a two phase model and a three phase model, with the question put as to which is best supported by the data. Each model gives different Individual Agreements for the A,B,C,D and E dates and for A model agreements. The agreements increase with each model as shown below.
Can I make any deduction from this?
Finally, all three models were put into one overall model, mainly to see if they could be summarised on one chart. (The MCMC treats each as a different run). This caused the single phase to produce an Individual Agreement warning of (50) for ’E’ and the time span between its boundaries to decrease, mainly due to a backwards shift of the end date. The two phase boundaries also decreased, although its Individual Agreements stayed the same. The three phase boundaries stayed the same as previously and gave unaltered Individual Agreement and A model agreement. It was not the intention to do an analysis linking all three models, but it was interesting to see the single phase date E being 'warned out' and the three phase model holding up.
Would there be any justification for saying the three phase model best fits the data?
I apologise if this is a bit long winded, but I couldn't figure out how to explain it more succinctly,
Many thanks,
Ray Kidd
Single Phase model
Plot() A model (96.1)
{
Sequence()
{
Boundary("Start SinglePhase");
Phase("SinglePhase")
{
R_Date("A", 3865, 40); Ind Agreement (87.3)
R_Date("B", 3845, 40); (110.4)
R_Date("C", 3834, 29); (116.6)
R_Date("D", 3828, 29); (116.6)
R_Date("E", 3750, 45); (73.6)
};
Boundary("End SinglePhase");
};
};
Two Phase model
Plot() A model (116.7)
{
Sequence()
{
Boundary("Start TwoContig");
Phase("TwoContig 1")
{
R_Date("A", 3865, 40); Ind Agreement (93.4)
R_Date("B", 3845, 40); (114.3)
R_Date("C", 3834, 29); (115.6)
R_Date("D", 3828, 29); (112.5)
};
Boundary("Transition 1/2");
Phase("TwoContig 2")
{
R_Date("E", 3750, 45); (102.9)
};
Boundary("End TwoContig");
};
};
Three Phase model
Plot() A model (129.1)
{
Sequence()
{
Boundary("Start ThreeContig");
Phase("ThreeContig 1")
{
R_Date("A", 3865, 40); Ind Agreement (105.0)
R_Date("B", 3845, 40); (111.8)
};
Boundary("3Contig Transition 1/2");
Phase("ThreeContig 2")
{
R_Date("C", 3834, 29); (120.1)
R_Date("D", 3828, 29); (120.1)
};
Boundary("3Contig Transition 2/3");
Phase("ThreeContig 3")
{
R_Date("E", 3750, 45); (101.7)
};
Boundary("End ThreeContig");
};
};
All three models as one
· Plot() A model (129.3)
Sequence()
Boundary("Start SinglePhase")
Phase("SinglePhase")
R_Date("A", 3865, 40)
R_Date("B", 3845, 40)
R_Date("C", 3834, 29)
R_Date("D", 3828, 29)
R_Date("E", 3750, 45)
Boundary("End SinglePhase")
R_Combine("OneSinglePhase")
R_Date("A", 3865, 40)
R_Date("B", 3845, 40)
R_Date("C", 3834, 29)
R_Date("D", 3828, 29)
R_Date("E", 3750, 45)
· 
Sequence()
Boundary("Start TwoContig")
Phase("TwoContig 1")
R_Date("A", 3865, 40)
R_Date("B", 3845, 40)
R_Date("C", 3834, 29)
R_Date("D", 3828, 29)
Span("SpanTwoContig 1")
Boundary("Transition 1/2")
Phase("TwoContig 2")
R_Date("E", 3750, 45)
Boundary("End TwoContig")
· 
Sequence()
Boundary("Start ThreeContig")
Phase("ThreeContig 1")
R_Date("A", 3865, 40)
R_Date("B", 3845, 40)
Span("SpanThreeContig 1")
Boundary("3Contig Transition 1/2")
Phase("ThreeContig 2")
R_Date("C", 3834, 29)
R_Date("D", 3828, 29)
Span("SpanThreeContig 2")
Boundary("3Contig Transition 2/3")
Phase("ThreeContig 3")
R_Date("E", 3750, 45)
Boundary("End ThreeContig")
·
Christopher Ramsey
Thanks for this message. There are a couple of issues here:
First is model comparison. The way that you are supposed to do this
is to use Bayes factors. However OxCal does not calculate Bayes
factors directly as I have not worked out the best way to do this with
the range of models available. Their use is also the subject of much
statistical debate.
What OxCal does calculate is an averaged Likelihood ratio for each
model (against a null model which contains no contraints, groupings
etc). This is given as Fmodel under the Model specification window.
The Amodel value is derived from this - as shown in the specification
and does normally fall into a range of about 60-140 for a reasonable
model.
Acceptable values for Fmodel vary depending on n.
If you want to get an average likelihood ratio between two models you
need to compare the Fmodel terms. In your case the range is a factor
of about 2 (which is not enough to be very significant). In general
though I think the use of these measures for model comparison is
fraught with problems. Even with Bayes factors the values considered
significant are pretty arbitrary.
What you can say using any of these measures though is yes your third
model is slightly more likely to be correct.
Secondly you report that when run together you get different agreement
indices for the individual dates. This should not be the case and
when I tried running the models together or separately on the server I
got (within a few % - which is the noise on the MCMC) the same when
running both ways.
I hope this helps - a bit at least!
Christopher