Hi,
I would suggest starting by detecting the individual behavior movements (like licking, orienting etc) that makeup copulation rather than using JAABA directly to detect copulation. If those are the only behaviors that are of interest, then this should suffice.
However, if you need to detect copulation then you can train a new detector where you input the predictions of these individual behavior detectors as input features. To use the predictions of previous JAABA detectors as input, after creating a JAABA project, use "Classifier -> Change Score Features..." menu to add the jab files for the individual behaviors (e.g. licking.jab, orienting.jab). And when you select features, make sure to select the "score" features.
As for detecting background in CTrax I'll suggest posting your query to the Ctrax mailing list. Or you can try using APT (
https://github.com/kristinbranson/APT) with which you can track animals with more details which helps in training accurate behavior detectors in JAABA. We'll be releasing the multi-animal version of APT soon.
Mayank