Evaluation Pipeline for Jailbreaking Track in Testing Phase

[Avinaash Anand K.]

Dear Organizers, 
I wanted to enquire if the evaluation pipeline would provide us with J(M) &  S(M) at a prompt level during the testing phase (Aggregated / Individually across the two LLMs - released and  heldout). To give the participants with more information to modify their approaches to programatically generate Jailbreak prompts. 

Thanks, 
Avinaash

[Zhen Xiang]

Dear Participants,

In the testing phase, you will not be provided with the details of J(M) &  S(M). The leaderboard will include only the aggregated score for each model and an average score (over the models) for ranking. We don't expect too many modifications to the method during the testing phase since we want to encourage the generalization of the method.

Best,

Organizers