Depending on how much data you have, I would first try to fine-tune fc8. If this is doing okay, and not over-fitting, then you can try fine-tuning the fully connected layers and conv layers. Additionally, since you only have two classes, you might consider using SigmoidCrossEntropyLoss as opposed to SoftmaxWithLoss.