I have a few features that are more expensive to compute than most, and I would like to see whether the network actually ends up relying heavily on them or whether I can leave them out, so as to reduce test time for each new example. I can try training and testing with and without each of these features in turn to see how heavily each affects the prediction accuracy, but I was wondering whether there is an easier way. Is there some way to visualize the "weight" a certain input has in the trained network, or its contribution to the final classification?