It depends on the model (and if using
PR 2016) and if training or testing. In testing, the
torrvision/crfasrnn with PR 2016 uses about 4 GB memory with minibatch size 1, while
this and
ParseNet with PR 2016 use about 1.3 GB memory with minibatch size 1 (and 5.3 GB with minibatch size 10), which should fit on your card right now if you want to test them out.