Many think that externalist theories of consciousness (Byrne & Hilbert, Dretske, Hill, Lycan, Tye, Noe) represent our best shot at "naturalizing" consciousness. But many others (especially recent proponents of "phenomenal intentionality program") think such theories can be ruled out from the armchair, on the basis of thought-experiments about brains in vats, swampmen, spectrum inversion, and other exotic cases. The externalists have criticized their arguments but I suggest that their criticisms don't quite hit the nail on the head. I identify what I regard to be the "real trouble" with these arguments, focusing on recent presentations due to Horgan and Tienson, Hawthorne, Shoemaker, Chalmers, and others. In fact, I distinguish between a few different armchair arguments, and I argue that different armchair arguments fall short for different reasons. If anything, armchair considerations might support phenomenal externalism! At the end I suggest that the only way to resolve the dispute, if at all, requires getting out of the armchair and looking at work in psychophysics and neuroscience. I briefly suggest that this work actually supports a kind of phenomenal internalism, which may have important consequences for mind-body problem.