Hi Scott,
Unfortunately, Swivel does not currently understand times by themselves: they have to be attached to a date (so Swivel would have understood the time as a time if the two columns of your data set had been in one column). As it is right now, Swivel is probably only understanding the time column as text (which is probably why you are not getting any graphs out of the data).
I'll make sure that this is added to our engineers' list of other data types so that you will be able to graph date vs. time properly.
-Seema