I would get a list of the current user's followers and maintain it in
a cache. Then, when displaying an arbitrary list of users, check each
user to see if that user exists in the cache. They they do, show the
"Unfollow" button. If they don't, show the "Follow" button.