I could be wrong but off the top of my head I'm not sure there is a 100% reliable way to test whether any arbitrary string is in HFS or Posix format (since technically the forward slash is legal in HFS paths, colons are legal in Posix paths, for partial paths neither character is strictly necessary in either format, and thus a string containing one or the other - or even both - will generally be technically valid when interpreted using both formats).
If you know you're dealing with fully qualified paths though then 99% of the time just checking for a leading slash should yield the desired result. For partial paths then checking whether the string contains a colon or slash is generally the best you can do…
If the file or folder referred to is supposed to already exist though, then most of the time the fact that it doesn't when its path interpreted as one may offer a hint that you should try the other.
Also (not that this helps much in the real world) strings containing consecutive runs of either character are generally illegal in the format which uses it as a delimiter (but not in the other).
-Mark