I usually just track GitHub issues myself, so my reply is late.
Guard shouldn't need a Gemfile. If there is one, it switches to 'bundler mode', because users expect it to be have that way. (It's convenient).
In general, I'd say that using Guard without a Gemfile isn't recommended. It's better to have a "user Gemfile", where you include in your Gemfile e.g. Gemfile.user if it exists.
(This way you're adding Guard to the Gemfile, but not into the project's source control).
There are ways to run Guard on a different Gemfile than in the current directory - but that's trickly probably. Basically you should specify a BUNDLE_GEMFILE environment variable.
Also, Guard needs a Guardfile. Otherwise it makes no sense, because Guard wouldn't be useful. But, you can pass a custom Guardfile, so there doesn't have to be one in the same directory.
You can also run Guard in a different directory than your project - and it can use the project's Gemfile.
So there are options, but I think Rémy's suggestion is the best (to use Listen directly).
The decision is overall:
1. Use Guard because of plugins you're using - but since you're using Guard plugins, it's best to add Guard and it's plugins into your Gemfile anyway
2. Use Listen, that way you only need Listen installed and a simple script to e.g. call a Rake task to synchronize when files change.