The short answer is "Yes, but...." How much weight; balance front v rear; riding style; rider's ability to adapt all play into it. There is no formula that predicts an individual outcome. As Jim points out, the archives contain long discussions about this very subject in minute detail.
General thought: if you're thinking about something to hold your pocket junk (wallet, phone, keys, etc) plus lunch & a light jacket, probably no problem. Big boxy bar bag with a lock, tools, tubes, more clothes, maps, lots'o'food ( long day's worth), rain jacket, etc., you're gonna notice it. Will it cause a problem? Only one way for you to find out.
Caveat: stuff expands to fill available space.
dougP
On Saturday, October 27, 2012 10:01:04 PM UTC-7, lungimsam wrote: