Groups can have child groups, so the way I handle it is to have a high level group that just defines connection parameters, and then the more specialised groups such as Web servers or db servers get added as children of the high level group in my inventory .
You can also define group_vars/all if that makes sense for you.
Inventory looks a bit like this
[windows]
[win_db]
Windb1
Windb2
[win_web]
Winweb1
Winweb2
[windows:children]
win_db
win_web
Hope this helps
Jon