Speaking personally, I use four templates: (based on Debian 9)
base: For sys-*, vault, gpg, shopping, banking, etc.
office: Libreoffice, thunderbird extensions, latex. For work and personal VMs.
dev: Developer tools, compilers, etc. For dev VMs.
untrusted: Media software (vlc, etc.) as well as Chrome.
This lets me keep the individual templates to a more manageable size and prevents me from accidentally mixing up my workflow across VMs.
I would be open to using a more stripped-down base template but I'm not convinced it's worth it.