Did some google-ing and re-read a couple chapters of my books. It looks like the solution here is to either write a fragment shader for each potential combination of textures that has a number of samplers that correlates or, have 1 fragment shader that has all of the potential samplers and then push a uniform block indicating which of the samplers are populated.
Anyone have any better solutions?
PS my goal is to render the CryEngine updated sponza.obj from 2010 then maybe render some terrain from a height map and put them together.
So far the old old sponza loads now I'm getting hung up on some some vulkan resources that I seem to be mis-managing.