{PhotosetLayout} comes like this:
'XYZ....' where X,Y,Z is a number like 1,2,3 etc.
Each X,Y,Z represents a ROW.
The value of each X,Y,Z represents "The Number of Photos in that Row"
So, suppose {PhotosetLayout} = "121"
That means you have 3 rows total, Row 1 has 1 photo, Row 2 has 2 photos, Row 3 has 1 photo
If {PhotosetLayout} = "13"
You have 2 rows total: Row 1 has 1 photo, Row 2 has 3 photos.
I personally have implemented this using JavaScript and the {JSPhotosetLayout} variable.
If you want to implement using PURE CSS, I imagine you'd have to figure out every possible combination of {PhotosetLayout} and create a CSS class to represent the photos correctly. I'm sure someone has published a list of those, but to AFAIK, Tumblr itself does not have a published list of all possible {PhotosetLayout} options.