Sorry for this initial oversight in the upgrade. This has been fixed, but it looks like a few users (you included) were affected in the initial roll-out, and they'll have to update their Report Configuration sheets manually.
How to fix it (i.e. get the old behavior):
The v4 Reporting API changed it's default includeEmptyRows value from false to true, and that's what's causing this problem for you. To get the old behavior you can set the hidden configuration parameter
Include Empty Rows to TRUE, and it should be exactly as before.
Please let me know if you experience any other issues as a result of the upgrade. You can also check out the updated documentation to see what's changed: