I've thought about a workaround for this case using Preview URL's, though I haven't come to use it in practice yet:
1) Create the dataset without files (or placeholder files only).
2) Add a note in the description about any files that will be available upon request and instructions for how to make a request (e.g. by using the 'Contact' button for the dataset).
3) Publish the dataset.
4) Create a new draft version of the dataset with the files, but do not publish it.
5) When someone requests access to the file, generate and share a Preview URL to the draft version.
Of course, this has some drawbacks and limitations, but it should work as a somewhat manual access procedure. I guess you can also combine it with a Guestbook if you have at least one unrestricted file.
Regards,
Falco