Not very Google friendly, I would say.
May be that was done for a reason or this might be default option because Harvard has similar robots file.
Any thoughts to help us out?
Thanks in advance,
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/489cf5e5-b39a-4789-877d-da943ce6478d%40googlegroups.com.