Exclude License bundles from Google indexing?

[Matthias Letsch]

Hi there,

we stored a permission letter for a publication under the license bundle that was supposed to be handled internally. By chance, we came across a Google search result that leads directly to the bitstream URL of the PDF of this letter. This isn't necessarily a big deal, but we've removed the letter because it shouldn't be publicly accessible.

Is it possible to exclude bitstreams of the license bundle from search engine indexing?

Thank you and kind regards,

Matthias

[DSpace Technical Support]

Hi Matthias,

My immediately suggestion would be to consider using your robots.txt file to tell search engines not to index the "Full Item Page" (/items/[uuid]/full), as that's the only page where Licenses are linked to.  That should cause search engine spiders to skip that file.

You'd have to modify the "src/robots.txt.ejs" file in the User Interface, and add something like this (warning: this is untested):

Disallow: /items/*/full

I *think* that would work, but my robots.txt syntax may be a little rusty.

Tim