Why is Google Scholar not indexing DSpace repositories?

[Rafael Bértoli]

Hello,

We have noticed that Google Scholar is not properly indexing the items in the repository we manage using DSpace 7: https://digital.cic.gba.gob.ar/home. Currently, only about 20% of the publications appear as the canonical version, while in some cases they are listed only as alternative versions of the work.

This issue is of critical importance for us because we have a migration to DSpace 9 of another repository ready to be deployed in production. However, we are concerned about potentially losing visibility in Google Scholar, so we started investigating other repositories running DSpace 7/8/9.

As a general observation, most repositories we checked using DSpace 7/8/9 either did not appear in Google Scholar searches at all or had very low indexing coverage. The few repositories we found with a good indexing rate appeared to be very similar to those that were poorly indexed, at least in terms of meta tags. In some cases, the well-indexed repositories seemed to have more extensive visual customization, but this should not affect Google Scholar indexing.

At this point, we have not been able to reach any clear conclusion from our investigation. We cannot determine whether this is an issue related to DSpace in its newer versions (7/8/9) or simply an indexing behavior on Google Scholar’s side.

We are also aware of the issue reported on GitHub: https://github.com/DSpace/dspace-angular/issues/4509. However, it does not seem that the absence of this element alone fully explains the indexing problem.

Does anyone running a repository on these versions have good indexing in Google Scholar? If so, did you need to apply any specific configuration changes, or did the repository get indexed correctly without additional adjustments?

Thank you in advance.

[Carolyn Sullivan]

I have no advice regarding specific configuration changes, but the technical lead at Google Scholar who you should contact regarding indexing issues is Ollie Hamilton: oll...@google.com

Best,

Carolyn.

[Rafael Bértoli]

Thank you for the contact. I will reach out to them directly with my question. In the meantime, I would appreciate knowing if anyone else has experienced this issue.

[Sean Carte]

Hi Rafael

You're not alone. See this thread: 

https://groups.google.com/g/dspace-tech/c/07j7rGu35NY/m/HDxlKStACAAJ?utm_medium=email&utm_source=footer&pli=1

Please let us know if you have any success with Ollie.

Sean

--
All messages to this mailing list should adhere to the Code of Conduct: https://lyrasis.org/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/21f075f8-c975-446e-a117-d5e11681a10fn%40googlegroups.com.

[Annabel Gooder]

We’ve also had issues with Google Scholar indexing since we upgraded to DSpace 7 in December 2024. It lost a large number of our records, and while it seems to be adding new records ok it hasn’t recovered a significant number of the lost ones. We’ve since moved to DSpace 8, but it hasn’t helped.

Ngā mihi

Annabel

 

Annabel Gooder

Digital Library Specialist

Te Kahu Tauira | Student and Scholarly Services

Waipapa Taumata Rau | University of Auckland

--

All messages to this mailing list should adhere to the Code of Conduct: https://lyrasis.org/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/a615ca20-8319-47f1-90fb-c5f367414b2fn%40googlegroups.com.

[Rafael Bértoli]

It seems this is a general issue affecting some DSpace repositories. Below are a few repositories that appear, based on what we can see, to be poorly indexed in Google Scholar, and we cannot understand why: Cambridge University Library: Kyoto University's: UDESC: Harvard DASH: National University of SIngapore: Fraunhofer-Publica: Infocience: King Abdullah University of Science & Technology: UNSW: UGSpace: Makerere University Institutional Repository: UPSpace: University of Nottingham Repository: ReIn UW:

I also include some repositories that seem to be working properly:

Universidad de los Andes:

UNAL:

Jagiellonian University Repository:

Chalmers Open Digital Repository:

I am aware that the "site:" search does not fully represent what is indexed in Google Scholar, but in many cases there is not even a single item indexed, and this does not seem to be the expected behavior. It is also worth mentioning that I did not include any older repositories, but in general, DSpace 5/6 repositories tend to be well indexed, or at least within acceptable ranges.

[DSpace Technical Support]

Hi All,

I just wanted to chime in here and note that I continue to have yearly meetings with the Google Scholar team to touch base on how DSpace sites are being indexed "in the wild".  As of yet, they have not brought me any general concerns specific to DSpace (which is normally what they do when they realize there's a common issues affecting many DSpace sites).   

That doesn't necessarily mean there's not issues affecting many DSpace sites, but it does mean that Google Scholar may not realize whether a specific issue is common or not.  When they realize there's a common issue, they always reach out to me to get a bug ticket created so that it can be resolved quickly.

There are so many factors that may impact indexing these days, including whether sites are accidentally blocking or limiting access Google Scholar bots. For example, last year I did hear from Google Scholar that they were finding some DSpace sites that were accidentally blocking the Google Scholar bots while attempting to deal with bad-behaving AI-related bots.  So, these days, it's also possible for indexing problems to be related to firewall issues or other bot-management tools/services that are in front of your DSpace.

Overall, my primary recommendation is two things:

1. First, double check that you are following all recommendations of our SEO guide.  These recommendations were created in conjunction with the Google Scholar team: https://wiki.lyrasis.org/display/DSDOC9x/Search+Engine+Optimization  If you've missed or ignored one recommendation, then it may decrease the indexing of your site in Google Scholar.

2. Second, please also reach out to Google Scholar directly if you are having continued issues.  They may be able to narrow down the problem for you... it could be DSpace-related, or could be something else that blocking or slowing down their bot.  If it turns out to be DSpace-related, then Google Scholar will report back to us any common issues they are seeing so that we can create a bug ticket & get them solved in future versions of DSpace.

Overall, I wish there's more advice we can offer here. But, we don't control how/when Google Scholar indexes your site, and cannot debug indexing issues they may be having.  Only Google Scholar can do that, and they will report back to us if there's ways to improve DSpace to further improve/streamline indexing.

Thanks,

Tim