What’s wrong with the New Scholarometer since February 2019
42 views
Skip to first unread message
Shiu Kum Lam
Mar 1, 2019, 7:15:16 PM3/1/19
to Scholarometer
The old version of Scholarometer showed citation ranking of the top 10 authors almost in any field, over the last 10 years or so. That was stimulating and incentivising, particularly to young scholars, offering them something to emulate. Now Scholarometer has become a ruler and not a meter. It says you can use it to compare; but if you are a computer illiterate, you can’t. It tells you you are 6-foot; it doesn’t say you are the tallest in your class any more. It has become an attachment of Google Scholar. It is so sad that Scholarometer has lost its uniqueness and identification. It will be forgotten.
Fil Menczer
Mar 1, 2019, 8:19:34 PM3/1/19
to Shiu Kum Lam, Scholarometer
Dear Shiu Kum,
Thank you for the feedback. We would like to better understand your
criticism. The new version is showing the same normalized metric and
%ile as the old version. So if it gives you a value of, say, 70% then
you know the scholar is in the top 30% (or has higher impact than 70%
of scholars). We also provide the same data on fields, votes, and h_s
values.
We do understand that you are referring to the interface, on the old
website, that showed the top scholars in each discipline. However,
please note that the new website offers the *same* functionality (but
much faster). On the homepage, you can search for any discipline, and
once the network appears, you can click on the "Top Authors" button to
see the top authors in that field, just as you could on the old
website.
The new design may be causing some confusion, but we hope you will
give the new Scholarometer another shot; the data is much more
reliable because we no longer have ambiguous names in the database. On
top of that, the old version was no longer usable/functional due to
new limitations in Google Scholar; the new version of the extension
allows the system to work again, and is much more robust.
All the best,
-Scholarometer Team
Filippo Menczer
Professor of Informatics and Computer Science
Center for Complex Networks and Systems Research
Indiana University Network Science Institute
http://cnets.indiana.edu/fil
On Fri, Mar 1, 2019 at 7:15 PM Shiu Kum Lam <careca...@gmail.com> wrote:
>
> The old version of Scholarometer showed citation ranking of the top 10 authors almost in any field, over the last 10 years or so. That was stimulating and incentivising, particularly to young scholars, offering them something to emulate. Now Scholarometer has become a ruler and not a meter. It says you can use it to compare; but if you are a computer illiterate, you can’t. It tells you you are 6-foot; it doesn’t say you are the tallest in your class any more. It has become an attachment of Google Scholar. It is so sad that Scholarometer has lost its uniqueness and identification. It will be forgotten.
>
> --
> You received this message because you are subscribed to the Google Groups "Scholarometer" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scholaromete...@googlegroups.com.
> To post to this group, send email to schola...@googlegroups.com.
> Visit this group at https://groups.google.com/group/scholarometer.
> For more options, visit https://groups.google.com/d/optout.
Thank you for the feedback. We would like to better understand your
criticism. The new version is showing the same normalized metric and
%ile as the old version. So if it gives you a value of, say, 70% then
you know the scholar is in the top 30% (or has higher impact than 70%
of scholars). We also provide the same data on fields, votes, and h_s
values.
We do understand that you are referring to the interface, on the old
website, that showed the top scholars in each discipline. However,
please note that the new website offers the *same* functionality (but
much faster). On the homepage, you can search for any discipline, and
once the network appears, you can click on the "Top Authors" button to
see the top authors in that field, just as you could on the old
website.
The new design may be causing some confusion, but we hope you will
give the new Scholarometer another shot; the data is much more
reliable because we no longer have ambiguous names in the database. On
top of that, the old version was no longer usable/functional due to
new limitations in Google Scholar; the new version of the extension
allows the system to work again, and is much more robust.
All the best,
-Scholarometer Team
Filippo Menczer
Professor of Informatics and Computer Science
Center for Complex Networks and Systems Research
Indiana University Network Science Institute
http://cnets.indiana.edu/fil
On Fri, Mar 1, 2019 at 7:15 PM Shiu Kum Lam <careca...@gmail.com> wrote:
>
> The old version of Scholarometer showed citation ranking of the top 10 authors almost in any field, over the last 10 years or so. That was stimulating and incentivising, particularly to young scholars, offering them something to emulate. Now Scholarometer has become a ruler and not a meter. It says you can use it to compare; but if you are a computer illiterate, you can’t. It tells you you are 6-foot; it doesn’t say you are the tallest in your class any more. It has become an attachment of Google Scholar. It is so sad that Scholarometer has lost its uniqueness and identification. It will be forgotten.
>
> You received this message because you are subscribed to the Google Groups "Scholarometer" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scholaromete...@googlegroups.com.
> To post to this group, send email to schola...@googlegroups.com.
> Visit this group at https://groups.google.com/group/scholarometer.
> For more options, visit https://groups.google.com/d/optout.
Shiu Kum Lam
Mar 3, 2019, 1:04:57 PM3/3/19
to Scholarometer
Thank you, Fil, for taking the time to respond.
While I realise that there must have been restrictions, I could only hope, for the fulfilment of Scholarometer’s longtime vision, that the former way of ranking could be restored in future, for the following reasons:
1. It provides a way of measurement of scholarship through a formal ranking system, hitherto unavailable, and measurement of scholarship is uniquely difficult yet vital for any academic individual or institution.
2. In this era of big data, it provides a unique, rational, and systematic (in fact, the then only available) method of making use of the big-data approach to measure and compare scholarship.
3. Comparison of scholarship across disciplines is tough. Disciplines vary widely in size, and the absolute citation numbers may vary according to the size of the discipline itself, so that comparison, i.e. ranking, within a discipline provides a way of assessment of academic achievement.
Best
SK
Former (retired) Dean of Medicine, University of Hong Kong
While I realise that there must have been restrictions, I could only hope, for the fulfilment of Scholarometer’s longtime vision, that the former way of ranking could be restored in future, for the following reasons:
1. It provides a way of measurement of scholarship through a formal ranking system, hitherto unavailable, and measurement of scholarship is uniquely difficult yet vital for any academic individual or institution.
2. In this era of big data, it provides a unique, rational, and systematic (in fact, the then only available) method of making use of the big-data approach to measure and compare scholarship.
3. Comparison of scholarship across disciplines is tough. Disciplines vary widely in size, and the absolute citation numbers may vary according to the size of the discipline itself, so that comparison, i.e. ranking, within a discipline provides a way of assessment of academic achievement.
Best
SK
Former (retired) Dean of Medicine, University of Hong Kong
Tom Beckman
Mar 5, 2019, 6:35:29 PM3/5/19
to Scholarometer
I don't see Top Authors on the Scholarmeter page when I entered a discipline, Integrative and Complementary Medicine:
I'm curious as to why an h-index of 44 is 33rd percentile in this discipline.
Thanks.
Tom
Fil Menczer
Mar 5, 2019, 7:19:43 PM3/5/19
to Tom Beckman, Scholarometer
Dear Tom,
Thanks for the email -- it allowed us to find a bug, which is already
fixed! Please consider that you may need to clear your browser cache
(or force-reload the page) to load the new scripts.
You will see there are three authors with this discipline as a
reliable tag, one of which has h < 44, leading to a percentile of 1/3
= 33%. Obviously we need more annotations to get more meaningful
stats.
-Scholarometer Team
Thanks for the email -- it allowed us to find a bug, which is already
fixed! Please consider that you may need to clear your browser cache
(or force-reload the page) to load the new scripts.
You will see there are three authors with this discipline as a
reliable tag, one of which has h < 44, leading to a percentile of 1/3
= 33%. Obviously we need more annotations to get more meaningful
stats.
-Scholarometer Team
Shiu Kum Lam
Mar 13, 2019, 9:00:38 AM3/13/19
to Scholarometer
Dear Fil
The top author button you indicated usually yields between 1 to 3 or 4 authors in many disciplines I’ve tried, and certainly not the top 10 as in the original Scholarometer, with graphic presentation of each author’s h index. The new version, in this respect, is thus far from satisfactory, to fulfill the 3 scholarstic functions I outlined earlier.
This is quite a pity. Scholarometer has lost its value and in fact its way. It’s not a meter to compare scholarship. Most scholars would simply use Google’s scholar, from which Scholarometer is now based, and to which it now belongs.
SK Lam
The top author button you indicated usually yields between 1 to 3 or 4 authors in many disciplines I’ve tried, and certainly not the top 10 as in the original Scholarometer, with graphic presentation of each author’s h index. The new version, in this respect, is thus far from satisfactory, to fulfill the 3 scholarstic functions I outlined earlier.
This is quite a pity. Scholarometer has lost its value and in fact its way. It’s not a meter to compare scholarship. Most scholars would simply use Google’s scholar, from which Scholarometer is now based, and to which it now belongs.
SK Lam
Fil Menczer
Mar 13, 2019, 4:08:06 PM3/13/19
to Shiu Kum Lam, Scholarometer
Dear SK,
Thank you for the feedback. The decrease in the number of authors for
some disciplines is due to limits in collecting reliable data. Our
database contained a lot of data that was not reliable due to the
limits of our machine learning tools to disambiguate names. As
researchers, we had to prioritize data quality. Therefore we now have
a more stringent criterion for inclusion that eliminates unreliable
author names. As we collect more data through the new Scholarometer
browser extension, eventually we will repopulate the database with
more reliable author data.
The graphic presentation of each author’s h-index (there used to be a
bar, now we just show the number) is an interface detail that we could
tweak if users find it important.
Regarding the criticisms about Scholarometer vs Google Scholar, we
would like to point out that Scholarometer has *always* been based on
data from Google Scholar; that has not changed. Similarly, our
calculation of the universal h_s index to compare scholarship, which
is not available in Google Scholar or other tools, has not changed.
Regards,
-Fil
Thank you for the feedback. The decrease in the number of authors for
some disciplines is due to limits in collecting reliable data. Our
database contained a lot of data that was not reliable due to the
limits of our machine learning tools to disambiguate names. As
researchers, we had to prioritize data quality. Therefore we now have
a more stringent criterion for inclusion that eliminates unreliable
author names. As we collect more data through the new Scholarometer
browser extension, eventually we will repopulate the database with
more reliable author data.
The graphic presentation of each author’s h-index (there used to be a
bar, now we just show the number) is an interface detail that we could
tweak if users find it important.
Regarding the criticisms about Scholarometer vs Google Scholar, we
would like to point out that Scholarometer has *always* been based on
data from Google Scholar; that has not changed. Similarly, our
calculation of the universal h_s index to compare scholarship, which
is not available in Google Scholar or other tools, has not changed.
Regards,
-Fil
Reply all
Reply to author
Forward
0 new messages