Tools for academic paper discussion (arXaliv post mortem)

[Ralph Furmaniak]

Hi everybody,

This message is sent to those on the arXaliv list and those who
otherwise were involved in discussions surrounding academic publishing
and arXiv overlays, in order to restart such discussions. So as to
not spam anyone, everyone not on the list is BCCed. If you want to
join in please join the group at
https://groups.google.com/forum/#!forum/arxaliv

The main goal here is to discuss the lessons of arXaliv, to ask what
kind of system would be most helpful, and to see if we can get enough
people helping out to truly accomplish something. Everything is
open-ended now, and the proposals below are but one way forward. Feel
free to forward to anyone else who may be interested.

The problems with arXaliv were as follows.
() The reddit system was brittle and not suited to the particular needs.
() It was a one man project that, after some reddit updates changing
their backend, began to cost hundreds a month.

The lessons were as follows.
() Votes are meaningless but a good comment is priceless.
() Identity and ownership of ones work is important.
() There needs to be canonical references via either arXiv or MathSciNet.
() It's important to plan and get enough buy-in before beginning.

Some ways to make progress:
() There's no canonical source for comments on papers so there are
instances where a result is wrong and it is common knowledge in the
field, but this is not openly said.
() Make a way to keep track of the developments on the arXiv, what you
have read, what you'll skip, what you intend to read, what you intend
to comment on.
() There should be somewhere to go to find out the key papers to study
in each area and follow the important developments.

Here is one possible path forward.
() Make a simple system that aggregates blogs and discussions about
papers in one place.
() Optionally make a simple blogging system for those who do not want
to set it up themselves (off the shelf. Ghost could be good for that,
or Wordpress).
() Make a website that acts as a personal inbox for the arXiv,
allowing you to mark each item as "skip" "to read" "to comment on",
encouraging everyone to write at least a bit on each.
() The system will be open source and the data should be stored
additionally in a publicly available and mirror able format.
() Make a roadmap for subject areas, possibly as an off-the-shelf Wiki.

For those who already have an academic blog I would love to hear what
kind of discovery system would be useful, whether manual or automatic,
whether opt-in or opt-out, whether a ping back or a crawl, whether
it's important to stay on your existing platform or whether you would
prefer to move some writings to a standalone system.

Once we know the path forward we should put together a formal
collaboration, deciding on a managing board and an editorial board.
Make sure there are enough contributors, and a community, set some
milestone goals. So far for web development in addition to myself
Jack Cable has graciously offered to help this summer but it'd be
great to have others in the loop who can contribute and keep things
moving.

Cheers,
Ralph

[Viv]

Notice that SciRate has been back up for some time now:  https://scirate.com/  It is working, used, has at least some comments for some papers (still perhaps a small community actively commenting, and mostly in quant-ph) and I use it in preference to ArXiiv to read/catch up in my field (mostly quant-ph).  AFAIK it interfaces to all of ArXiv, so you can head over there and try it out and see how it compares with what you had in mind.  I still use ArXiv for searching for papers though.

-- Viv

[dylanc...@gmail.com]

Here are some other interesting attempts to solve the problem: 

http://blogs.ams.org/mathgradblog/2013/11/12/selected-papers-network/#sthash.2hw6vgWn.dpbs

http://johncarlosbaez.wordpress.com/2013/06/14/the-selected-papers-network-part-2/

For some reason, it seems fairly difficult to get people to continue adding content to these types of websites.  This seems strange to me, since I have a lot of things to say about papers I have read, but nowhere (standard) to say them.  Viv, it is promising that you find scirate useful for quant-ph.  Maybe we could get something going for some particular subfields of math.  I think it is worth having arxaliv even though scirate exists because the costs of maintaining the website are relatively low (it has already been developed!), and we have the freedom to experiment with the format of the website.  Furthermore, it does not seem like scirate has caught on to an enormous degree.  On the other hand, I think that not-so-successful websites like scirate and selected-papers-network are evidence that it is very difficult to amass usership for a website such as Arxaliv.

[Aram Harrow]

My sense is that there is a big chicken-and-egg problem.  People only want to leave comments when those comments get read, but similarly it's only worth going to websites if there's useful content there.

So scirate is used by the quantum computing community and people from there can expect useful content.  But there's no reason for someone in another field to start using it.

I think astronomy has a very successful network, which again hasn't penetrated other fields.

One possible way around this (along the lines of Ralph's email) is to add features (like a a bibliography manager) that people would find useful even without network effects.  But I think websites like academia.edu have done this and also not created effective commenting communities.

Another possibility is to figure out a sort of federated comment-sharing system (kind of like Selected Papers) so that content gets accumulated even if not every site is successful.  But this has its own challenges.

aram

--
You received this message because you are subscribed to the Google Groups "Arxaliv" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arxaliv+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ralph Furmaniak]

Interesting, it is good to know what has managed to work in various other fields.  What do you think of starting off by being a comprehensive index of existing reviews and blog posts, and using that to bootstrap a commenting system?  Particularly if there is a browser plugin that can inline links (on arxiv, mathscinet, wherever else) to existing discussions.

Sent from smtp+tls

[Aram Harrow]

That sounds like a great idea!

By the way, the site used by the astro-ph community is
www.voxcharta.org

-aram