Re: [arxiv-api] Looking for someone to build us an interface with arxiv

[Meredith Lucks]

Hi Alex and Everyone,
 I am interested in the project, what did you have in mind as far as the interface, etc?
Let me know if you are interested in talking more about it
Cheers!
Meredith Lucks

On Wed, Apr 27, 2011 at 1:14 PM, Alex Kleiner <alex.k...@gmail.com> wrote:

Hello,

My name is Alex Kleiner, I am looking for a freelancer to create a web interface with arxiv.

This is a small pilot project for now that may be expanded in the future.

Please reply here if you are interested.

Thanks,

Alex.

--
You received this message because you are subscribed to the Google Groups "arXiv api" group.
To post to this group, send email to arxi...@googlegroups.com.
To unsubscribe from this group, send email to arxiv-api+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/arxiv-api?hl=en.

--
M.Lucks
910.236.9623
http://artists.gawker.com/5124109/meredith-lucks

[Massimo Di Pierro]

Hello everybody,

I have been playing around with the arxiv API. First of all good job.

I built this:

http://tests.web2py.com/arxiv/default/demo
http://tests.web2py.com/arxiv

It allows searching the arXive, download pdf, and comment on papers
(including using latex formulas rendered by google chart API). It
allows star rating, user tagging and user bookmarking but for now I
have disabled those functions. It is inspired by this http://tests.web2py.com/nersc
which is the new NERSC interface for Lattice QCD data.

Is anybody interested in help? Do you have suggestions for
improvement? Are there features you would like me to add?
I do not want to host it. Is anybody interested?

I am planning to add an auto-count that rates the paper based on
popularity of clicks.
I am also planning to cluster papers based clicks by the same users
and display cluster maps.
I have a script to do nightly downloads to my kindle but I would like
to integrate it with the web app.

I encountered some problems which I have not been able to solve and
perhaps you can help:

1) I can search by id_list but when I query I do not get an id_list in
the entries. I am building it form the id.
2) I cannot find a way to download the paper abstract
3) Technically the structure of the entry is invalid because author is
repeated and that is not allowed. Because of this python feedparser
can only get the first author. I can find a workaround to this by
writing my own parser but perhaps you have a better solution.

Massimo

[Thorsten S]

can you clarify? you are looking for a list of all the article
identifiers matching your query? as you point out this is simply the
collection of all the identifiers in the individual entries in the
response feed. is that not sufficient?

> 2) I cannot find a way to download the paper abstract

you can use OAI-PMH to get the full metadata in various representations

http://export.arxiv.org/oai2?verb=Identify
http://export.arxiv.org/oai2?verb=ListMetadataFormats
and then
http://export.arxiv.org/oai2?verb=GetRecord&metadataPrefix=arXiv&identifier=oai:arXiv.org:1101.0001

> 3) Technically the structure of the entry is invalid because author is
> repeated and that is not allowed. Because of this python feedparser
> can only get the first author.  I can find a workaround to this by
> writing my own parser but perhaps you have a better solution.

the author element is repeatable see RFC 4287 and note that (*)
indicates that the element is repeatable

4.1.2. The "atom:entry" Element

The "atom:entry" element represents an individual entry, acting as a
container for metadata and data associated with the entry. This
element can appear as a child of the atom:feed element, or it can
appear as the document (i.e., top-level) element of a stand-alone
Atom Entry Document.

atomEntry =
element atom:entry {
atomCommonAttributes,
(atomAuthor*
& atomCategory*
& atomContent?
& atomContributor*
& atomId
& atomLink*
& atomPublished?
& atomRights?
& atomSource?
& atomSummary?
& atomTitle
& atomUpdated
& extensionElement*)
}

and also in the same section:

o atom:entry elements MUST contain one or more atom:author elements,

Cheers
T.

>
> Massimo

[Peter Melchior]

Hello Massimo and everybody,

the website you set up contains many ideas (bookmarking, tagging,
commenting) we also found important for the scientific discussion of
the latest papers. Our main motivation was the lack of public
discussion/feedback regarding the quality of papers published at an
steadily increasing pace. Since we didn't find anything adequate, I
created the website

http://www.paperrater.org

initially as a tool for our group, but it became quickly apparent that
we want to open it to the public.

The website is online since a couple of months now, and so far used by
(a fairly small number of) astronomers and cosmologists, but the
features are available for all arXiv categories. It makes use of the
RSS feed of the daily submissions and the arXiv API for papers, which
were posted before the time we started reading in the feeds. It also
links to the ADS (Astrophysics Data System), which holds the records
for the vast majority of refereed papers in physics and astronomy.
This way it is not limited to papers on arXiv, although it's probably
most useful for them.

We have implemented several features to connect papers on the basis of
tags they share, or users on the basis of papers they have read. This
is particularly useful to catch up with the literature after vacation
or trips.

In summer I plan a relaunch with several improvements, (e.g. a feature
to export the list of bookmarked papers as bibtex library, but also
LaTeX support). Massimo, if you are interested we may join our
efforts.

Best regards,

Peter Melchior

[Massimo Di Pierro]

Hello Peter, nice system, what is it built with?

I posted the source code on mine under google code (mercurial) at

http://code.google.com/p/arxiv-interface/

with a BSD license.

You may want to look into this file (each function is a web page)

http://code.google.com/p/arxiv-interface/source/browse/models/db_archiv.py

and these file:

http://code.google.com/p/arxiv-interface/source/browse/models/arxiv.py
http://code.google.com/p/arxiv-interface/source/browse/controllers/default.py

The rest is pretty much scaffolding stuff. It requires web2py
(web2py.com).

It would be great if we could collaborate.
I have to finish a few other projects during the summer but I can work
more on this in fall.

Massimo

> > have disabled those functions. It is inspired by thishttp://tests.web2py.com/nersc