Implementing and using own store backened for hypothesis

[Apurva Jalit]

Hello all,

I was wondering if we could tweak hypothesis browser extension to work with our own store (backened server). I intend to extract some data from the annotations created by the user and store this extracted information on our servers. Hence we thought we could implement the complete store required by hypthesis ourselves which in turn would make it easy for us to collect all the data we require. 

1. Is it possible to use a store other than the default hypothesis store "https://hypothes.is/api"?(I suppose that is the endpoint used for all the communication done by the browser extension and the backened server)
2. What all changes would be required in the extension code for that?
3. Any documentation or guidelines related to the store implementation to support the browser extensions?

To put in other words, can I use hypothesis like annotatorjs with a custom store? 

Thanks,

Apurva Jalit

[Jon Udell]

Hi Apurva,

You'll probably want to look here first: https://github.com/openannotation/annotator-store

I've been meaning to try the H client against that backend, have not done so yet, but was informed by our lead developer today that he has recently done so. If you try the experiment, please do let us know how it goes.

Cheers,

Jon Udell

Hypothesis

[Jon Udell]

That would be awesome, thank you, I would love to see a C# backend.

Cheers,

Jon

On Mon, May 18, 2015 at 9:24 PM, Apurva Jalit <apurva...@gmail.com> wrote:

Hey,

Since I had already used annotatorjs, I have a store implementation done in C# for that. I can share the git repo link here if some one is interested, it just needs to be rectified a little as of now.

I will try implementing a store backened for H on similar lines and will update here. Will be tracing calls made by the script while creating and searching for annotations and base my store APIs on those.

Thanks,

Apurva Jalit

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@list.hypothes.is.
To post to this group, send email to d...@list.hypothes.is.
To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/9e65e358-3814-4f15-9cc5-94d13d227bed%40list.hypothes.is.

[Jon Udell]

We plan to offer a mechanism for minting dev tokens but meanwhile Raymond Yee cooked up a hack that a few of us are using: https://github.com/rdhyee/hypothesisapi/tree/0.1/hypothesisapi. That's Python not C# but shows how it's done.

Cheers,

Jon

On Wed, May 20, 2015 at 8:39 AM, Apurva Jalit <apurva...@gmail.com> wrote:

Hey,

While trying to create a C# backened, I figured out what all APIs I need to consume at the backend to support browser extension. One of them is "token". I can see in the app.js file how a token is parsed and used but I am not able to figure out how to generate a token in the backend. The function call request.create_token_response() creates a token I suppose, but can you help me understand how does it do that? Is there any standard it follows? 

Thanks,

Apurva Jalit

[Jon Udell]

Thanks Randall. Apurva, sorry, my answer was looking through the wrong end of the telescope, I guess you already knew that part :-).

On Wed, May 20, 2015 at 10:07 AM, Randall Leeds <til...@hypothes.is> wrote:

Yes. The token is a JWT. Annotator has documentation for this.

On May 20, 2015 8:39 AM, "Apurva Jalit" <apurva...@gmail.com> wrote:

Hey,

While trying to create a C# backened, I figured out what all APIs I need to consume at the backend to support browser extension. One of them is "token". I can see in the app.js file how a token is parsed and used but I am not able to figure out how to generate a token in the backend. The function call request.create_token_response() creates a token I suppose, but can you help me understand how does it do that? Is there any standard it follows? 

Thanks,

Apurva Jalit

[Apurva Jalit]

Ok. Thank you.. will look up :)

Thanks,

Apurva Jalit

[Randall Leeds]

http://docs.annotatorjs.org/en/v1.2.x/authentication.html

[Apurva Jalit]

After going through the different API requests made by the hypothes.is browser extension, I have certain queries as follows. Understanding them would help me go ahead with the implementation in synch with the present framework.

1. Authentication and the user management: I can see the three parameters being used for these: X-csrf token, session and client-ID. Can you just help me understand what is the role of each, which component creates it (browser extension or backened server) and a brief about how it is used in the whole framework. 
2. I can see a section for facebook and twitter in the annotation data. What is that exactly used for? (Just to understand what to do with it)
3. If possible can you share with me complete data types of the parameters exchanged between the backend and the extension code for various API calls.

Thanks,

Apurva Jalit

[Randall Leeds]

On May 25, 2015 4:53 AM, "Apurva Jalit" <apurva...@gmail.com> wrote:
>
> After going through the different API requests made by the hypothes.is browser extension, I have certain queries as follows. Understanding them would help me go ahead with the implementation in synch with the present framework.

> Authentication and the user management: I can see the three parameters being used for these: X-csrf token, session and client-ID. Can you just help me understand what is the role of each, which component creates it (browser extension or backened server) and a brief about how it is used in the whole framework. 

X-CSRF-Token
This is a standard cross site request forgery token. We abuse it a little bit by sending the token in responses to authentication and session requests, since cookie policies might prevent us from reading it from the cookie value.

Session is the session id.

X-Client-Id
This uniquely identifies this page view so that real time events don't echo back over the WebSocket to the client that caused them. It's not critical, just a performance hack.

> I can see a section for facebook and twitter in the annotation data. What is that exactly used for? (Just to understand what to do with it)

Not used for anything right now, except for fields that stand in for canonical URLs.

> If possible can you share with me complete data types of the parameters exchanged between the backend and the extension code for various API calls.

The core API is the same as Annotator's storage plugin, which is documented in h and Annotator and annotator-store.

The rest is to be considered unstable.

[Brandon Barker]

Hi All,

I'm also doing some work based on Raymond Yee's hypothesisapi module.

I seem to be getting 403 Forbidden for all of my attempts so far. Here is the current state of my python script, if you spot anything that might be off:

#!/usr/local/bin/python3.4

import os
import glob
import sys

import argparse
import json
import requests

from hypothesisapi import *
from rdflib import Graph, RDF
from rdflib.namespace import Namespace, FOAF

OA = Namespace('http://www.w3.org/ns/oa#')

parser = argparse.ArgumentParser()
parser.add_argument('-u', dest='user', action='store',
                    default='arXiv', help='hypothes.is user (default: arXiv)')

parser.add_argument('-p', dest='password', action='store',
                    help='hypothes.is password')

parser.add_argument('-d', dest='trackbacks_dir', action='store',
                    default='/data/shared/data/trackbacks',
                    help='directory with turtle (.ttl) trackback files;' +
                         'default=/data/shared/data/trackbacks')

args = parser.parse_args()


# Set up Hypothes.is config
H = API(args.user, args.password)
H.login()
#APP_URL = "https://hypothes.is/app"
API_URL = "https://hypothes.is/api/annotations"

#Read in trackbacks and POST them to hypothes.is:
for tfile in glob.glob(args.trackbacks_dir + "/*.ttl"):
    G = Graph()
    G.parse(tfile, format="n3")
    for trackback, _, article in G.triples( (None, OA.hasTarget, None) ):
        for _, _, comment in G.triples( (trackback, OA.hasBody, None) ):
            print(article)
            print(comment)
            payload = {
                "user" : "acct:%s...@hypothes.is" % args.user,
                "uri"  : article,
                "text" : comment,
                "permissions": {
                    "read"  : ["group:__world__"],
                    "update": ["acct:%s...@hypothes.is" % args.user],
                    "delete": ["acct:%s...@hypothes.is" % args.user],
                    "admin" : ["acct:%s...@hypothes.is" % args.user]
                    }
                }
            headers = {'Authorization': 'Bearer ' + H.token.decode("utf-8"),
                       'X-Annotator-Auth-Token' : H.token.decode("utf-8"),
                       'x-csrf-token' : H.csrf_token
                       }
            print(json.dumps(payload))
            r = requests.post(API_URL, headers=headers, data=payload)
            print(r.status_code)
            print(r.text)

Note: I'm aware I'm attempting to pass two forms of the authorization token here; not sure which is right/preferred, but all permutations thus far seem to not help.



Output:

</html>
http://arxiv.org/abs/0804.2273
http://en.wikipedia.org/wiki/Open_Archives_Initiative_Object_Reuse_and_Exchange
{"permissions": {"admin": ["acct:ar...@hypothes.is"], "update": ["acct:ar...@hypothes.is"], "read": ["group:__world__"], "delete": ["acct:ar...@hypothes.is"]}, "uri": "http://arxiv.org/abs/0804.2273", "user": "acct:ar...@hypothes.is", "text": "http://en.wikipedia.org/wiki/Open_Archives_Initiative_Object_Reuse_and_Exchange"}
403
<html>
 <head>
  <title>403 Forbidden</title>
 </head>
 <body>
  <h1>403 Forbidden</h1>
  Access was denied to this resource.<br/><br/>
Unauthorized: create failed permission check

[Apurva Jalit]

Hello All,

I have created a basic h-store backend in C# using ASP.NET MVC5 and SQL server as database. 

It currently lets a user perform basic user use cases (login, logout, register, change password) and annotation use cases (create notes, highlights, search on notes for existing highlight.

It does not have a complete website yet. Once I complete it and tie rest of the loose ends will update this thread.

Any reviews, suggestions, corrections are welcome!

Thanks,

Apurva Jalit

On Thu, May 28, 2015 at 2:00 PM, Apurva Jalit <apurva...@gmail.com> wrote:

Sorry for my previous mails. I found out the issue. 

Annotation ID was not set and hence the problem. I am sorry for sending out emails for these stupid issues!

Thanks,

Apurva Jalit

On Thu, May 28, 2015 at 1:37 PM, Apurva Jalit <apurva...@gmail.com> wrote:

I just noticed that the token generated by me has an invalid signature. Could this be the reason behind the problem? Any kind of validation on the client side for that? 

But I think I can safely assume that this should not be a problem since it is letting me create a note.  But just thought I would bring it to your notice in case this helps.

Thanks,

Apurva Jalit

On Thu, May 28, 2015 at 12:02 PM, Apurva Jalit <apurva...@gmail.com> wrote:

I am facing a problem when trying to edit a note created by the user who is currently logged in.

I am attaching with this mail all the XHR requests made the h chrome extension to my backend. As you can see, in search we have 1 note for the given page. It is created by the user "aps" and it is the same user currently logged in also. But that note on the sidebar is not providing me with an option to edit.

In the same context, can you clarify what does the 'acct:' prefix added to the username mean here? Am I abusing it and causing this issue here? What do we look for in a note to identify which user has created it so as to identify whether current user can be given an option to edit/delete the note? 

Thanks,

Apurva Jalit

[Apurva Jalit]

Forgot to add the link to repo :P.

You can find the repository at https://github.com/apurvajalit/h-store-.net

Thanks,

Apurva Jalit