API listing not including create API on local h instance

[son...@oecg.no]

Hi,

I see that 

https://hypothes.is/api/

is listing 

links: 

{

• annotation: 

  {

  • create: 

    {

    • method: "POST",
    • url: "https://hypothes.is/api/annotations",
    • desc: "Create an annotation"

    },

  
  
• (even when not logged in - my first thought was that it might be permission-related, but seems not?), while my own similar URL on a self-hosted h instance[1] in dev mode does not. Any ideas as to why? It lists the other annotation actions. In the real use case, I am using JWT tokens and it works fine after some tweaking (basically creating users hypo side on demand - which I can see postgres side), but I suspect the missing call to actually store annotations is due to the create call not being in our local /api/ listing of available APIs? Any ideas or pointers?

And if this guess at why the client is not trying to actually store annotations is way off, I am grateful for ideas for where to debug as well.

Thanks a lot,

Sondre

[1] - Basically "make dev" in h and client after setting CLIENT_OAUTH_ID, CLIENT_URL, APP_URL, SIDEBAR_APP_URL, PACKAGE_SERVER_HOSTNAME and the mentioned code tweaks for user creation on demand (not very pretty for now...). I also had to do a hack in devserver.py, as it was not too happy about being on https, but with https terminated in front of it - so while the dev server is http, the URLs will be https - but I think I sorted that one out based on the flow of the jwt grant token stuff now working.

PS. Is wss protocol required for this use case? I did some magic in haproxy to be able to do the "Connection: Upgrade" etc. but do not see connections from the hypo client, but I am able to make a dummy secure websockets connection from the js console to the server, and seeing some log lines when I do so, so seems the port mapping etc. is fine.

[Robert Knight]

Hello,

The output of `/api/` does not take the user’s authorization into account. It always lists all available API calls. It is generated automatically from `@api_config` decorators in the client. The “Create an annotation” entry is created by https://github.com/hypothesis/h/blob/d05c7e81b265607ce90a753081d7c1c29fac6e74/h/views/api/annotations.py#L67 for example. Did you modify that code at all?

The client uses the output of `/api/` to map the action it wants to perform to the HTTP endpoint (method and URL). If this information is missing, I would expect that to cause an error when the client attempts to create an annotation.

Regards,

Robert

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@list.hypothes.is.
To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/a4170751-9750-4197-81d3-7515dc8a12eb%40list.hypothes.is.

[son...@oecg.no]

Nope, did not touch that file. All the other calls end up in the listing even on my server. I noticed the annotation in questions had a permission:

@api_config(
    versions=["v1", "v2"],
    route_name="api.annotations",
    request_method="POST",
    permission="create",
    link_name="annotation.create",
    description="Create an annotation",
)

But as you say that URL lists all the calls even when not logged in on the official hypo backend.

What is the actual code doing the "reflection"/annotation magic to return the /api/ JSON? Maybe I can add a bit of debugging to find out what's going on on my server.

Thanks a lot for getting back to me,

Sondre

To unsubscribe from this group and stop receiving emails from it, send an email to d...@list.hypothes.is.

[Robert Knight]

The `/api/` route is handled by the `index` function here: https://github.com/hypothesis/h/blob/d05c7e81b265607ce90a753081d7c1c29fac6e74/h/views/api/index.py#L9. This uses metadata populated by functions in `h/views/api/helpers/links.py`, which in turn are triggered by calls to the `api_config` function in `h/views/api/config.py`.

When the server starts up a configuration “registry” object is created with an `api_links` sub-object which is populated by calls to `api_config`. The `/api/` endpoint then produces a JSON representation of the `api_links` object.

Kind Regards,

Robert

To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@list.hypothes.is.
To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/77cb925f-be5a-4a84-931b-aa58e4679ddf%40list.hypothes.is.

[son...@oecg.no]

Thanks again for responding and helping out so quickly.

Added a bit of logging and the metadata registration seems very similar to what happens for an entry that does end up in my server's /api response:

$ cat hypo_bootstrap_20191028.log |grep "14:25"|grep "annotation.create"

14:25:51 worker | api_config called for link_name annotation.create

14:25:51 worker | add_api_view for annotation.create

14:25:51 worker | register_link annotation.create

14:25:51 web    | api_config called for link_name annotation.create

14:25:51 web    | add_api_view for annotation.create

14:25:51 web    | register_link annotation.create

$ cat hypo_bootstrap_20191028.log |grep "14:25"|grep "annotation.delete"

14:25:51 worker | api_config called for link_name annotation.delete

14:25:51 worker | add_api_view for annotation.delete

14:25:51 worker | register_link annotation.delete

14:25:51 web    | api_config called for link_name annotation.delete

14:25:51 web    | add_api_view for annotation.delete

14:25:51 web    | register_link annotation.delete

The delete is shown similarly to what I see on https://hypothes.is/api/ (as is read, update, flag, hide, unhide), but the create is nowhere to be seen in the JSON. Any ideas where to look next?

I did a check on my local code vs. the repos, and I see I have this little hack in format_nested_links:

Maybe I need some failsafe here (check on None or something). Will dig a bit more here.

Btw, is there any exception type logfile when I run "make dev" or is the std output the only place to look?

Cheers,

-S-

Robert

To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/77cb925f-be5a-4a84-931b-aa58e4679ddf%40list.hypothes.is.

[son...@oecg.no]

Now I got the create link in the API listings! Thanks a lot. I will try to backtrack to see exactly which change made it, maybe it was the None-check on url replace (again, the combo of actually running over https, but not as seen from python which I found no way to control via config/env vars), now done via a tiny local function inside format_nested_links.

[Robert Knight]

Exceptions will appear in `make dev` output. All JSON API outputs come from the “web” process, so the “worker” output is irrelevant here.

If you have made local code changes, can you post a diff of the code changes in your local copy of h vs the `master` branch that you forked from?

On 28 Oct 2019, at 13:39, son...@oecg.no wrote:

Thanks again for responding and helping out so quickly.

Added a bit of logging and the metadata registration seems very similar to what happens for an entry that does end up in my server's /api response:

$ cat hypo_bootstrap_20191028.log |grep "14:25"|grep "annotation.create"

14:25:51 worker | api_config called for link_name annotation.create

14:25:51 worker | add_api_view for annotation.create

14:25:51 worker | register_link annotation.create

14:25:51 web    | api_config called for link_name annotation.create

14:25:51 web    | add_api_view for annotation.create

14:25:51 web    | register_link annotation.create

$ cat hypo_bootstrap_20191028.log |grep "14:25"|grep "annotation.delete"

14:25:51 worker | api_config called for link_name annotation.delete

14:25:51 worker | add_api_view for annotation.delete

14:25:51 worker | register_link annotation.delete

14:25:51 web    | api_config called for link_name annotation.delete

14:25:51 web    | add_api_view for annotation.delete

14:25:51 web    | register_link annotation.delete

The delete is shown similarly to what I see on https://hypothes.is/api/ (as is read, update, flag, hide, unhide), but the create is nowhere to be seen in the JSON. Any ideas where to look next?

I did a check on my local code vs. the repos, and I see I have this little hack in format_nested_links:

To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@list.hypothes.is.
To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/d8e9cc0d-1b7e-4d5f-ab8f-59acfdfe0fa9%40list.hypothes.is.
<diff_format_links.png>

[son...@oecg.no]

As mentioned (we wrote about the same time, I guess) this is working now. I think it was the failsafe on my https replacement hack that fixed my self-introduced bug.

The client now complains about no groups, but that is a completely separate thing and maybe related to params to the call to signup (that we do behind the scenes if we have not seen the user before).

self.annotation.group = self.annotation.group || groups.focused().id;

(here groups.focused is undefined for us)

I see that on the main hypo backend, this is returned as [{"name": "Public", "id": "__world__", "public": true}] for a fresh user.

Cheers,

-S-

To view this discussion on the web visit https://groups.google.com/a/list.hypothes.is/d/msgid/dev/d8e9cc0d-1b7e-4d5f-ab8f-59acfdfe0fa9%40list.hypothes.is.
<diff_format_links.png>

[son...@oecg.no]

Sorry for the slight spamming, but just in case someone comes across this thread googling for the group stuff: 

This was very likely caused by running "make services" before the correct default authority was set up. I fixed manually in postgres via "make sql" and then 

update "group" set authority = 'myauthority' where id = 1;

The authority here was "localhost" by default.