Re: [api-craft] Re: Flow to handle POST async to external API

[mca]

your story is pretty long. can you condense it a bit?

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

On Fri, Mar 6, 2015 at 7:03 PM, Erik Azar <erik...@gmail.com> wrote:

I've not received a reply...has anybody had to deal with a scenario like this before?

On Friday, February 27, 2015 at 11:39:35 PM UTC-5, Erik Azar wrote:

I'm looking for feedback on a scenario I'm currently working through. It requires async call to 3rd party to either create a new resource or return an existing one, and handling the scenario where the resource already existed.  Sorry it's a bit long, but I wanted to make sure I explained it with as much detail as I could.

Lets say I have a service named Foo and a remote 3rd party named service named BarRemote.

TL;TR

Here is the flow I'm thinking of using:

1) client POSTs to Foo, resource doesn't exist locally, creates placeholder resource locally

2) Foo makes async call to BarRemote with resource POSTed then returns 202 and Location URI (123)

2) client GET short polls Location URI (123)

3) eventually the 3rd party responds with resource and metadata saying it already existed in their system, moves 123 to 456, replaces 123 with BarRemote resource

4) client calls GET (123) again, a 200 is returned with the resource and HATEOAS link to 456 resource

5) client sees the HATEOAS link and GETs (456)

6) client presents the two resources in a UI allowing the user to select which attributes to PUT on (123) from (456)

Detailed Version

I have a scenario where a client calls Foo via POST. The Foo service checks for an existing resource locally in a DB, if it's not found it is added locally and an async call is made to BarRemote passing the Foo resource from the client. A 202 response with Location header pointing to the new local resorce (https://api.acme.com/foo/123) is returned to the client.

Some time passes and a response is returned by BarRemote with a resource.  The resource already existed in their service so I need to replace the resource created via POST.

What I'm trying to work through is how to handle notifying the client the resource already existed (returning the 3rd party resource version to the client) and letting the client PUT or PATCH the current resource with attributes from the resource they originally POSTed.

One thought I had is:

When BarRemote returns a resource it replaces the resource at 123, the Foo resource is assigned a new ID (456), and metadata is added linking it to 123, the returned BarRemote resource is updated with metadata linking it to 456.

The client calls GET https://api.acme.com/foo/123, Foo looks at the resource and sees the metadata, it returns the resource with a 200 and a HATEAOS link to https://api.acme.com/foo/456.

The client can check if the HATEOAS link exists, if it does it could GET it and display a UI to allow the client to update (123) with specific attributes of (456).

Note: All data stored locally is only saved for a short period of time before it expires (~1-2 months).

Service Reviews

POST and min are provided we return 201.  If all parameters are include we submit it and 202. If the min are specified we return a 400 with missign required fields

PUT send more parameters but still not enought to submit we reutrn a 200. Add parameters but that causes other requirements for parameters we'll return 200 with errors collection.

PUT with all requirement parameters, we submit to payer and return a 202

PUT again, and request already existed in payer system how do we represent that?

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Jack Repenning]

On Mar 6, 2015, at 4:03 PM, Erik Azar <erik...@gmail.com> wrote:

I've not received a reply...has anybody had to deal with a scenario like this before?

"Had to deal with it," yes.

"Liked the solution we ended up with," decidedly not.

We did much like what you sketch. Two basic issues:

* Very complicated; we were publishing an "m2m" API (no human handy to select or intervene), and found this just too complicated for our partner/customer/client-implementers.

* Reliability is a problem as well: if the third party is down or inaccessible or slow, your whole app is down.

We grew to wish we'd done it another way:

1. client POSTs to Foo, resource unexists: create and return a limited-as-little-a-possible local resource

2. client requests may receive a "pending" response until the back ends get synced

3. meanwhile, "my server" has some sort of queue of pending resources and changes, and some sort of periodic attempt to poke 3rd party

4. eventually

    a. server and 3rd party get happy, client requests start working normally, or

    b. server and 3rd party can agree to create it, but have some disagreement about changes that have been allowed since then, or

    c. server and 3rd party reconnect, negotiate, and decide the pending resource should never have been allowed in the first place

If 4.a., everybody's happy.

If 4.b., the set of requests that return "pending" gets smaller  until someone corrects the denied transitions.

If 4.c., client is a sad puppy and has to start all over again.

Clients *may* poll for readiness (so you have to implement the polling function).

Risk of a 4.b. or 4.c. can be somewhat mitigated in the design of the server-to-3rd-party protocol (e.g., use GUIDs instead of database IDs as the primary object identifiers). With such 4.b/c events can be minimized to the point you can afford to fix them up with humanity rather than automation, and see vice calls rather than API calls.

-- 
Jack Repenning
Repenni...@gmail.com

[Petros Ziogas]

This looks overly complicated for an API consumer.

One thought I had that might simplify this a bit is to create a transient model (which in a way you already have) and use this as a connection with the other "real" model. Notice here that I am proposing a completely different model.

e.g. using order as the final object

- client posts to /order-creation-request gets back an object with status 'PENDING'

- client can poll /order-creation-request as many times as he wants and check on the status field. If order is created then the object returned can contain an order_id or url

- the order-creation-request object is not deleted, it keeps reporting the results of the request.

In that process you can differentiate the data to also make a distinction between new orders or orders that were already there.

Petros