RFC: Angular 2 HTTP Implementation (prototype)

[Jeff Cross]

I've been working on a new design for managing HTTP connections in Angular 2 along with Ben Lesh and Caitlin Potter. https://github.com/jeffbcross/http-design

I'd love your feedback on the prototype!

Take a look at the Readme to see some of the philosophy and approach (actual implementation has changed some).

Then take a look at the tests and code of the prototype to get a better feel for how it's implemented. Note: APIs and names are subject to change, I'm mostly interested in how folks feel about the semantics and ideas. There are only 18 tests right now, so it's not too much reading to get an idea how it works.

You can run the typescript compiler and tests if you want to try it out.

$ cd prototype

$ npm i

$ ./ts-build.sh # will watch files and auto-compile on changes

$ karma karma.conf.js

Some noteworthy concepts

Observables

The library is completely observable-based, and is implemented with RxJS. This gives us a few things.

1. Ability to observe data streams like downloads and uploads, as well as state changes of a connection.
2. Simple and flexible retrying of connections that error
3. A powerful means of transforming request and response data
4. Performance benefits like cold observables and cancelable connections

Immutability

There are also portions of the design that prevent accidental global state, such as having an immutable base connection config. The only way to set global defaults is to copy the base config and replace the binding with dependency injection during bootstrap (or from some parent component).

In-band testing

Instead of declaring http request expectations before hand, this library favors mocking responses within tests after a request has been made. I'm in favor of adding auto-responders for convenience, just haven't implemented it yet.

I'd like to have general discussion in this group, but feel free to open issues on the repository as well.

Thanks!

[Sander Elias]

Hi Jeff,


Would you prefer to get my questions and/or remarks here, or in the github repo?

As you asked here, here are my first observations.

• Http
  
• HttpJsonp
  
• HttpNode
  
• HttpMock
  
• HttpFetch TBD

I get all of those, except HttpNode. Why a different handler for a node server? Angular should be server agnostic right? Or is it some other kind of node?

Also websockets seem to be missing. One could argue that's not an http-handler, but still.

I like the fact that observable's are there! that is a really good addition. It will make a lot of code a whole lot simpler! In fact, there are already some really nice communication handlers available for RX already (dom.ajax dom.fromWebSocket, dom.fromWebWorker to name a few) Is the planning to incorporate those? 

with kind regards

Sander

[Jeff Cross]

Thanks for the reply, Sander! Comments inline.

On Wednesday, April 1, 2015 at 12:28:41 AM UTC-7, Sander Elias wrote:

Hi Jeff,


Would you prefer to get my questions and/or remarks here, or in the github repo?

General remarks are best here, issues can be created in Github repo is something needs to be tracked.

 

As you asked here, here are my first observations.

• Http
  
• HttpJsonp
  
• HttpNode
  
• HttpMock
  
• HttpFetch TBD

I get all of those, except HttpNode. Why a different handler for a node server? Angular should be server agnostic right? Or is it some other kind of node?

I'm not sure what you mean by "server agnostic." The HttpNode library would be the same interface as Http, but used in a nodejs environment, using nodejs http client. Tobias is exploring server-side rendering of Angular applications, which may include executing angular code on a server. In this context, an application could use dependency injection to bind HttpNode (or some other arbitrary Http library) to Http, so all calls to Http would actually be made to HttpNode.

The Http libraries will also be usable outside the context of Angular. Http should provide nice testing with DI, and nice ergonomics otherwise.

 

Also websockets seem to be missing. One could argue that's not an http-handler, but still.

Yeah, a WebSocket implementation is on the radar, and will be observable-based. But it's a completely different beast. I expect within a few weeks I'll post something here about WebSocket implementation.

 

I like the fact that observable's are there! that is a really good addition. It will make a lot of code a whole lot simpler! In fact, there are already some really nice communication handlers available for RX already (dom.ajax dom.fromWebSocket, dom.fromWebWorker to name a few) Is the planning to incorporate those? 

Glad you're on board with Observables!

In terms of incorporating those directly in core, it's not likely, but the core should make it simple for developers to incorporate those libraries into their applications. Is there a feature from dom.ajax that you think is worth considering implementing?

 

with kind regards

Sander

[Rob Wormald]

Hey Jeff,

My only comment would be the suggestion of standardizing around the what-wg fetch spec. We've been using the polyfill from GitHub for a while with DI.js, and we've been very happy with it.

To use angular 1.x terms, instead of the $httpBackend exposing function(method, url, post, etc....) - it would expose fetch(url,options){ ... 

At that point, the '$http' 2.0 module would expose the GET/POST/PUT/DELETE interface, and delegate to whatever DI injected fetch-api-conformant backend.  

Favoring fetch means a) we get the native implementation when available, b) intercepting fetch events in service workers seems to dovetail nicely with caching, offline use, etc, which seem to be a theme of ng2Data, c) its super-duper easy to mock. 

It also makes translating into non-http implementations a lot easier (and likely more interoperable) - for example, I have a SailsJS client that exposes the same GPPD 'public' interface, and then swaps out the backend for socket.io - it's just a matter of translating the fetch(url,options). Ditto for https://www.npmjs.com/package/node-fetch, https://github.com/matthew-andrews/isomorphic-fetch. React Native also provides this as the core network API. 

The native implementation in Chrome uses Streams - and my understanding is Streams and Observables are quite closely related?

I'm sure you know all this already, but a couple of good primers for readers: https://github.com/slightlyoff/ServiceWorker/blob/master/explainer.md#network-intercepting / http://updates.html5rocks.com/2015/03/introduction-to-fetch

Apologies if this misses the point, but wanted to mention it and our experience with using it. 

Cheers!

Rob

[Sander Elias]

Hi Jeff,

I inlined too.

General remarks are best here, issues can be created in Github repo is something needs to be tracked.

Good, so conversation like this goes here, actionable items go into github!

 As you asked here, here are my first observations.

• Http
  
• HttpJsonp
  
• HttpNode
  
• HttpMock
  
• HttpFetch TBD

I get all of those, except HttpNode. Why a different handler for a node server? Angular should be server agnostic right? Or is it some other kind of node?

I'm not sure what you mean by "server agnostic." The HttpNode library would be the same interface as Http, but used in a nodejs environment, using nodejs http client. Tobias is exploring server-side rendering of Angular applications, which may include executing angular code on a server. In this context, an application could use dependency injection to bind HttpNode (or some other arbitrary Http library) to Http, so all calls to Http would actually be made to HttpNode.

Alright, now it makes sense to me, I forgot that NG2 does also target the server. That’s really really good, I’ll come back on this in a little while ;)

The Http libraries will also be usable outside the context of Angular. Http should provide nice testing with DI, and nice ergonomics otherwise.

That is a solid base to build on. It also makes it easier to swap out the communications layer. All wins.

 

Also websockets seem to be missing. One could argue that's not an http-handler, but still.

Yeah, a WebSocket implementation is on the radar, and will be observable-based. But it's a completely different beast. I expect within a few weeks I'll post something here about WebSocket implementation.

Good to hear. However, this sounds as the other handlers will work without observables? I think it makes sense to base all communication on observers. That way its way easier to change any part in the system, and also it makes composing the communication stack much easier.

I like the fact that observable's are there! that is a really good addition. It will make a lot of code a whole lot simpler! In fact, there are already some really nice communication handlers available for RX already (dom.ajax dom.fromWebSocket, dom.fromWebWorker to name a few) Is the planning to incorporate those? 

Glad you're on board with Observables!

This makes us a happy bunch :)

On a serious note, If the data-layer is going to embrace observables, and there is going to be a httpNode interface, it does make a lot of sense to be able to send and receive observeables themselves from client to server. That way communication with the backend will be a seamless part off the application. This is kind of the lower layer of what netflix falcor does. (I know that does a bit more, but it starts with this!)
It will then be very easy to migrate application logic between the server and the client.

I can get deeper into this, but it would help if I know your experience with observables.
Let me give you an example with some pseudo code:

//client:
    Observable.fromEvent(submitButton,"click")
        .map(pullDataFromForm)     // pull the data into the observable
        .filter(allDataValid)      // function that checks all form data
        .toServer('endpointname')  // <-- your part ;)
        .subscribe(result => alert('data saved on server!'), oops => alert('uhoh?'))

//server:
    Observable.createEndpoint('endpointname') // <-- again, your part!
        .filter(allDataValid)                 // server needs to validate too right, why not reuse?
        .respond(saveDatatoDbAndAnswerClient) // partly yours, a subscribe method that can respond back to the client.

The kicker, the server can be in the backend, or in the frontend! How do you think about a system like that?

In terms of incorporating those directly in core, it's not likely, but the core should make it simple for developers to incorporate those libraries into their applications. Is there a feature from dom.ajax that you think is worth considering implementing?

If you build the idea I just proposed, I could not care less ;) But just in case I didn’t convince you, I would like to get an observable back from all handlers that has those properties:

For a successful operation, the result will contains the following:

• response - (Object): The response from the XmlHTTPRequest/websocket/fetch/carrier-pigeon. Parsed into JSON if the responseType set.
• status - (Number): The status code.
• responseType - (String): The Response type.
• xhr - (Request): The Request from the request.
• originalEvent - (Object): The original event from the callback handler.

For a failed operation, the result will contain the following:

• type - (String): The type of rejection. This will be either ‘error’ or ‘abort’.
• status - (Number): The status code.
• xhr - (Request): The Request from the orginating request.
• originalEvent - (Object): The original event from the callback handler.

This might look familiar to you if you browsed trough the RX documentation ;)

Regards
Sander

​

[Sander Elias]

Hi Rob,

I do like fetch, and it’s an huge improvement over XMLHttpRequest. But as an interface it is almost equal to $http from angular 1, with as addition it can carry more kinds of data beside JSON.
Shouldn’t we do better in NG2?

Observables bring just that to the table. They will make it possible to abstract any kind off communications into a single interface. Interceptors will disappear. You don’t need those any longer.
an example:

  let comm = observable.fromWebSocket('someUrl') //pretend that this is a kind of stock ticker

  comm
     .filter(userHasIntersest)  // a function to checks if this msg is important to me
     .filter(checkIfNeeded)     // is this a msg that is of interst to this routine
     .map( msg =>  msg.currentValue) // I just need the value, so I'm mapping that out
     .subscribe(updateScreenValue) // a function that takes an value, and puts it on a display somewhere

  comm
     .filter(needUserData)   // function that checks if the other side needs some extra data
     .subscribe(question => {
          showModal(question).then( answer => {
             comm.onNext({id:question.id,answer:answer})
          }
      })

This is not even pseudo code, if you fill in the missing functions, this will actually work already!
You can imagine that if you switch to another form off communication, that re-factoring this, is not too much work.
Sure, you have to create the observables that handles the communication, but the code stay’s exactly the same.
in stead of websockets, you can use longpolling or streams or whatever bidirectional communication you fancy . makes no difference to this code.

Regards
Sander

​

[Alex R]

Hi Jeff,

One concern I don't see directly addressed in the spec/tests is the need for application-level functionality: request logging, performance monitoring, common error handling, etc.

I believe an application can supply a custom ConnectionConfig to achieve most of this, but would like to see this functionality represented in the tests.

--
You received this message because you are subscribed to the Google Groups "angular-data-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angular-data-d...@googlegroups.com.
To post to this group, send email to angular-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/cb7442b0-c25c-47d8-8485-2b57dc4f62ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[inglor]

The fetch API already uses whatwg streams (observables) so this feature you're asking for is already in both for writing requests and reading responses :)

[inglor]

This design looks very nice in first sight - it's also very similar to what the whatwg people are working on in fetch. 

[Jeff Cross]

Hey Rob,

I think the fetch spec is great to look at for a few reasons. I particularly like that it standardizes Request and Response interfaces, and I think the Angular implementation should adhere to these (it currently does not). 

I have to admit that I've been following the ServiceWorker spec for some time, and fetch to some degree, but haven't really played with fetch much. In terms of answering the question "should angular http base off of its semantics completely?" I think Observables provide a lot of built-in value over Promises for something as "dynamic" as http requests, particularly for retrying & canceling. I'll take some time to get more familiar with it and thinking about how it could inspire Angular's http more.

[Jeff Cross]

On Wednesday, April 1, 2015 at 9:44:22 PM UTC-7, Sander Elias wrote:

Hi Jeff,

I inlined too.

General remarks are best here, issues can be created in Github repo is something needs to be tracked.

Good, so conversation like this goes here, actionable items go into github!

Yep 

 As you asked here, here are my first observations.

• Http
  
• HttpJsonp
  
• HttpNode
  
• HttpMock
  
• HttpFetch TBD

I get all of those, except HttpNode. Why a different handler for a node server? Angular should be server agnostic right? Or is it some other kind of node?

I'm not sure what you mean by "server agnostic." The HttpNode library would be the same interface as Http, but used in a nodejs environment, using nodejs http client. Tobias is exploring server-side rendering of Angular applications, which may include executing angular code on a server. In this context, an application could use dependency injection to bind HttpNode (or some other arbitrary Http library) to Http, so all calls to Http would actually be made to HttpNode.

Alright, now it makes sense to me, I forgot that NG2 does also target the server. That’s really really good, I’ll come back on this in a little while ;)

The Http libraries will also be usable outside the context of Angular. Http should provide nice testing with DI, and nice ergonomics otherwise.

That is a solid base to build on. It also makes it easier to swap out the communications layer. All wins.

 

Also websockets seem to be missing. One could argue that's not an http-handler, but still.

Yeah, a WebSocket implementation is on the radar, and will be observable-based. But it's a completely different beast. I expect within a few weeks I'll post something here about WebSocket implementation.

Good to hear. However, this sounds as the other handlers will work without observables? I think it makes sense to base all communication on observers. That way its way easier to change any part in the system, and also it makes composing the communication stack much easier.

Observables will be pervasive in Angular 2 Data, and Angular 2 in general. Though there are still some core promise-based APIs, like in the compiler.  As libraries are developed for WebSockets, browser storage, etc, observables will be the main wrapper for data. Is this what you were referring to by "other handlers"?

I like the fact that observable's are there! that is a really good addition. It will make a lot of code a whole lot simpler! In fact, there are already some really nice communication handlers available for RX already (dom.ajax dom.fromWebSocket, dom.fromWebWorker to name a few) Is the planning to incorporate those? 

Glad you're on board with Observables!

This makes us a happy bunch :)

On a serious note, If the data-layer is going to embrace observables, and there is going to be a httpNode interface, it does make a lot of sense to be able to send and receive observeables themselves from client to server. That way communication with the backend will be a seamless part off the application. This is kind of the lower layer of what netflix falcor does. (I know that does a bit more, but it starts with this!)
It will then be very easy to migrate application logic between the server and the client.

I can get deeper into this, but it would help if I know your experience with observables.
Let me give you an example with some pseudo code:

//client:
    Observable.fromEvent(submitButton,"click")
        .map(pullDataFromForm)     // pull the data into the observable
        .filter(allDataValid)      // function that checks all form data
        .toServer('endpointname')  // <-- your part ;)
        .subscribe(result => alert('data saved on server!'), oops => alert('uhoh?'))

//server:
    Observable.createEndpoint('endpointname') // <-- again, your part!
        .filter(allDataValid)                 // server needs to validate too right, why not reuse?
        .respond(saveDatatoDbAndAnswerClient) // partly yours, a subscribe method that can respond back to the client.

The kicker, the server can be in the backend, or in the frontend! How do you think about a system like that?

Well I think there's no reason why code couldn't be shared between client and server in this context. It's outside the scope of the http library to facilitate this. This library is pretty low-level, not meant to be too smart. And though we're exploring node support as a reference for server-side rendering, we aren't necessarily looking to standardize on any particular server as we explore the Angular 2 Data design.

[Caitlin Potter]

Just a quick note on this,

The Observables idea doesn’t seem to have floated around much in fetch-land, where Stream interfaces seem to be preferred.

Streams are cool, but potentially very difficult to polyfill — and might be slightly more complicated to use than Observables*.

It’s hard to pick which horse to go with, but the Observable* abstraction seems good to me

--
You received this message because you are subscribed to the Google Groups "angular-data-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angular-data-d...@googlegroups.com.
To post to this group, send email to angular-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/9ae1b89b-805f-487a-ba19-e4223ec5597a%40googlegroups.com.

[Jeffrey Cross]

Yeah, to be clear, I'm in no hurry to ditch observables :).

To unsubscribe from this group and stop receiving emails from it, send an email to angular-data-dev+unsubscribe@googlegroups.com.
To post to this group, send email to angular-data-dev@googlegroups.com.

[Rob Wormald]

Good point. If we are considering node in this discussion, Streams are very much the core primitive in Node for i/o though, and something a fair portion of front end devs are familiar with from Gulp (even if they don't know it!)

I think there's a lot of power in Observables, but Promises and Streams are the two primitives that are widely supported today. Promises specifically form the building blocks for es7 stuff like async / await, and it seems like observables are easy to plug in downstream from them. 

Came across this discussion while researching  https://github.com/kriskowal/gtor/issues/6 which has some good info, specifically with iterators / generators / iterables in mind. 

See also https://gist.github.com/josh/21c49a052370e92aa9d5 for an Interop with what-wg streams. 

Rob

Sent from my iPhone

To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/20BE66C1-D3E0-4449-9A96-B5071C0D73C8%40gmail.com.

[Rob Wormald]

See also http://jakearchibald.com/2015/thats-so-fetch/ which talks about Streams in relation to fetch. 

Sent from my iPhone

--

[Sander Elias]

Hi Inglor,

I was ignoring streams on purpose and hoped (I should have known better!) that nobody would notice :)

Streams are like observables, but they are not the same thing. Observables are much nicer to build interfaces, and have much more potential to abstract the low-level stuff away. Also, you can't wrap everything in a (useful) stream, while you can do that with an observable.

If you get into observables, you will notice that Observables stand to IO/events/pub-sub and streams as Angular stands to jquery. For a little while you will have a hammer, and everything will look like a nail :)

Regards

Sander

[Sander Elias]

Hi Alx,

Sorry to butt in on your question. However, if all interfaces utilize observables, there is build in support for all of those.

Let me show you with some code:

```

   var loggedAndMeteredhttp = http2Observable({'url:'goGetStuff'})

      .map( (x) => { log(x); return x;})  //log all requests

      .map( (x) => { log(seconds()-x.startSeconds);return x}) //log performance data 

    loggedAndMeteredhttp 

      .subscribe( fnHandleNextData, fnTerminatedWithError, fnDoneWithoutError)

```

You can compose observables in any configuration you want. When you get the hang of it, it will become clear to you.

Regards
Sander

[Sander Elias]

Hi Rob,

I'm pretty sure one can find an exception, but as far as I'm aware of now, you can wrap events/promises/streams easily into observables, but not (in a useful way) the other way around. This makes it a pretty powerful abstraction level for all async and sync data flows in your app's (a data flow is any kind of [data|event|interaction|translation] coming from anywhere, including server/user/p2p/me)

Did you try playing with rxjs already?

Regards

Sander

[Sander Elias]

Observables will be pervasive in Angular 2 Data, and Angular 2 in general. Though there are still some core promise-based APIs, like in the compiler.  As libraries are developed for WebSockets, browser storage, etc, observables will be the main wrapper for data. Is this what you were referring to by "other handlers"?

Well, the xmlhttprequest/fetch handlers. But I can rest assured that those will be wrapped in an observable too?

 

//client:
    Observable.fromEvent(submitButton,"click")
        .map(pullDataFromForm)     // pull the data into the observable
        .filter(allDataValid)      // function that checks all form data
        .toServer('endpointname')  // <-- your part ;)
        .subscribe(result => alert('data saved on server!'), oops => alert('uhoh?'))

//server:
    Observable.createEndpoint('endpointname') // <-- again, your part!
        .filter(allDataValid)                 // server needs to validate too right, why not reuse?
        .respond(saveDatatoDbAndAnswerClient) // partly yours, a subscribe method that can respond back to the client.

The kicker, the server can be in the backend, or in the frontend! How do you think about a system like that?

Well I think there's no reason why code couldn't be shared between client and server in this context. It's outside the scope of the http library to facilitate this. This library is pretty low-level, not meant to be too smart.

That is good, very good. But keep this scenario in mind, and make sure another library(yours or 3rth party) can extend it in this way. (I'm thinking of thinks like Jafar's falcor)

 

And though we're exploring node support as a reference for server-side rendering, we aren't necessarily looking to standardize on any particular server as we explore the Angular 2 Data design.

That's what server agnostic means. I would not expect less ;)

Regards
Sander

[Jeff Cross]

For the sake of this thread, we can consider Observables the decided upon interface for values in http. If others in the group feel it's important to discuss the right abstraction for async values and sequences of values, feel free to open a separate topic with use cases where observables fall short. I'd be happy to elaborate in another thread why we felt that Observable is generally the right abstraction for this type of work.

[Jeff Cross]

Alex Rickabaugh and I had a meeting today where we outlined some more real-world use cases that should be possible and simple with the http library. Use cases such as:

1. Simple in-memory caching of a response for a given url
2. Performance monitoring, such as logging server time to fulfill a request
3. Deduplication (if many components request the same url at the same time, coalesce into a single request)
4. Multipart requests
5. Configurable retry/backoff
6. Manual retry, ie prompt user to retry
7. Streaming responses (as well as streaming upload progress)
8. Global error handling

The library as is makes some of these difficult, since transformations on requests and responses only take either a request or response observable as their input. For something like caching, you want to be able to see both at the same time. Alex is exploring creating a lower-level interceptor function that could be attached to a connection config and takes a request and delegate function as its arguments. He can elaborate more on the design.

After the meeting, I started thinking that maybe we should focus on keeping the http library simpler, and focus on applications creating re-usable services based on http which do the things applications want. The http library could ship with some helpers as needed. I.e. if I want to always give a connection three chances to succeed, I could create a shared service like this:

export function threeStrikes (req:Request|string) {

  return this.http(req).retry(3);

}

@Inject(threeStrikes);

export class MyComponent {

  constructor(threeStrikes) {

    threeStrikes('http://foo.bar').subscribe(onNext, onError);

  }

}

I could compose that with another service that adds auth headers to every request.

export function authHeadAdder (req) {

  let newRequest = req.addHeader('auth-token', authService.token());

  return newRequest;

}

@Inject(threeStrikes, authHeadAdder))

export class MyComponent {

  constructor(threeStrikes, authHeadAdder) {

    authHeadAdder(req).map(threeStrikes).subscribe(onNext, onError);

  }

}

There could even be a service that would take a config object instead of using Observable operators.

export function declarativeHttp (config) {

  let responseObservable = Rx.Observable.create(() => { /*do stuff*/});

  if (config.responseInterceptors === true) {

    config.responseInterceptors.forEach(i => { responseObservable = responseObservable.map(i) }

  }

}

The benefit of reducing the scope of the http() call is that it reduces the complexity and burden of the http library, and delegates common operations to the observable implementation. This makes it easier to build and maintain the different variations of http (xhr, fetch, node, jsonp, etc), and to maintain them. But does this create an overall better developer experience? I think it's a little bit more expressive than attaching a transformer to a config and trusting that the library will be called at the right time. And an application's services can be shared just as easily as a shared configs.

Anyways, it's getting late, and I've got a head cold, so perhaps these are bad ideas.

Thoughts? Ben Lesh?? You've been quiet in all this discussion about observables.

[Sander Elias]

Hi Jeff,

The benefit of reducing the scope of the http() call is that it reduces the complexity and burden of the http library, and delegates common operations to the observable implementation. This makes it easier to build and maintain the different variations of http (xhr, fetch, node, jsonp, etc), and to maintain them. But does this create an overall better developer experience? I think it's a little bit more expressive than attaching a transformer to a config and trusting that the library will be called at the right time. And an application's services can be shared just as easily as a shared configs.

Thoughts? Ben Lesh?? You've been quiet in all this discussion about observables.

Yes! This is what I tried to communicate all along. users of the http2 observables can use composition to [add|substract|modify]  whatever they need, and exactly on the place/time they need it. With a couple of samples like the one you just gave, it becomes easier to pick up.

All the 8 points you mentioned can be solved in this way.

Make sure your base operator provides all the data that's needed, but nothing more. The source of rx.dom.ajax might provide some sensible defaults here. 

Regards

Sander

[jhusain]

EDIT: Sorry comments below were made before I was fully caught-up on thread. Seems the decision has been made to use Observable, which was my primary concern.

I don't think we should use fetch. The problem is that fetch does not support cancellation. This is a pretty big hole in functionality, and it is what spawned this epic issue thread on GitHub. https://github.com/whatwg/fetch/issues/27

I'm all for using standards, But only when they are well-designed. Given fetch's limitations, I don't see much upside in using it. Platform support doesn't buy us very much. It's a very thin wrapper around XMLHttpRequest that you can polyfill a few hundred lines. In contrast using observables everywhere will give us valuable cancellation semantics, and seamless composition between events, async requests, and animations. I think this is a better story for angular developers.

On Wednesday, April 1, 2015 at 5:52:39 PM UTC-7, Rob Wormald wrote:

[Benjamin Gruenbaum]

After your final comments on that thread I tend to agree - it doesn't look too bright.

I've spent a few hours with ReadableStream and writing pseudo code with fetch and I feel like I'm enjoying Observables much better (push feels more natural than pull+promises)

I'm still in favor of an API that has multiple abstractions rather than a hammer. I'd love to see promises tasks and observables all together for their respective scenarios - single subscriber with cancellation, broadcast single value+time and push iteration. I think that modeling everything through observables is risky and constraining the API users consume at points is super important.

I'll do my best to play devil's advocate here against observables (although I'm an Rx fanboy at heart) - I think the baseline Jeff Ben and Caitlin wrote is a good start.

On Apr 3, 2015, at 17:34, jhusain <jhu...@netflix.com> wrote:

I don't think we should use fetch. The problem is that fetch does not support cancellation. This is a pretty big hole in functionality, and it is what spawned this epic issue thread on GitHub. https://github.com/whatwg/fetch/issues/27#issuecomment-88980014

--

You received this message because you are subscribed to the Google Groups "angular-data-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angular-data-d...@googlegroups.com.
To post to this group, send email to angular-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/fb6cfdca-8e00-42ac-ae3c-c4df70f13d2c%40googlegroups.com.

[Rob Wormald]

I agree with this feeling risky, but I suppose nothing moves forward without a bit of risk.

I'm all for Observables and any similar awesome things - but not at the expense of ease-of-use and compatibility (not saying Observables aren't, but I'd encourage all of you to spend some time in #angularjs and see what it's like down here in the trenches with new Angular users...). Google's own Javascript SDK's now return promises (finally!) and I'd really like to avoid another generation of wrapping everything (like we've just finished doing for callbacks...)

Jeff mentioned he likes that fetch is standardizing the Request/Response types, and I think if we could make that work, with the ability to have some flexibility on what the "next step" in the chain is (Observable/Stream/Promise/Task) that would be ideal. It certainly would be nice to be able to cancel the occasional big upload, but given the choice, 99% of the time I'd opt for thenables simply because the resolve/reject paradigm frankly works great today.

Given that Observables will be the thing, it would be great to know how I'd accomplish something like the ServiceWorker/fetch method, where I can intercept *all* outgoing requests leaving the UI with a Worker and do with them as I please:

UI:

window.fetch('someurl.json').then(...

self.addEventListener('fetch', function(event) {
  event.respondWith( //check cache, or use indexedDB, or actually make request to backend...

I guess my question is, what does the interoperability story look like? 

I promise (ha) my next email will be less grumpy and have more interesting things to talk about :D 

Rob

To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/71A6E904-0BC9-4358-BE0C-F332CD77F5DC%40gmail.com.

[ben]

Ease of use shouldn't at all suffer. The contract difference between a Promise and an Observable is really one method and a name:

         

// basic use
myPromise.then(doSomething, failureCallback);
myObservable.subscribe(doSomething, failureCallback, doneCallback);

// error handling
myPromise.catch(err => Promise.resolve('continue')).then(x => console.log(x));
myObservable.catch(err => Observable.return('continue')).subscribe(x => console.log(x));

The major difference is that Promises can't represent the majority of async use cases in web development, and can't be retried or repeated. Promises can't be cancelled. They're just read-only views to a future.

Think of all of the async things you do in web development (here are a few examples):

- HTTP (Ajax/JSONP)

- Sockets

- Animations

- DOM events

- Workers

Of those, the only one that is suited well to promises is HTTP. Because it's the only async item in that list that deals with just one value. All of the others are going to emit multiple values.

Could you use Promises for, say, DOM events? Sure. But do you really want to allocate a Promise for every mouse move event, keydown, or click? That would be horribly inefficient.

Observables can represent any collection of things over any amount of time (including zero things over no time at all). They can be merged, zipped, concatenated, forked, filtered, mapped, flatMapped, etc. Just like you would with Arrays and something like LoDash. That's not even to speak *anything* of the value added by providing a path to reactive streams. I'm still talking about using them in a very naive way.

So, this statement is liable to escalate the discussion, but I'll put it out there to get people thinking: Allocating a Promise over an Observable offers no real benefit beyond familiarity, and is in fact less beneficial because of what you lose in doing so.

Promises were a great solution and replacement for callbacks as of a few years ago. Imagine if we hadn't gone forward with Promises because callbacks were "easier" or "more familiar"? Staying with Promises over observables poses that same sort of risk, in my mind. Callbacks and promises will still always have their place, in particular callbacks, but in terms of a framework like Angular, I think it's best to lean on the power and compositional nature of Observables as much as possible.

And yes, I'm drinking the Koolaid with a funnel. Because it's the right thing to do.

[Rob Wormald]

Thanks for this Ben, very helpful! Inline questions...

Think of all of the async things you do in web development (here are a few examples):

- HTTP (Ajax/JSONP)

- Sockets

- Animations

- DOM events

- Workers

Of those, the only one that is suited well to promises is HTTP. Because it's the only async item in that list that deals with just one value. All of the others are going to emit multiple values.

 

I guess this is sort of my point - Observables make complete sense for all of those things, but Promises *do* work well for HTTP, which is why I'd like to see support for them in this specific case (which I guess is sort of covered by HttpFetch). Of course, there's absolutely no reason I can't use window.fetch my own damn self, so perhaps I should just hush :D 

Could you use Promises for, say, DOM events? Sure. But do you really want to allocate a Promise for every mouse move event, keydown, or click? That would be horribly inefficient.

And straight up insane! 

 

Observables can represent any collection of things over any amount of time (including zero things over no time at all). They can be merged, zipped, concatenated, forked, filtered, mapped, flatMapped, etc. Just like you would with Arrays and something like LoDash. That's not even to speak *anything* of the value added by providing a path to reactive streams. I'm still talking about using them in a very naive way.

This I totally understand. The "array-through-time" idea should help a lot of people with the concept, and as I tinker more with them I do love the compositional bits. 

 

So, this statement is liable to escalate the discussion, but I'll put it out there to get people thinking: Allocating a Promise over an Observable offers no real benefit beyond familiarity, and is in fact less beneficial because of what you lose in doing so.

Consider it escalated. Genuine question - what (if anything) do we lose by choosing Observables over Promises? The whole cancellable-promise idea feels dirty to me, as it pollutes the idea of a promise, so I see the benefit there. Things that come to mind include error-handling, yield/await/async/other shiny es7 things? 

 

Promises were a great solution and replacement for callbacks as of a few years ago. Imagine if we hadn't gone forward with Promises because callbacks were "easier" or "more familiar"? Staying with Promises over observables poses that same sort of risk, in my mind. Callbacks and promises will still always have their place, in particular callbacks, but in terms of a framework like Angular, I think it's best to lean on the power and compositional nature of Observables as much as possible.

Is it wrong to think of a Promise as a sort-of single-value Observable? This is how i'm reconciling it in my promise-lizard brain at the moment. Thoughts?

 

And yes, I'm drinking the Koolaid with a funnel. Because it's the right thing to do.

I'm trying, friend, I'm trying :-)

Rob

[Rob Wormald]

Sipping the koolaid myself now. One of the things that helped me really get into Promises was this awesome site: http://promise-nuggets.github.io

It does a great job of going through all those tricky async processes and explaining why Promises are Better then Callbacks. 

Is there an equivalent thing for Observables out there, and if not, such a thing would be very useful. I link promise nuggets at least once a day whilst explaining promises. 

Sent from my iPhone

To view this discussion on the web visit https://groups.google.com/d/msgid/angular-data-dev/feb56790-9466-4131-b2b8-0278de162b87%40googlegroups.com.

[Sander Elias]

Hi Ben,

Good to hear you are in on the observables!

Of those, the only one that is suited well to promises is HTTP. Because it's the only async item in that list that deals with just one value. All of the others are going to emit multiple values.

Only if you use HTTP for it most known full-round feature. There is more to HTTP then meets the eye, for example long-polling and/or chunked transfers. Those can't be handled with promises. And that's for the current version. HTTP2 is becoming a reality too. And I hope for a better interface to that then fetch!

Regards

Sander