Survey of Preferred HTTP Client Libraries

[Kevin Swiber]

Hey Craftspeople!

We talk a lot about the server side of APIs, but not so much the client.  I'd like to take a survey of what HTTP client libraries you're using when you do build API clients.

What makes you excited about it?  What are your gripes?  What factors are important when you're evaluating HTTP client libraries?

For example, I do a lot of work in Node.js, and I often use the request module.

Things I like about it:

• Minimal required API usage, but flexible enough to grow in complexity as I need it.
• Pipes response streams.
• Follows redirects.
• Easily converts an object into `application/x-www-form-urlencoded` for form submissions.

Things I don't like:

• Doesn't understand HTTP caching rules.
• No option for plugging in persistence (helpful for both cookies and caching).

What's your favorite HTTP client library?  Any language, any platform.  I love them all. :)

(Note: The end goal is to compile this feedback and make it available on the Internets for posterity.)

Thanks!

-- 

Kevin Swiber

@kevinswiber

https://github.com/kevinswiber

[Mike Kelly]

I work in ruby and tend to use Faraday

https://github.com/lostisland/faraday

It gives you a consistent interface over multiple client libs via adapters, and it enables the use of rack middleware in the request response cycle.

Hactor is my API for writing hypermedia clients. It's built on top of Faraday:

https://github.com/mikekelly/hactor

Cheers,
M

[Kevin Swiber]

On Apr 9, 2013, at 5:40 AM, Mike Kelly <mikeke...@gmail.com> wrote:

On 9 Apr 2013 10:21, "Kevin Swiber" <ksw...@gmail.com> wrote:
>
> What's your favorite HTTP client library?  Any language, any platform.  I love them all. :)

I work in ruby and tend to use Faraday

https://github.com/lostisland/faraday

It gives you a consistent interface over multiple client libs via adapters, and it enables the use of rack middleware in the request response cycle.

Cool.  I like the Rack-like request pipeline.  So if I wanted to add caching in Faraday, would it just be a matter of using an HTTP cache middleware?  Is there a list of middleware you commonly use with Faraday?

Hactor is my API for writing hypermedia clients. It's built on top of Faraday:

https://github.com/mikekelly/hactor

Hey, that's cool.  I'll check it out.

Thanks,

[Jørn Wildt]

I am a big fan of the C# library Ramone (of course, its my own project ...): https://github.com/JornWildt/Ramone.

It has support for a bunch of media types, OAuth1+2, async requests, object serialization using JSON/XML and hyper media elements (links, link templates and key/value forms a'la HTML).

/Jørn

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Mike Kelly]

On Tue, Apr 9, 2013 at 10:51 AM, Kevin Swiber <ksw...@gmail.com> wrote:
> On Apr 9, 2013, at 5:40 AM, Mike Kelly <mikeke...@gmail.com> wrote:
>
>
> On 9 Apr 2013 10:21, "Kevin Swiber" <ksw...@gmail.com> wrote:
>>
>> What's your favorite HTTP client library? Any language, any platform. I
>> love them all. :)
>
> I work in ruby and tend to use Faraday
>
> https://github.com/lostisland/faraday
>
> It gives you a consistent interface over multiple client libs via adapters,
> and it enables the use of rack middleware in the request response cycle.
>
> Cool. I like the Rack-like request pipeline. So if I wanted to add caching
> in Faraday, would it just be a matter of using an HTTP cache middleware? Is
> there a list of middleware you commonly use with Faraday?
>

yea:
https://github.com/lostisland/faraday_middleware

more listed here too:
https://github.com/lostisland/faraday/wiki#middleware

> Hactor is my API for writing hypermedia clients. It's built on top of
> Faraday:
>
> https://github.com/mikekelly/hactor
>
> Hey, that's cool. I'll check it out.
>

Cool, let me know what you think :)

Cheers,
M

[Felipe Sere]

We built a tiny wrapper around the Apache HTTP Client to make it more readable.
we basically do client.get(… ,… ,…, …) and we explicitly added parameter for Accept and Content-Type headers to really force people to think about them.

Other than that it is actually pretty bare metal. But it works. I would like to teach it Caching though...

[Jérôme Louvel]

May I suggest our high-level Restlet API for Java (especially the ClientResource class), which wraps either the Apache HTTP Client or java.net.HttpURLConnection lower-level classes?

http://restlet.org

We also have a port to JavaScript (Node.js and AJAX) in progress:

https://github.com/restlet/restlet-framework-js

Best regards,

Jerome

2013/4/9 Felipe Sere <felip...@gmail.com>

[Felipe Sere]

Restlet looks a little like the RESTeasy client service, in that you use the same API for the server as for the client. Basically both are tied to the same Java interface with the right annotations. We used this. And I didn't like it. 

It hid all kinds of details (Headers, Status codes) from us, making us less flexible.

How does RESTlet fare in that regard? Is everything abstracted and boiled down or can I see Headers and Status codes?

Cheers,

Felipe

[Jérôme Louvel]

Indeed, the Restlet API has a uniform design for both client and server sides, not only based on Java annotations like RESTeasy.

The Restlet API has a high-level ClientResource class that can use annotations or not, and that builds on top of a lower-level Request/Response API.

It does exposes HTTP statuses. See this class:

http://restlet.org/learn/javadocs/snapshot/jee/api/index.html?org/restlet/data/Status.html

Raws headers can be retrieve for debugging purpose, but they are all mapping to equivalent Java artifacts. 

For example Accept* headers are supported by the ClientInfo.accepted* properties:

http://restlet.org/learn/javadocs/snapshot/jee/api/index.html?org/restlet/data/ClientInfo.html

[Ian Joyce]

In .NET I've been happy with System.Net.Http.HttpClient (http://msdn.microsoft.com/en-us/library/system.net.http.httpclient.aspx). 

[Peter Monks]

Good topic!

In Java I tend to use the Apache HTTP Components [1] - they're low level but give the level of visibility / control that's lacking from java.net.HttpURLConnection (what's up with that capitalisation btw?  "Http" and "URL"??).  In Clojure land I've been playing with http-kit [2] - also low level, but that's the Clojure philosophy ("less is more"), and one that I'm slowly starting to appreciate (turns out functions do indeed compose very nicely - who knew?).

Speaking more generally for a moment, I don't like object mapping approaches (such as those used in Spring Social, Jackson databind, etc.).  I've found that mapping flexible data structures (i.e. JSON, XML and friends) to hard coded static class models imposes big costs in terms of system evolvability, without a commensurate benefit.  Even in languages like Java I prefer reading JSON, XML, etc. into the language's equivalent data structures (i.e. lists, maps and primitives in Java) and processing it there.

Cheers,
Peter

[1] http://hc.apache.org/

[2] http://http-kit.org/

 
 

[Adrian Cole]

In java, I currently use the terrible built in lib for the sake of less deps and a chance of operating normally on app engine and android.  This approach needs significant test infrastructure as things change between java runtimes [1]. 

 I'm interested in the jaxrs 2 client lib framework, but haven't used it [2].  Some of my colleagues work on and use the powerful ribbon library particularly as restful ops have common system concerns like load balancing handled there [3].  OR mapping is a long story :)

-A

[1] http://netflix.github.io/#blog

[2] http://java.dzone.com/articles/whats-new-jax-rs-20

[3] https://github.com/Netflix/ribbon

[André Tavares]

I work mostly with Python and use Requests (http://docs.python-requests.org/en/latest/)... it beats the hell out of urllib2 (the module that comes with python for HTTP stuff).

[Felipe Sere]

Speaking more generally for a moment, I don't like object mapping approaches (such as those used in Spring Social, Jackson databind, etc.).  I've found that mapping flexible data structures (i.e. JSON, XML and friends) to hard coded static class models imposes big costs in terms of system evolvability, without a commensurate benefit.  Even in languages like Java I prefer reading JSON, XML, etc. into the language's equivalent data structures (i.e. lists, maps and primitives in Java) and processing it there.

I disagree :-)

When I am doing Java (99%) I actually like the fact the compiler gives me warnings about types and such.

If you come from a loosely typed background, you might be used to it. But in my tiny OO-world I prefer safety.

Plus, there ARE evolvable ways of creating Java-equivalents. If you are using Jackson, you can annotate whatever you need and have the mapper ignore the rest. if you are using XML, you might actually write an XPath expression to extract values rather than JAX-B.

Even a good old-fashioned regular expression gives you evolvability.

I think the big deal he is to consciously code "adapter" that convert ${your business domain} to JSON/XML and not try to save some bytes by annotating business-logik with all kinds of annotations.

[Ted M. Young [@jitterted]]

Funny, I'm researching this right now for use in a Java-like language (Gosu) that runs on the JVM.

For my use, annotated classes to indicate the resource representation sometimes works against me, so I'm looking at both approaches. Mainly I'm looking at Jersey's Client (implementation of JAX-RS [1]) from both the 1.1 [2] and 2.0 [3] releases (the 1.1 Jersey Client is proprietary, the 2.0 version of JAX-RS specifies the Client API). I'm also considering the lower-level Apache HTTP Components as previously mentioned. Restlet's on the list as well.

I've looked at the Ribbon library that Adrian mentioned, but it's overkill for what we need right now, though it's definitely in the pipeline for consideration for an integration project I'll be working on.

;ted

--

http://jitterted.com/tidbits

http://about.me/tedmyoung

[1] http://jax-rs-spec.java.net/

[2] http://jersey.java.net/nonav/documentation/latest/user-guide.html#client-api

[3] http://jersey.java.net/nonav/documentation/snapshot/client.html

[Jørn Wildt]

Coming from the .NET world, I am with Felipe on this one. I am also used to work with typed data and prefer object serialization for this reason. To my "defense" I will say that it should be done right (as Felipe also points out):

1) Use something like an "adapter" or "view model" to represent what is going on on the wire and do not annotate real business domain classes with all sorts of XML/JSON serialization attributes.

2) Use a serializer that ignores unknown elements in the input.

This way I get the following benefits:

1) The same set of "on the wire" adapter classes can be used on both the server and the client. This means that I can either ship the adapter classes as a quick get-started SDK for other .NET devs - or use it internally when I am in control of both client and server. In this way object serialization works a "typed template", not only for generating output but also for reading input.

2) Clients can use what ever way they want for reading the server output and can live happily without ever knowing that the output has been deserialized from objects.

3) Evolvability on the client side: if the client deserializer ignores unknown input then the server is free to add new stuff without breaking the client.

The only drawback I have found so far is that one cannot do a safe evolvable GET + PUT of data: if the client deserializes the input and drops something new (and thus ignored) from it then it won't be able to serialize the same data back into the wire format and use it with a PUT (which as "complete replace" semantics) without loosing data.

I would love to hear from others what kind of pitfalls to avoid with object serialization.

/Jørn

[Peter Williams]

Years ago I built the resourceful[1] library for ruby because there just were not any other good HTTP clients for ruby at the time. Things are a little better now in that regard but I still think resourceful is one of the best HTTP libs for ruby. Good things it has:

  * client side caching
  * automatic redirection following (Any client that doesn't have this is broken. Period.)
  * gzip transfer encoding
  * pluggable authentication handling (ships with basic and digest support)

Things l'd like that are missing

  * futures based responses for easily handling concurrent requests
  * good proxy support
 

[1]: https://github.com/paul/resourceful

Peter
Barelyenough.org

[mehdi medjaoui]

Hi API craft community

We use Webshell.io a HTTP layer based on the http module of node.js. (actually we have build it ;)

Webshell.io is an API that executes Javascript on top of Node.js with a collaborative framework for client-side and server-side APIs.

It is a Javascript-as-a-Service platform for APIs.

As a HTTP client library (from browser or server call) we support :

- minimal and simple http request: get(“myurl.com”);

- Post files and buffers

- XML or JSON answer

- redirection follow

- multipart mixed in multipart form data (like email format or opera file sending)

- Authentication of the request (query auth, http basic auth, oauth 1 & 2...)

- browser request

- Requests parallelism for synchronous Javascript (we automatically make your javascript asynchronous, you don’t have to manage it yourself with callbacks, you write it synchronously)

- cache, persistence and streams (in progress)

Any feedbacks or questions are welcomed.

[David Eriksson]

I'm very happy with Guzzle for PHP. It is much nicer than using the raw cURL interface!

\David

http://www.divideandconquer.se/