A proposal for a HTTP Client

[Benjamin Eberlei]

See https://github.com/php-fig/fig-standards/pull/24

From the introduction:

Many libraries and applications require an HTTP client to talk to other servers. In general all these libraries either ship their own HTTP client library, or use low-level PHP functionality such as file_get_contents, ext/curl or the ext/socket. Depending on the use-cases there are very tricky implementation details to be handled such as proxies, SSL, authentication protocols and many more. Not every small library can afford to implement all the different details. However there are only a few http client libraries that are widely used between different projects, because of NIH or fears of vendor lock-in.

Motivation:

Doctrine has about 4 projects that need an HTTP client. Currently every projects implements them itself, which is annoying. We could abstract this into our "Common" library, however that would mean we would start being a "http client" vendor. Now personally, i don't care about http clients and would rather let others do this, however i also don't want to face vendor lock-in by deciding for any of the many http clients. In a perfect world (with this standard) Doctrine could ship one very simple http client using sockets, and if anyone needs something more awesome, he could just use any other provider of a PSR http client.

Questions:

1. Do the voting members think this is something to standardize?
2. API: I didn't include a Request object as i think its not really necessary. The proposal is expclicitly very simple.
3. How to proceed?
4. Feedback?

greetings,
Benjamin

[Benjamin Eberlei]

Ah more questions of course:

* Did i forget something on the request?
* Did i forget something on the response?
* Should the http client be bound to a host? so that $url is not containing a host? Or should the client wiggle this itself?

[Hari K T]

Hi Benjamin Eberlei, 

ie really a great proposal looking from my point of view . I would have loved if php itself had one in core :-) .

Hope all will support this in a single voice :-)

Thank you

Hari K T
M: +91-9388758821 | W: http://harikt.com/blog

--
You received this message because you are subscribed to the Google Groups "PHP Standards Working Group" group.
To post to this group, send email to php-st...@googlegroups.com.
To unsubscribe from this group, send email to php-standard...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/php-standards?hl=en.

[Kris Wallsmith]

Thanks for proposing this, Benjamin. I think this is an excellent area for standardization. We are all in the business of HTTP, after all.

However, I think we should take a step back and start with interfaces for an HTTP request and an HTTP response. Once those are done, then we can move on to an HTTP client.

Kris (author of Buzz)

--

[Jordi Boggiano]

On 24.03.2012 17:19, Kris Wallsmith wrote:
> However, I think we should take a step back and start with interfaces
> for an HTTP request and an HTTP response. Once those are done, then we
> can move on to an HTTP client.

I realize this is a long shot, but in my dream world the Symfony
HttpFoundation classes for request/response would be made standard.
Their use in the framework & then adoption by Drupal, PPI (?) and
perhaps others make them a great candidate.

For those that don't know it:

https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpFoundation/Request.php
https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpFoundation/Response.php

The use of public properties for some things make them hard to be turned
into interfaces, but anyone can rely on that component and just use
those classes. I guess Composer adoption isn't broad enough for people
to agree with me that this is ok though.

Cheers

--
Jordi Boggiano
@seldaek - http://nelm.io/jordi

[Evert Pot]

On Mar 24, 2012, at 5:06 PM, Benjamin Eberlei wrote:

> See https://github.com/php-fig/fig-standards/pull/24
>
> From the introduction:
>
> Many libraries and applications require an HTTP client to talk to other servers. In general all these libraries either ship their own HTTP client library, or use low-level PHP functionality such as file_get_contents, ext/curl or the ext/socket. Depending on the use-cases there are very tricky implementation details to be handled such as proxies, SSL, authentication protocols and many more. Not every small library can afford to implement all the different details. However there are only a few http client libraries that are widely used between different projects, because of NIH or fears of vendor lock-in.
>
> Motivation:
>
> Doctrine has about 4 projects that need an HTTP client. Currently every projects implements them itself, which is annoying. We could abstract this into our "Common" library, however that would mean we would start being a "http client" vendor. Now personally, i don't care about http clients and would rather let others do this, however i also don't want to face vendor lock-in by deciding for any of the many http clients. In a perfect world (with this standard) Doctrine could ship one very simple http client using sockets, and if anyone needs something more awesome, he could just use any other provider of a PSR http client.
>
> Questions:
>
> 1. Do the voting members think this is something to standardize?

I personally feel that this group should focus on interface, not implementation.
There is definitely something amiss in the PHP world, where a lot of project have trouble depending on external components.

But introducing a new one, doesn't really solve that I feel. What we need is more people on board with composer.phar,
and have solid, minimal components that projects like doctrine feel comfortable with to add as a dependency.

I _do_ think there is a lot of value in a common request and response interface though. Not just for HTTP client, but
also to wrap $_SERVER, php://output, header(), etc..

If these 2 interfaces exist, it would also not be terribly hard to define a very simple interface for making requests.
After all, at that point all that's really required is a method that takes a Request object, and returns a Response object.

At that point all doctrine would have to do is to provide a way for users to inject the 'Client' interface.

I feel the end-result of your proposal is a nobel and useful one, but I have serious doubts this should be done through PSR.

Evert

[Lukas Kahwe Smith]

On Mar 24, 2012, at 12:35 , Evert Pot wrote:

> I personally feel that this group should focus on interface, not implementation.
> There is definitely something amiss in the PHP world, where a lot of project have trouble depending on external components.

agreed .. we might elect one implementation as a reference implementation, but more important is that we propose interfaces, a guideline for how to interpret these interfaces and a test suite to check for compliance.

regards,
Lukas Kahwe Smith
m...@pooteeweet.org

[Benjamin Eberlei]

Just realized i forgot cookies.

I am not sure if we need interfaces for request+response. These are value objects, and if simple and clean don't need interfaces IMHO.

The client needs.

[thom...@gmail.com]

Expand

Sent from my BlackBerry® wireless device

---

From: Benjamin Eberlei <kon...@beberlei.de>

Sender: php-st...@googlegroups.com

Date: Sat, 24 Mar 2012 18:26:17 +0100

To: <php-st...@googlegroups.com>

ReplyTo: php-st...@googlegroups.com

Subject: Re: A proposal for a HTTP Client

[Evert Pot]

On Mar 24, 2012, at 6:26 PM, Benjamin Eberlei wrote:

> Just realized i forgot cookies.

If you're taking this approach, you will find that you probably forgot a lot of HTTP features :)

If you want to keep things simple, I'd strongly suggest focusing on the following:

1. Request message
- http method
- url
- http version
- headers
- stream for body

2. Response message:
- http status code
- human-readable status
- http version
- headers
- stream for body

Keep in mind that multiple headers can be set with the same name, so a key-value list for
headers may not cover all the use cases.

Using these basic components, you can represent (as far as i know?) any http message.
Anything can be implemented with these components. Cookies would not be an issue, for instance.

Utility objects and classes could be used to aid constructing these headers and deliver them.

These guidelines as a basis, this would allow me:

1. To also wrap the request and response object in my current PHP process.
2. Create a custom client that's capable of asynchronous requests through libevent
3. Create a client that gives me multiple Response objects for every hop in a case of redirects, or a situation where HTTP/1.1 100 Continue is returned.

Your simple client may well be good enough for your use-cases, but in order for something like
this to be successful, it'd be important to create something that doesn't require people to
use other tools as soon as they need anything more complex.

Evert

[Lukas Kahwe Smith]

On Mar 24, 2012, at 13:59 , Evert Pot wrote:

> Your simple client may well be good enough for your use-cases, but in order for something like
> this to be successful, it'd be important to create something that doesn't require people to
> use other tools as soon as they need anything more complex.

why?

how is not covering the more complex cases going to be a reason to not adopt this for the simple cases?

[Evert Pot]

As a user needing a simple HTTP client, I agree.. no reason to use something more complex if you don't need to.
If you are a provider of an HTTP client (such as Symfony), I'm sure you would agree that you wouldn't want to provide an incomplete client.

Evert

[Benjamin Eberlei]

hm, maybe it makes more sense to focus on pecl_http getting a sane OOP api before it goes into PHP core and until then live with the HTTP client dilemma we have in userland?

[Lukas Kahwe Smith]

incomplete client? implementing in interface does not preclude you from adding other methods. so i dont see you issue at all.
and there can be other higher level standards that build on this simple interface.

[Evert Pot]

I'm referring to the proposal on github. My point was that it's too simple, and best to first focus on just the Request and Response messages before attempting to define the client.
The current proposal does not have a Request interface, but just a simple set of arguments for making requests.

Was there a miscommunication?

Evert

[Lukas Kahwe Smith]

Apparently so, since i understood you are saying that people wouldnt implement this interface, because its too simple.
Now as for Request/Response, I agree it would be great to also standardize these, but its going to be much harder and the discussion can run in parallel.

As for Benjamin's comment about pecl http: I totally agree that its a good idea to bring Michael into this discussion (actually one of the first things I did after the PR was opened was to ping him on IRC). That being said, we are discussing interfaces here and only implementations in so far as to ensure that the interfaces make sense once they are implemented. Furthermore if possible it is of course also nice to ensure compatibility with existing libs, but as there are already so many, the risk is high that it will just alienate the authors of the given libs if we pick a "favorite".

[Michael Dowling]

As the creator of Guzzle (https://github.com/guzzle/guzzle), I'm very interested in this proposal.  I've added several comments to the PR, which I'd be happy to discuss here or on github.  Quick rundown of my comments:

• Why return an array for headers that have only one value?  Why not return an array only when there are multiple headers of the same name (this is basically what Guzzle does)?
  
• HttpException should probably be an interface so that it's easier to plug into existing projects.
• I'd like to see $headers move before $content in the arguments of HttpClient::request()
• Should $content in HttpClient::request() accept an array or string?  An array would allow you to easily send form style POST requests.  I'm fine with just expecting a string to be passed to this method, but I want to see what others think.
  
• I think that the $options array needs to have a standardized array of options or should be removed from HttpClient::request().  Allowing HTTP client vendors to specify their own custom options would still cause vendor lock-in or abstraction code to be present in consumers of these clients.
  
• As I think most clients will offer adapters instead of changing their interfaces, would it make sense to add a method to the HttpClient that would allow access to the more advanced adapted HTTP client (e.g. getAdaptedClient())?  I don't think it's a good idea to change the interfaces of Guzzle to support a much simpler HTTP client, so I'm probably going to offer an adapter.

The impetus for this proposal stems from the fact that several Doctrine projects need an HTTP client.  With the currently proposed interfaces and lack of a Request interface, you will not be able to send HTTP requests in parallel, which is something I'm thinking Doctrine would benefit from greatly.  For example, when working with unit of work pattern, you often need to do several actions when flush is called (delete things, update things, add things).  Without the use of parallel requests, you'll need to do all of these requests serially.  The way that this is handled in Guzzle is by using a client object as a sort of builder that returns requests.  These requests can then be sent one at a time or in parallel by calling the ``send`` method of the client and passing an array of requests (http://guzzlephp.org/tour/http.html#send-http-requests-in-parallel).

hm, maybe it makes more sense to focus on pecl_http getting a sane OOP api before it goes into PHP core and until then live with the HTTP client dilemma we have in userland?

While that would be awesome, I don't think it's feasible to wait on pecl_http to be updated or put into PHP's core for you or others to release new libraries.  It didn't make it into PHP 5.4, so you'd require your users to install the pecl extension or wait on PHP next.

I think this proposal could work as long as we're realistic-- provide only an interface and not an implementation, limit the scope as much as possible, and expect existing libraries to provide adapters if the proposed PSR interface differs from the original client's interface.

Thanks,
Michael

On Saturday, March 24, 2012 1:11:32 PM UTC-5, Benjamin Eberlei wrote:

hm, maybe it makes more sense to focus on pecl_http getting a sane OOP api before it goes into PHP core and until then live with the HTTP client dilemma we have in userland?

On Sat, Mar 24, 2012 at 7:01 PM, Lukas Kahwe Smith <m...@pooteeweet.org> wrote:

On Mar 24, 2012, at 13:59 , Evert Pot wrote:

> Your simple client may well be good enough for your use-cases, but in order for something like
> this to be successful, it'd be important to create something that doesn't require people to
> use other tools as soon as they need anything more complex.

why?

how is not covering the more complex cases going to be a reason to not adopt this for the simple cases?

regards,
Lukas Kahwe Smith
m...@pooteeweet.org

--
You received this message because you are subscribed to the Google Groups "PHP Standards Working Group" group.
To post to this group, send email to php-st...@googlegroups.com​.

To unsubscribe from this group, send email to php-standards+unsubscribe@​googlegroups.com.

[Benjamin Eberlei]

i disagree on pecl/http. If it ships with a sane API that does not require wrapping, then developing a PHP clone of this NOW is much better task for all our limited time than trying to find a standard to wrap pecl/http into.

I agree on your points though, the API was a long shot and i already want to remove request in its own object, that way we dont have to argue about order in the request() method.

To view this discussion on the web visit https://groups.google.com/d/msg/php-standards/-/uTPxNJwxz0QJ.

To post to this group, send email to php-st...@googlegroups.com.

To unsubscribe from this group, send email to php-standard...@googlegroups.com.

[Michael Dowling]

I think it would be great for pecl/http to be updated, and I'd consider using it in Guzzle instead of curl (as long as it has the same callback support that curl provides).  However, providing a new interface for pecl/http and implementing a PHP only version is just reinventing the wheel on top of many existing wheels (Guzzle, Buzz, Zend HTTP client, PEAR, etc).

I like the concept of introducing a simple set of interfaces for libraries that need a simple HTTP client.  I don't think the PSR group getting into the business of providing implementations is a good idea though (maybe I misunderstood though).

-Michael

[Paul M Jones]

Hi all,

To suggest a course of action that appears to have been fruitful with PSR-1: Compile a list of known HTTP request/response/client libraries, examine them for commonalities, publish the results, and go from there.

Guzzle, Buzz, ZF, Solar (and soon Aura), Symfony, Lithium, Cake, PECL HTTP, etc. all have HTTP request/response classes. They'd be the likely targets for research.

That way at least we have an idea of what people are *actually* doing, and not be limited to our own ideas of what we *think* people *ought* to be doing.

--

Paul M. Jones
http://paul-m-jones.com/

[Evert Pot]

If we feel it helps, I can ask someone from the httpbis group to provide input too.
We should be a bit further ourselves though :)

Evert

[Alexandre Gaigalas]

We should also look how the PHP native HTTP stream wrapper works. After researching a lot of clients I've settled on stream_get_contents() for the body + stream_get_metadata() for the response headers and stream_context_create() for the request declaration, even though they don't have an OOP interface.

ref: http://www.php.net/manual/en/context.http.php

A thin OOP layer based on these functions would make more sense to me than any other. A reference implementation would also be very easy to set up without relying on any specific vendors.

If we keep up this proposal, I believe we should also add to the compiled list of clients how they handle HTTP-specific workflows. Not all clients nowadays are able to handle transfer encodings, protocol upgrades, cookie persistence, file uploads, keep alives and many other core HTTP features.

Maybe the clean request() method should even be clear about *not* treat all those workflows, to keep things uniform without having to implement a large subset of the spec. If we consider another RFCs besides 2616 there is a *lot* of things to specify. Subsequent revisions of this possible PSR should address more complex workflows as clients adopt the basic interface.

From the spec point of view, there is no valid Request without a Response and vice-versa. Browser-to-server implementations tend to separate these concepts mainly because of their "phisical" separation, but as an HTTP client self-contained this seems to be unecessary. The Request/Response pair is defined by the spec as an HTTP Message, and makes a lot of sense to me that a client could need only a Message interface to be clean and consistent.

--
You received this message because you are subscribed to the Google Groups "PHP Standards Working Group" group.

To post to this group, send email to php-st...@googlegroups.com.

To unsubscribe from this group, send email to php-standard...@googlegroups.com.

[Alexandre Gaigalas]

Oops. Small correction: The spec defines an HTTP Message as an abstract format for Request and Response formats. They differ mainly on the request/status line and the related workflows.

[Suissa]

I agree.

2012/3/25 Alexandre Gaigalas <alex...@gaigalas.net>

--
Jean C. Nascimento aka Suissa
WebDeveloper Sênior - SP

http://about.me/suissa

      

[Beau Simensen]

2. API: I didn't include a Request object as i think its not really necessary. The proposal is expclicitly very simple.

Could we define standard HTTP Request and Response interfaces (and an HTTP Client that is just a pass-thru for Request -> Response) in one PSR and define a Simple Client that leverages the HTTP Response from the other PSR?

/* PSR-HTTP-Client */

use Whizbang\Client as PsrClient; // implements PsrClientInterface

use Whizbang\Request as PsrRequest; // implements PsrRequestInterface

$psrClient = new PsrClient;

$psrRequest = new PsrRequest();

(PsrResponseInterface) $psrResponse = $psrClient->makeRequest($psrRequest);

/* PSR-HTTP-SimpleClient */

use Whizbang\SimpleClient as PsrSimpleClient; // implements PsrSimpleClientInterface

$psrSimpleClient = new PsrSimpleClient;

(PsrResponseInterface) $psrReponse = $psrSimpleClient->makeSimpleRequest(/* specialized set of simple arguments */);

If so, vendors would be able to opt supporting either PSR-HTTP-Client or PSR-HTTP-SimpleClient (or both). Users would be able to decide which route they would need to go down for their particular application.

[Beau Simensen]

Should the http client be bound to a host? so that $url is not containing a host? Or should the client wiggle this itself?

I think this would only be useful under very specific circumstances. In general I don't want to have to worry about the URL I am accessing being the same host as the last URL I accessed.

Being able to support this might be a good case for having a Request interface, though? It would provide flexibility to handle this kind of thing in the application's logic as opposed to offloading it to the HTTP Client magic.

    $apiRequestFactory = new MyApiRequestFactory('http://example.com/api/baseUrl');

    $psrResponse = $psrClient->makeRequest($apiRequestFactory->createPost('category/programming/tag/php'));

    $psrResponse = $psrClient->makeRequest($apiRequestFactory->createGet('products'));

    $psrResponse = $psrClient->makeRequest(new Request('GET', $product->getFavoritesUrl()));

It would be pretty trivial to provide a set of custom Request types and Request factories (GetRequest($url,...) , PostRequest($url,...), RestResourceRequestFactory($baseUrl)) as references. I'm not sure if they should be a part of the PSR, but it might not be a terrible idea.

[Beau Simensen]

Does anyone else think this might be a good approach? If so I'd like to try and get the ball moving on this one again.

[Alexandre Gaigalas]

I like it. Let me know if I get things right:

The final product of this recommendation would be a set of interfaces that compliant HTTP clients written in PHP must conform to. Some of them are optional based on the features that the client provides, so we can also use these interfaces to check if a specific client is compliant to something.

Right so far? Where do we go from here?

--

You received this message because you are subscribed to the Google Groups "PHP Standards Working Group" group.

To view this discussion on the web visit https://groups.google.com/d/msg/php-standards/-/CWSymx3hW2wJ.

[Norv N.]

I think it's a good approach. Request and Response interfaces are expected and necessary. FWIW, I rather see a good standard proposal as one that covers first the essential interfaces of the domain.

The HTTP client interface can be defined in parallel or after these are established.

[Dowling, Michael]

Assuming this proposal comes to fruition, what's the expectation on existing HTTP clients?  Are users expecting breaking changes, adapters, or something else?

-Michael

From: "Norv N." <no...@simplemachines.org>
Reply-To: <php-st...@googlegroups.com>
Date: Thu, 7 Jun 2012 07:08:08 -0700 (PDT)
To: <php-st...@googlegroups.com>
Subject: Re: A proposal for a HTTP Client

--

You received this message because you are subscribed to the Google Groups "PHP Standards Working Group" group.

To view this discussion on the web visit https://groups.google.com/d/msg/php-standards/-/W4twfs1ovXIJ.

[Beau Simensen]

On Thursday, June 7, 2012 12:10:39 PM UTC-5, Michael Dowling wrote:

Assuming this proposal comes to fruition, what's the expectation on existing HTTP clients?  Are users expecting breaking changes, adapters, or something else?

I think that my expectation is that existing HTTP clients would not need to be changed. That is my expectation for the Cache proposals as well. New implementations would be welcome to implement the proposals directly but existing vendors can provide support for the interfaces by whatever means necessary. If that means BC breaks for their users, that is up to them.

People create different implementations for different reasons and people choose those implementations based on the differences. Cache and HTTP clients can be very specialized and a user can choose one that meets their specialized needs if they have specialized needs. I think it is important to not get in the way of the vendors in this case.

On the other hand, applications or libraries that just needs "a cache" or "an HTTP client" may not care about the details or specifics and can be written instead to accept a PSR cache or HTTP client implementation. A framework can be configured to use Buzz or Guzzle or XYZ as its "PSR HTTP client" and generic libraries and objects can consume it by way of dependency injection. Specialized domain objects can instead accept the actual implementation by way of dependency injection where access to functionality that isn't available by the PSR HTTP client is required.

[Kris Wallsmith]

I do not agree with the Client/SimpleClient distinction. I've designed Buzz so the only thing a client needs to worry about is executing an HTTP request and populating the response. Since you pass the response object to the client, it isn't even expected to create that object.

interface ClientInterface

{

    function send(RequestInterface $request, MessageInterface $response);

}

If you want an easier interface you can wrap one around a client. In Buzz this is called a browser and has nice methods like…

public function get($url, $headers = array())

public function post($url, $headers = array(), $content = '')

public function call($url, $method, $headers = array(), $content = '')

public function submit($url, array $fields, $method = RequestInterface::METHOD_POST, $headers = array())

public function send(RequestInterface $request, MessageInterface $response = null)

It is important that these are two separate concepts.

However, I still think we should focus our current efforts on defining request and response interfaces. I've included Buzz's here for discussion…

interface MessageInterface

{

    function getHeader($name, $glue = "\r\n");

    function getHeaders();

    function setHeaders(array $headers);

    function addHeader($header);

    function getContent();

    function setContent($content);

}

interface RequestInterface extends MessageInterface

{

    const METHOD_OPTIONS = 'OPTIONS';

    const METHOD_GET     = 'GET';

    const METHOD_HEAD    = 'HEAD';

    const METHOD_POST    = 'POST';

    const METHOD_PUT     = 'PUT';

    const METHOD_DELETE  = 'DELETE';

    const METHOD_PATCH   = 'PATCH';

    function getMethod();

    function setMethod($method);

    function getResource();

    function setResource($resource);

    function getProtocolVersion();

    function getHost();

    function setHost($host);

    function isSecure();

}

Thanks,

Kris

[Beau Simensen]

On Thursday, June 7, 2012 12:54:47 PM UTC-5, Kris Wallsmith wrote:

interface ClientInterface
{
    function send(RequestInterface $request, MessageInterface $response);
}

What are the benefits to passing in a $response vs having the object create it?

I do not agree with the Client/SimpleClient distinction.

What part do you disagree with? That there would be a SimpleClient at all?

Here is what I currently have in mind. I think that Client would be similar to Buzz Client interface and SimpleClient would be similar to Buzz Browser. Maybe it is the naming that is a bit off?

PSR-HTTP-Client:

namespace Fig\Psr\HttpClient;

interface RequestInterface
{