[boost] [beast] Request for Discussion
694 views
Skip to first unread message
Vinnie Falco
Sep 23, 2016, 10:58:53 AM9/23/16
to bo...@lists.boost.org
Beast provides low level HTTP and WebSocket interfaces built on
Boost.Asio, the project is here:
https://github.com/vinniefalco/Beast
The plan is to solicit feedback from the Boost community before
submitting a request for a formal review.
First a brief update: I have attended CppCon 2016 and provided a 15
minute Lightning Talk on the library. There should be a video up at
some point with all the talks, including mine. I spread the word about
Beast and discovered that there are several folks in the wild already
using the library and happy with it. There are also a couple of
commercial products that are using it.
One point brought up during the conference is that Beast does not
offer sufficient higher level HTTP functionality. This omission is
intentional. It is my idea to offer the building blocks (serializing
and deserializing HTTP/1 messages, in synchronous and asynchronous
flavors, and a universal HTTP message model), upon which other
developers more knowledgeable than me could build those interfaces.
The higher level pubic operations requested were:
1. Perform HTTP GET or POST and receive the response. Presumably this
would handle both plain and TLS connections, handle redirects, and
handle HEAD requests. Not sure about traversing proxies.
2. Expose an asynchronous HTTP server class which callers can
instantiate with some sort of handler that returns the HTTP response
given a request. It should be possible, in a minimal number of lines,
to quickly get a server up and running.
The requestor informed me that this functionality did not need to
provide every possible feature or handle every use case. It just
needed to handle the common use-cases.
For the record, I am against this higher level functionality for a
number of reasons. The higher the level of functionality, the more
contentious the feature becomes, and the harder it is to satisfy a
quorum of reviewers. No one would dispute that we need the ability to
read a complete HTTP message from a socket. But not everyone will
agree that a HTTP server needs an option to allow multiple listening
sockets with different level of permissions.
Another problem with offering higher level functionality is that it
could be less likely to be approved for standardization in the C++
standard library. For example, the attributes of ease of use and fully
functional oppose each other. If the functionality offered by a
hypothetical std::http::client is not sufficient, a developer would be
forced to write their own with little possibility to reuse parts of
the implementation from the stdlib. I doubt the committee would
approve of such a thing. I know I wouldn't. Admittedly I have little
experience or knowledge on what would or would not get approved.
A point was raised that a low-level-only HTTP implementation in Boost
has the potential to discourage new users that need to perform simple
operations like HTTP GET from using Boost in the future, given that
robustly handling such operations would require the developer to write
a significant amount of code. I can see some validity here. On the
other hand, if Boost.Asio was submitted for a formal review today,
would those same points be raised? Would Asio get rejected because it
doesn't offer public interfaces for doing simple tasks like requesting
an object from a web server? I wonder if perhaps the Boost formal
review process has become defective.
For reference here is what the current API offers:
Example program that performs HTTP GET, then receives and prints the response:
https://github.com/vinniefalco/Beast/blob/master/examples/http_example.cpp
This is a high performance asynchronous HTTP server that serves local
files as static pages. It is not an official API, it is part of the
examples. Users who want a server need to copy the source code and
modify it to suit their needs:
https://github.com/vinniefalco/Beast/blob/master/examples/http_async_server.hpp
My initial purpose for requesting this discussion is two-fold:
1. Determine the level of consensus on the issue that Beast needs to
do more for HTTP than the lowest level of functionality (reading and
writing messages). I'm open to hearing all arguments for and against.
2. Get feedback on what official public interfaces to higher level
functionality might look like. Specifically, a client that handles GET
and POST, and a generic asynchronous server. And what features those
interfaces would support. For example, is traversing a proxy a
necessary feature for an official client interface? The best answers
would provide function signatures or class declarations using the
existing Beast HTTP types. In case its not clear, I am asking
stakeholders and reviewers to help design the library.
I'll open a discourse on the HTTP client to illustrate the proposed
style of discussion:
We want to provide a HTTP client implementation that lets users
perform simple tasks like fetch a resource from a remote host. Calling
code should be compact, letting people get their job done without
getting mired in boilerplate. Here's a possible signature with example
use:
response_v1<string_body>
get(std::string const& url, error_code& ec);
error_code ec;
auto res = get("http://boost.org", ec);
if(! ec)
std::cout << res;
else
std::cerr << ec.message() << '\n';
The get function would automatically perform the name resolution,
connect the socket, request the resource with uri "/", and return the
result.
There's a problem here though, the caller is forced to use a message
with std::string as the container for the body. An obvious fix is to
add a template parameter:
template<class Body = string_body>
response_v1<Body>
get(std::string const& url, error_code& ec);
Better, but what if the body is not DefaultConstructible? We can add
some parameters to forward to the Body constructor:
template<class Body = string_body, Args&&... args>
response_v1<Body>
get(std::string const& url, error_code& ec, Args&&... args);
We're getting somewhere now. Most users will use the default body,
while an option exists for advanced users. But the implementation of
get now has a complication. If an error occurs, it still has to
construct a return value.
This problem is likely surmountable but lets table it and start a new
line of reasoning. What if the URI indicates https, e.g.
"https://boost.org"? We can hide that under the hood by using a
separate code path that uses asio::ssl::stream instead of
ip::tcp::socket. But how does the caller provide the root CA
certificates? How does the caller provide a client certificate? We are
left with a client that satisfies two thirds of users but is unusable
for the remaining one third. Is this Boost worthy? Is this std-lib
worthy? I think not.
Lets revisit the SSL issues later and think about a new problem. What
happens to the socket when the request is done? In the proposed
interface, the socket is destroyed. This means you can only make one
request on a connection.
At this point, I think its clear that the current interface is a
non-starter. Here's a better idea. Lets define a class called client
and make get a member function. Now we can have state, including
configurable settings with reasonable defaults.
class client
{
public:
template<class Option>
void
set_option(Option&&);
template<class Body = string_body, Args&&... args>
response_v1<Body>
get(std::string const& url, error_code& ec, Args&&... args);
};
client c;
error_code ec;
auto res = c.get("http://boost.org", ec);
if(! ec)
std::cout << res;
else
std::cerr << ec.message() << '\n';
We still have some of the original problems, but other problems are
solved. The class can keep the socket around, so that a subsequent
call to get with the same server will use the already-open socket. The
caller can set options on the client object instance before invoking
get, such as any TLS certificates.
Unfortunately we have new problems. Where is the io_service? If the
class creates it for you, how many threads? How do I use my existing
io_service, or is that even necessary? The class interface allows the
connection to be kept alive but what if now the caller wants to close
it (i.e. specify "Connection: close" in the request)?
And there are some problems which are applicable to any interface. How
does the caller adjust the parser options, for example change the
limit on the size of the body? What if the caller wants the request
made using a specific HTTP version? How does this handle basic
authentication? How can the caller adjust the headers in the request?
etc...
This is why I think that offering a HTTP client is a rabbit hole which
can only lead to a rejected formal review. Or maybe I am overthinking
this? I'd love to hear proposed solutions to the issues (which could
be just to say "forget about those features").
Despite my objection I have started work on these features anyway.
Here are the beginnings of a HTTP client:
https://github.com/vinniefalco/Beast/blob/88b485df7e6216282842f40cf99cc75dee1b82d4/test/http/get.cpp#L123
Here is some work on a generic server:
https://github.com/vinniefalco/Beast/blob/server/extras/beast/http/async_server.hpp#L84
Please reply to this thread instead of starting a new thread, and keep
the subject line prefix (e.g. "Re: [boost] [beast] ...")
Thank you
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Boost.Asio, the project is here:
https://github.com/vinniefalco/Beast
The plan is to solicit feedback from the Boost community before
submitting a request for a formal review.
First a brief update: I have attended CppCon 2016 and provided a 15
minute Lightning Talk on the library. There should be a video up at
some point with all the talks, including mine. I spread the word about
Beast and discovered that there are several folks in the wild already
using the library and happy with it. There are also a couple of
commercial products that are using it.
One point brought up during the conference is that Beast does not
offer sufficient higher level HTTP functionality. This omission is
intentional. It is my idea to offer the building blocks (serializing
and deserializing HTTP/1 messages, in synchronous and asynchronous
flavors, and a universal HTTP message model), upon which other
developers more knowledgeable than me could build those interfaces.
The higher level pubic operations requested were:
1. Perform HTTP GET or POST and receive the response. Presumably this
would handle both plain and TLS connections, handle redirects, and
handle HEAD requests. Not sure about traversing proxies.
2. Expose an asynchronous HTTP server class which callers can
instantiate with some sort of handler that returns the HTTP response
given a request. It should be possible, in a minimal number of lines,
to quickly get a server up and running.
The requestor informed me that this functionality did not need to
provide every possible feature or handle every use case. It just
needed to handle the common use-cases.
For the record, I am against this higher level functionality for a
number of reasons. The higher the level of functionality, the more
contentious the feature becomes, and the harder it is to satisfy a
quorum of reviewers. No one would dispute that we need the ability to
read a complete HTTP message from a socket. But not everyone will
agree that a HTTP server needs an option to allow multiple listening
sockets with different level of permissions.
Another problem with offering higher level functionality is that it
could be less likely to be approved for standardization in the C++
standard library. For example, the attributes of ease of use and fully
functional oppose each other. If the functionality offered by a
hypothetical std::http::client is not sufficient, a developer would be
forced to write their own with little possibility to reuse parts of
the implementation from the stdlib. I doubt the committee would
approve of such a thing. I know I wouldn't. Admittedly I have little
experience or knowledge on what would or would not get approved.
A point was raised that a low-level-only HTTP implementation in Boost
has the potential to discourage new users that need to perform simple
operations like HTTP GET from using Boost in the future, given that
robustly handling such operations would require the developer to write
a significant amount of code. I can see some validity here. On the
other hand, if Boost.Asio was submitted for a formal review today,
would those same points be raised? Would Asio get rejected because it
doesn't offer public interfaces for doing simple tasks like requesting
an object from a web server? I wonder if perhaps the Boost formal
review process has become defective.
For reference here is what the current API offers:
Example program that performs HTTP GET, then receives and prints the response:
https://github.com/vinniefalco/Beast/blob/master/examples/http_example.cpp
This is a high performance asynchronous HTTP server that serves local
files as static pages. It is not an official API, it is part of the
examples. Users who want a server need to copy the source code and
modify it to suit their needs:
https://github.com/vinniefalco/Beast/blob/master/examples/http_async_server.hpp
My initial purpose for requesting this discussion is two-fold:
1. Determine the level of consensus on the issue that Beast needs to
do more for HTTP than the lowest level of functionality (reading and
writing messages). I'm open to hearing all arguments for and against.
2. Get feedback on what official public interfaces to higher level
functionality might look like. Specifically, a client that handles GET
and POST, and a generic asynchronous server. And what features those
interfaces would support. For example, is traversing a proxy a
necessary feature for an official client interface? The best answers
would provide function signatures or class declarations using the
existing Beast HTTP types. In case its not clear, I am asking
stakeholders and reviewers to help design the library.
I'll open a discourse on the HTTP client to illustrate the proposed
style of discussion:
We want to provide a HTTP client implementation that lets users
perform simple tasks like fetch a resource from a remote host. Calling
code should be compact, letting people get their job done without
getting mired in boilerplate. Here's a possible signature with example
use:
response_v1<string_body>
get(std::string const& url, error_code& ec);
error_code ec;
auto res = get("http://boost.org", ec);
if(! ec)
std::cout << res;
else
std::cerr << ec.message() << '\n';
The get function would automatically perform the name resolution,
connect the socket, request the resource with uri "/", and return the
result.
There's a problem here though, the caller is forced to use a message
with std::string as the container for the body. An obvious fix is to
add a template parameter:
template<class Body = string_body>
response_v1<Body>
get(std::string const& url, error_code& ec);
Better, but what if the body is not DefaultConstructible? We can add
some parameters to forward to the Body constructor:
template<class Body = string_body, Args&&... args>
response_v1<Body>
get(std::string const& url, error_code& ec, Args&&... args);
We're getting somewhere now. Most users will use the default body,
while an option exists for advanced users. But the implementation of
get now has a complication. If an error occurs, it still has to
construct a return value.
This problem is likely surmountable but lets table it and start a new
line of reasoning. What if the URI indicates https, e.g.
"https://boost.org"? We can hide that under the hood by using a
separate code path that uses asio::ssl::stream instead of
ip::tcp::socket. But how does the caller provide the root CA
certificates? How does the caller provide a client certificate? We are
left with a client that satisfies two thirds of users but is unusable
for the remaining one third. Is this Boost worthy? Is this std-lib
worthy? I think not.
Lets revisit the SSL issues later and think about a new problem. What
happens to the socket when the request is done? In the proposed
interface, the socket is destroyed. This means you can only make one
request on a connection.
At this point, I think its clear that the current interface is a
non-starter. Here's a better idea. Lets define a class called client
and make get a member function. Now we can have state, including
configurable settings with reasonable defaults.
class client
{
public:
template<class Option>
void
set_option(Option&&);
template<class Body = string_body, Args&&... args>
response_v1<Body>
get(std::string const& url, error_code& ec, Args&&... args);
};
client c;
error_code ec;
auto res = c.get("http://boost.org", ec);
if(! ec)
std::cout << res;
else
std::cerr << ec.message() << '\n';
We still have some of the original problems, but other problems are
solved. The class can keep the socket around, so that a subsequent
call to get with the same server will use the already-open socket. The
caller can set options on the client object instance before invoking
get, such as any TLS certificates.
Unfortunately we have new problems. Where is the io_service? If the
class creates it for you, how many threads? How do I use my existing
io_service, or is that even necessary? The class interface allows the
connection to be kept alive but what if now the caller wants to close
it (i.e. specify "Connection: close" in the request)?
And there are some problems which are applicable to any interface. How
does the caller adjust the parser options, for example change the
limit on the size of the body? What if the caller wants the request
made using a specific HTTP version? How does this handle basic
authentication? How can the caller adjust the headers in the request?
etc...
This is why I think that offering a HTTP client is a rabbit hole which
can only lead to a rejected formal review. Or maybe I am overthinking
this? I'd love to hear proposed solutions to the issues (which could
be just to say "forget about those features").
Despite my objection I have started work on these features anyway.
Here are the beginnings of a HTTP client:
https://github.com/vinniefalco/Beast/blob/88b485df7e6216282842f40cf99cc75dee1b82d4/test/http/get.cpp#L123
Here is some work on a generic server:
https://github.com/vinniefalco/Beast/blob/server/extras/beast/http/async_server.hpp#L84
Please reply to this thread instead of starting a new thread, and keep
the subject line prefix (e.g. "Re: [boost] [beast] ...")
Thank you
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Paul A. Bristow
Sep 23, 2016, 11:49:45 AM9/23/16
to bo...@lists.boost.org
> -----Original Message-----
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Vinnie Falco
> Sent: 23 September 2016 15:58
> To: bo...@lists.boost.org
> Subject: [boost] [beast] Request for Discussion
>
> Beast provides low level HTTP and WebSocket interfaces built on
> Boost.Asio, the project is here:
> https://github.com/vinniefalco/Beast
> The plan is to solicit feedback from the Boost community before
> submitting a request for a formal review.
>
> First a brief update: I have attended CppCon 2016 and provided a 15
> minute Lightning Talk on the library.
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Vinnie Falco
> Sent: 23 September 2016 15:58
> To: bo...@lists.boost.org
> Subject: [boost] [beast] Request for Discussion
>
> Beast provides low level HTTP and WebSocket interfaces built on
> Boost.Asio, the project is here:
> https://github.com/vinniefalco/Beast
> The plan is to solicit feedback from the Boost community before
> submitting a request for a formal review.
>
> First a brief update: I have attended CppCon 2016 and provided a 15
> minute Lightning Talk on the library.
> One point brought up during the conference is that Beast does not
> offer sufficient higher level HTTP functionality. This omission is
> intentional. It is my idea to offer the building blocks (serializing
> and deserializing HTTP/1 messages, in synchronous and asynchronous
> flavors, and a universal HTTP message model), upon which other
> developers more knowledgeable than me could build those interfaces.
<big snip>
> offer sufficient higher level HTTP functionality. This omission is
> intentional. It is my idea to offer the building blocks (serializing
> and deserializing HTTP/1 messages, in synchronous and asynchronous
> flavors, and a universal HTTP message model), upon which other
> developers more knowledgeable than me could build those interfaces.
I'm not an expert on the subject, but I think you are right to be cautious about trying to be everything to all men.
I think you should propose what you have now for formal review.
And provide as many *examples* as possible of how it can be used.
If people decide to ask you to extend the features in future, you (or others) can,
but you are persuasive that premature 'standardization' sounds risky.
(C++ Standards people really like to have implementation, and widespread usage.
Fancier features can be added in later years - provided you have got the basics right!)
> This is why I think that offering a HTTP client is a rabbit hole which
> can only lead to a rejected formal review. Or maybe I am overthinking
> this? I'd love to hear proposed solutions to the issues (which could
> be just to say "forget about those features").
>
> Despite my objection I have started work on these features anyway.
> Here are the beginnings of a HTTP client:
> https://github.com/vinniefalco/Beast/blob/88b485df7e6216282842f40cf99cc75dee1b82d4/test/http/get.cpp#L123
>
> Here is some work on a generic server:
> https://github.com/vinniefalco/Beast/blob/server/extras/beast/http/async_server.hpp#L84
HTH
Paul
---
Paul A. Bristow
Prizet Farmhouse
Kendal UK LA8 8AB
+44 (0) 1539 561830
Vinnie Falco
Sep 23, 2016, 12:26:11 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 8:49 AM, Paul A. Bristow
<pbri...@hetp.u-net.com> wrote:
> I'm not an expert on the subject, but I think you are right to be cautious
> about trying to be everything to all men.
> ...
<pbri...@hetp.u-net.com> wrote:
> I'm not an expert on the subject, but I think you are right to be cautious
> about trying to be everything to all men.
> I think you should propose what you have now for formal review.
I appreciate the kind words and support for a formal review. I have
carefully studied all of the review comments provided during the
review of Boost.Http in 2015. In particular, a comment from Robert
Ramey:
from http://lists.boost.org/Archives/boost/2015/08/224700.php
"To me it's very unfortunate that so much effort has had to be
expended by a library author to produce a library that is not
accepted. It's also unfortunate that so many reviewers need to spend
this much time to dig up enough information to reach this consensus.
This illustrates my motivation behind the design of the boost library
incubator."
I think the idea behind the incubator is sound, and I agree with
Robert's assessment. Looking over the formal review comments for
Boost.Http, about half of the reviewers admitted they did not have
enough time to do a deeper dive. I would like to avoid these mistakes.
My approach is to get a higher awareness of Beast through discussions
on this mailing list, and through face to face meetings with
stakeholders and reviewers (for example at CppCon 2016, and C++ Now
2017). In the meanwhile, the number of users continues to grow.
The odds of success during a formal review can only go up by
collecting and addressing feedback from stakeholders before the review
begins. These are the set of individuals who participated in the 2015
review of Boost.Http:
Agustin K-ballo Berge
Antony Polukhin
Bjorn Reese
Darren Cook
David Sankel
Edward Diener
Glen Fernandes
Lee Clagett
Niall Douglas
Robert Ramey
Rodrigo Madera
Roland Bock
Tom Kent
Vinicius Oliveira
Vladimir Prus
I would like to hear from as many of these individuals as possible,
about their experience looking at Beast and taking it for a test
drive, before starting the formal review process.
Of special importance is to get feedback from Vinicius Oliveira, the
author of Boost.Http. We spent several hours talking about our
respective libraries with each other. Beast provides the lowest level
interfaces, while Boost.Http tries to offer the higher level
functionality that was recently requested. Specifically, Boost.Http
attempts to provide a turn-key server solution. I've heard that
Vinicius is busy working on a client, and has also implemented his own
parser.
I think public feedback on Beast from the author of Boost.Http is
crucial, since Beast claims to offer uncontroversial low level
functionality which Boost.Http would have to use if Beast is accepted
to Boost. Perplexingly, I have seen no evidence of any public dialogue
from Vinicius about his current client interface goals, or any plan to
refactor Boost.Http to use Beast. I'd like to hear his reasoning for
why a HTTP library needs to "support multiple backends" (what does
that mean?), a theme which came up often in his responses to review
feedback. Is this a feature that serves the needs of users?
Thanks
Robert Ramey
Sep 23, 2016, 2:09:23 PM9/23/16
to bo...@lists.boost.org
On 9/23/16 9:25 AM, Vinnie Falco wrote:
> I would like to hear from as many of these individuals as possible,
> about their experience looking at Beast and taking it for a test
> drive, before starting the formal review process.
Indeed. One of the big motivations behind the incubator is to
> I would like to hear from as many of these individuals as possible,
> about their experience looking at Beast and taking it for a test
> drive, before starting the formal review process.
permit the gathering of information before the review at a time
which is convenient for potential reviewers. That is, I wanted
to decouple the review from a narrow two week window.
It hasn't worked out that way. But I haven't given up hope. People
love the general developer's mailing list. My next idea is to replace
the comments implementation with a facade over the developer's mailing
list which selects posts regarding a particular library. This is in
keeping with the incubator implementation strategy of tailgating on
resources maintained by other people. Unfortunately, I haven't found an
easy way to do this yet.
>
> I think public feedback on Beast from the author of Boost.Http is
> crucial, since Beast claims to offer uncontroversial low level
> functionality which Boost.Http would have to use if Beast is accepted
> to Boost. Perplexingly, I have seen no evidence of any public dialogue
> from Vinicius about his current client interface goals, or any plan to
> refactor Boost.Http to use Beast. I'd like to hear his reasoning for
> why a HTTP library needs to "support multiple backends" (what does
> that mean?), a theme which came up often in his responses to review
> feedback. Is this a feature that serves the needs of users?
this axiom gets lost from time to time in Boost and other projects which
have to satisfy too many people. Bjarne referred to this inadvertently
in his keynote - "Boost libraries sometimes needlessly over complicated"
(unlike standard library components).
BTW - I very much liked your idea of handing out your tiny brochure at
CPPCon. I was especially a good idea to make it the size you did -
about 3 by 4 inches. This made it easy to stick in one's pocket and
kept it from having an accurate subtitle - http access for idiots.
Robert Ramey
Niall Douglas
Sep 23, 2016, 2:50:32 PM9/23/16
to bo...@lists.boost.org
On 23 Sep 2016 at 7:58, Vinnie Falco wrote:
> One point brought up during the conference is that Beast does not
> offer sufficient higher level HTTP functionality. This omission is
> intentional. It is my idea to offer the building blocks (serializing
> and deserializing HTTP/1 messages, in synchronous and asynchronous
> flavors, and a universal HTTP message model), upon which other
> developers more knowledgeable than me could build those interfaces.
Time to be mildly harsh.
> One point brought up during the conference is that Beast does not
> offer sufficient higher level HTTP functionality. This omission is
> intentional. It is my idea to offer the building blocks (serializing
> and deserializing HTTP/1 messages, in synchronous and asynchronous
> flavors, and a universal HTTP message model), upon which other
> developers more knowledgeable than me could build those interfaces.
I'll repeat my main observation made during the Http review:
Almost everyone here and in the wider C++ community wants somebody to
take the HTTP client API from the Python Requests module
(http://docs.python-requests.org/en/master/) and very closely
replicate it in C++ (it need not be Python Requests specifically,
just some well established highly regarded high level HTTP API battle
tested over many years - the key part here is DO NOT try reinventing
the wheel over well established practice).
We don't care about async, memory copying, performance, use of
std::string, parsing, use of ASIO, serialisation or any of that other
detail until somebody gets the bloody top end API implemented and
stop wasting time on these currently unimportant implementation
details.
AFTER somebody has implemented a close replica of Requests which is
fully functional - and very few here would care how it is internally
implemented so long as the public API design is solid - THEN the
refinement of implementation can begin.
As I was saying to David Sankel and Roland Bock yesterday when we
were discussing HTTP and BEAST, I really wouldn't care if you stuck
O(N^N) complexity loops into your implementation and throw in a
pointless sleep(500ms) for good measure in an initial implementation.
*I don't care about the quality of the implementation*. I only care,
for now, about the solid design of the top level API. Once we have a
working implementation of that which has passed Boost peer review,
everything else will follow naturally.
Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/
http://ie.linkedin.com/in/nialldouglas/
Howard Hinnant
Sep 23, 2016, 4:05:10 PM9/23/16
to bo...@lists.boost.org
On Sep 23, 2016, at 11:50 AM, Niall Douglas <s_sour...@nedprod.com> wrote:
>
> As I was saying to David Sankel and Roland Bock yesterday when we
> were discussing HTTP and BEAST, I really wouldn't care if you stuck
> O(N^N) complexity loops into your implementation and throw in a
> pointless sleep(500ms) for good measure in an initial implementation.
> *I don't care about the quality of the implementation*. I only care,
> for now, about the solid design of the top level API. Once we have a
> working implementation of that which has passed Boost peer review,
> everything else will follow naturally.
>
Fwiw, I have observed many examples of this purely “top-down” design over the decades. I have even seen some of them result in reasonably functional software. All too often, not even that. The result nearly always works, but often the top-level API dictates an inefficient implementation, and the top-level API gets locked-in by issues such as backwards compatibility and/or resource constraints.
>
> As I was saying to David Sankel and Roland Bock yesterday when we
> were discussing HTTP and BEAST, I really wouldn't care if you stuck
> O(N^N) complexity loops into your implementation and throw in a
> pointless sleep(500ms) for good measure in an initial implementation.
> *I don't care about the quality of the implementation*. I only care,
> for now, about the solid design of the top level API. Once we have a
> working implementation of that which has passed Boost peer review,
> everything else will follow naturally.
>
These are not hypothetical concerns. When a 2GHz 4-core dedicated computer can’t keep up with my wife’s typing speed, something has gone horribly wrong in many places up and down the software stack.
I am not alone in recognizing these shortcomings and many people have gravitated towards a “bottom-up” approach: Build a solid low-level foundation first which emphasizes performance. Higher-level software layers can add functionality. I have personally participated in several low-level projects with this “bottom-up” view: unique_ptr, mutex, condition_variable, thread, and more recently low-level date algorithms:
http://howardhinnant.github.io/date_algorithms.html
I’ve had considerable success with this approach, but it is not perfect. As I add the higher-level API I sometimes find a missing brick in the foundation. There is nothing like real-world testing of low-level foundation software to find missing or poorly specified functionality, and building that higher-level API upon the lower-level *is* that real-world testing.
If that higher-level testing can be done prior to setting the lower-level API in stone, one can achieve a valuable feedback loop where the lower-level level API can be improved while the high-level API is under development.
I have also found that a high-performance low-level API usually guides the higher-level design. It encourages the high-level API to not do things that are gratuitously inefficient, as long as the low-level API stands its ground on delegating expensive fancy features to the higher-level.
In the end, the very most important thing for software to be is correct. However, the next most important thing is run time performance:
http://howardhinnant.github.io/coding_guidelines.html
Correctness can be achieved with top-down or bottom-up. But bottom-up combined with early feedback from the top-down lends itself towards a software stack that is both fully functional and high performance.
<shameless plug>
This is the philosophy I have followed for the software stack discussed in this Cppcon 2016 talk:
http://schd.ws/hosted_files/cppcon2016/0f/Welcome%20To%20The%20Time%20Zone%20-%20Howard%20Hinnant%20-%20CppCon%202016.pdf
</shameless plug>
Howard
Robert Ramey
Sep 23, 2016, 4:18:43 PM9/23/16
to bo...@lists.boost.org
On 9/23/16 11:50 AM, Niall Douglas wrote:
> On 23 Sep 2016 at 7:58, Vinnie Falco wrote:
>
>> One point brought up during the conference is that Beast does not
>> offer sufficient higher level HTTP functionality. This omission is
>> intentional. It is my idea to offer the building blocks (serializing
>> and deserializing HTTP/1 messages, in synchronous and asynchronous
>> flavors, and a universal HTTP message model), upon which other
>> developers more knowledgeable than me could build those interfaces.
>
> Time to be mildly harsh.
>
> I'll repeat my main observation made during the Http review:
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++ (it need not be Python Requests specifically,
> just some well established highly regarded high level HTTP API battle
> tested over many years - the key part here is DO NOT try reinventing
> the wheel over well established practice).
I'm not understanding this point. Vinnie just said that it doesn't
> On 23 Sep 2016 at 7:58, Vinnie Falco wrote:
>
>> One point brought up during the conference is that Beast does not
>> offer sufficient higher level HTTP functionality. This omission is
>> intentional. It is my idea to offer the building blocks (serializing
>> and deserializing HTTP/1 messages, in synchronous and asynchronous
>> flavors, and a universal HTTP message model), upon which other
>> developers more knowledgeable than me could build those interfaces.
>
> Time to be mildly harsh.
>
> I'll repeat my main observation made during the Http review:
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++ (it need not be Python Requests specifically,
> just some well established highly regarded high level HTTP API battle
> tested over many years - the key part here is DO NOT try reinventing
> the wheel over well established practice).
currently offer more than a minimal level of functionality. Sounds to
me that you're agreeing with him
>
> We don't care about async, memory copying, performance, use of
> std::string, parsing, use of ASIO, serialisation or any of that other
> detail until somebody gets the bloody top end API implemented and
> stop wasting time on these currently unimportant implementation
> details.
>
> *I don't care about the quality of the implementation*. I only care,
> for now, about the solid design of the top level API. Once we have a
> working implementation of that which has passed Boost peer review,
> everything else will follow naturally.
Again, I'm not sure what side you're on here. But I will say that that
> *I don't care about the quality of the implementation*. I only care,
> for now, about the solid design of the top level API. Once we have a
> working implementation of that which has passed Boost peer review,
> everything else will follow naturally.
quality of implementation is the single most important thing. There's
two schools of thought here
a) Get your library to some advanced stage, post it and ask for feedback
before you invest a lot of effort in correctness, documentation, tests,
etc. The theory is that interested parties will contribute suggestions,
observations and fixes and help you along.
b) Make a completely finished but minimal library including tests, good
documentation, examples etc. Then hope that users try it, are happy
with what it does, then clamber for more features and enhancements.
c) Finish the whole damn thing
The worst is a) once someone downloads/clones your library and find that
it's hard to understand, has bugs or in any way frustrates you, they are
lost to you forever. They'll never come back.
The best of c) but that's very hard to do
b) Is the best most practical approach and that most likely to lead to
success. It's strongly supported by the incubator.
Robert Ramey
Vinnie Falco
Sep 23, 2016, 4:22:58 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 11:50 AM, Niall Douglas
<s_sour...@nedprod.com> wrote:
]> Almost everyone here and in the wider C++ community wants somebody to
C++ most definitely needs something like this! Look at these features
it has:
International Domains and URLs
Keep-Alive & Connection Pooling
Sessions with Cookie Persistence
Browser-style SSL Verification
Basic/Digest Authentication
Elegant Key/Value Cookies
Automatic Decompression
Automatic Content Decoding
Unicode Response Bodies
Multipart File Uploads
HTTP(S) Proxy Support
Connection Timeouts
Streaming Downloads
.netrc Support
Chunked Requests
Thread-safety
Custom Body Content Workflow
Streaming Uploads
POST Multiple Multipart-Encoded Files
Event Hooks
Custom Authentication
Streaming Requests
SOCKS proxies
Link Headers
Transport Adapters
Just look at that awesome list of features. The Python Requests
library is something we should hold up as a model of how we want a
full-featured HTTP client library to look and act.
Unfortunately, even a reduced subset of these features goes far beyond
my resources to implement. I also do not feel confident in my
knowledge of the domain or my abilities to provide a robust C++
interface for all of this.
I think that what Niall has done, probably inadvertently, is to
demonstrate just how broken the Boost review process has become. We
have Beast, which provides a great implementation of the low level
HTTP operations (read and write a message, provide a message model). I
am sure that someone or some group with expert knowledge in creating
robust HTTP clients could come along and build the C++ equivalent of
Python Requests on top of Beast. It should not be controversial when I
say that Beast offers useful functionality today.
And yet, there are strong opinions that Beast as a low level HTTP
building block is insufficient to be considered as part of a general
purpose library Boost. Once again I must ask, if Boost.Asio were
proposed today would it receive the same critique? Would an absence of
FTP and HTTP implementations make Christopher Kohlhoff's Asio library
get rejected in a formal review?
The modern consensus is that C++ libraries need to become more focused
and smaller, performing a single task and doing it really well. And
that is exactly the design principle of Beast - model the HTTP
message, serialize and deserialize HTTP messages synchronously or
asynchronously. This might not satisfy the majority of use cases but
it gives interested parties something they can work with. Who are the
interested parties? Anyone who wants to write a generic web server. Or
a full featured HTTP client. You want those features right, and you
want them in Boost? Then why would we reject a library that offers the
primitives for other people to create such high level implementations?
Thanks
<s_sour...@nedprod.com> wrote:
]> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++
Hello! I just had a look at this Requests library and it is AMAZING!
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++
C++ most definitely needs something like this! Look at these features
it has:
International Domains and URLs
Keep-Alive & Connection Pooling
Sessions with Cookie Persistence
Browser-style SSL Verification
Basic/Digest Authentication
Elegant Key/Value Cookies
Automatic Decompression
Automatic Content Decoding
Unicode Response Bodies
Multipart File Uploads
HTTP(S) Proxy Support
Connection Timeouts
Streaming Downloads
.netrc Support
Chunked Requests
Thread-safety
Custom Body Content Workflow
Streaming Uploads
POST Multiple Multipart-Encoded Files
Event Hooks
Custom Authentication
Streaming Requests
SOCKS proxies
Link Headers
Transport Adapters
Just look at that awesome list of features. The Python Requests
library is something we should hold up as a model of how we want a
full-featured HTTP client library to look and act.
Unfortunately, even a reduced subset of these features goes far beyond
my resources to implement. I also do not feel confident in my
knowledge of the domain or my abilities to provide a robust C++
interface for all of this.
I think that what Niall has done, probably inadvertently, is to
demonstrate just how broken the Boost review process has become. We
have Beast, which provides a great implementation of the low level
HTTP operations (read and write a message, provide a message model). I
am sure that someone or some group with expert knowledge in creating
robust HTTP clients could come along and build the C++ equivalent of
Python Requests on top of Beast. It should not be controversial when I
say that Beast offers useful functionality today.
And yet, there are strong opinions that Beast as a low level HTTP
building block is insufficient to be considered as part of a general
purpose library Boost. Once again I must ask, if Boost.Asio were
proposed today would it receive the same critique? Would an absence of
FTP and HTTP implementations make Christopher Kohlhoff's Asio library
get rejected in a formal review?
The modern consensus is that C++ libraries need to become more focused
and smaller, performing a single task and doing it really well. And
that is exactly the design principle of Beast - model the HTTP
message, serialize and deserialize HTTP messages synchronously or
asynchronously. This might not satisfy the majority of use cases but
it gives interested parties something they can work with. Who are the
interested parties? Anyone who wants to write a generic web server. Or
a full featured HTTP client. You want those features right, and you
want them in Boost? Then why would we reject a library that offers the
primitives for other people to create such high level implementations?
Thanks
Vinnie Falco
Sep 23, 2016, 4:27:34 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 1:18 PM, Robert Ramey <ra...@rrsd.com> wrote:
> On 9/23/16 11:50 AM, Niall Douglas wrote:
>> ...
> On 9/23/16 11:50 AM, Niall Douglas wrote:
>> Almost everyone here and in the wider C++ community wants somebody to
>> take the HTTP client API from the Python Requests module
>> (http://docs.python-requests.org/en/master/) and very closely
>> replicate it in C++
>
>> take the HTTP client API from the Python Requests module
>> (http://docs.python-requests.org/en/master/) and very closely
>> replicate it in C++
>
> I'm not understanding this point. Vinnie just said that it doesn't
> currently offer more than a minimal level of functionality. Sounds to me
> that you're agreeing with him
He's definitely disagreeing. Beast does these things:
> currently offer more than a minimal level of functionality. Sounds to me
> that you're agreeing with him
* Serialize HTTP/1 messages (sync or async)
* Deserialize HTTP/1 messages (sync or async)
* Model HTTP messages universally
This is the lowest level of functionality a library can offer with
respect to HTTP. All higher level operations can be expressed in terms
of these primitives. The corollary is all higher level operations are
impossible without any of these operations.
Niall and others are saying that a library offering the low level
building blocks of HTTP is insufficient to become part of Boost. He
and others believe that common use cases such as a full features HTTP
client or a generic asynchronous threaded HTTP server need to be part
of any approved Boost library that claims to solve any HTTP-related
problems.
I disagree, and I hope a sufficient number of other folks disagree as well.
Thanks
Andrey Semashev
Sep 23, 2016, 4:37:30 PM9/23/16
to bo...@lists.boost.org
On 09/23/16 21:50, Niall Douglas wrote:
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++
I'd like to ask you to not speak for others.
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++
Glen Fernandes
Sep 23, 2016, 4:51:25 PM9/23/16
to bo...@lists.boost.org
Niall Douglas wrote:
> Almost everyone here and in the wider C++ community wants [...]
Niall Douglas wrote:
> We don't care [...]
This familiar pattern returns.
Vinnie Falco wrote:
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model).
you have said is controversial.
In my opinion (and I speak for myself, not everyone here or the wider C++
community):
1. You have a decent library, both in terms of design and implementation.
Like
Paul, I suggest you put it up for review whenever you feel comfortable. [1]
2. You should keep the focus in your discussions on your library and not on
the Boost review process. If you have interest in improving the latter, do
it
independently of your effort to have your library reviewed.
Glen
[1] If you require more discussion about the interface to reach that level
of
comfort, you've already taken the necessary steps by soliciting feedback
here.
I would just trust the enthusiasm/interest by others in your library to do
the
rest.
--
View this message in context: http://boost.2283326.n4.nabble.com/beast-Request-for-Discussion-tp4688242p4688256.html
Sent from the Boost - Dev mailing list archive at Nabble.com.
> Almost everyone here and in the wider C++ community wants [...]
Niall Douglas wrote:
> We don't care [...]
This familiar pattern returns.
Vinnie Falco wrote:
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model).
> It should not be controversial when I say that Beast offers useful
> functionality today.
I don't believe Niall has demonstrated anything. I also do not think what
> functionality today.
you have said is controversial.
In my opinion (and I speak for myself, not everyone here or the wider C++
community):
1. You have a decent library, both in terms of design and implementation.
Like
Paul, I suggest you put it up for review whenever you feel comfortable. [1]
2. You should keep the focus in your discussions on your library and not on
the Boost review process. If you have interest in improving the latter, do
it
independently of your effort to have your library reviewed.
Glen
[1] If you require more discussion about the interface to reach that level
of
comfort, you've already taken the necessary steps by soliciting feedback
here.
I would just trust the enthusiasm/interest by others in your library to do
the
rest.
--
View this message in context: http://boost.2283326.n4.nabble.com/beast-Request-for-Discussion-tp4688242p4688256.html
Sent from the Boost - Dev mailing list archive at Nabble.com.
Andrey Semashev
Sep 23, 2016, 5:11:31 PM9/23/16
to bo...@lists.boost.org
your submission (or underestimate the positive effect of community
review). I'm sure some people, myself included, think that designing
HTTP/FTP/whatever protocol from bottom up is a totally reasonable approach.
It may be difficult during the review to show the immediate practical
gain from a lower layer component, but if you can demonstrate how a
higher level layer could benefit from your library, you should be fine.
If you can show that porting a library like Boost.HTTP to Beast is
possible and beneficial, that would be even better. I hope you and
Vinicius Oliveira will be able to talk and cooperate on this matter.
Peter Dimov
Sep 23, 2016, 5:17:37 PM9/23/16
to bo...@lists.boost.org
Vinnie Falco wrote:
...
library more than meets Boost's criteria for inclusion.
You need to not let these objections frustrate you. The basic problem here
is that C++ programmers still, in $current_year, have no standard way of
opening a stream to https://whatever.com/archive.tar.gz and reading it. This
is incredibly annoying. Yes, you're right that a simple API would mean that
I can't pass a client certificate to it. At this point though anything,
ANYTHING, is better than what we have, which is nil.
So it comes to no surprise that whenever the words "Boost", "HTTP" and
"library" occur together in a sentence, everyone (to a first approximation)
hopes that we'll finally get that simple API.
You don't have to provide something like that, and this is not a
prerequisite for accepting the lower-level Beast. But many people will be
happy if you do.
Plus, there's also the matter that to evaluate the lower-level API, you have
to have something built on top of it with which to evaluate it.
...
> This is the lowest level of functionality a library can offer with respect
> to HTTP. All higher level operations can be expressed in terms of these
> primitives. The corollary is all higher level operations are impossible
> without any of these operations.
I'm fairly sure - without even having to look - that the proposed Beast
> to HTTP. All higher level operations can be expressed in terms of these
> primitives. The corollary is all higher level operations are impossible
> without any of these operations.
library more than meets Boost's criteria for inclusion.
You need to not let these objections frustrate you. The basic problem here
is that C++ programmers still, in $current_year, have no standard way of
opening a stream to https://whatever.com/archive.tar.gz and reading it. This
is incredibly annoying. Yes, you're right that a simple API would mean that
I can't pass a client certificate to it. At this point though anything,
ANYTHING, is better than what we have, which is nil.
So it comes to no surprise that whenever the words "Boost", "HTTP" and
"library" occur together in a sentence, everyone (to a first approximation)
hopes that we'll finally get that simple API.
You don't have to provide something like that, and this is not a
prerequisite for accepting the lower-level Beast. But many people will be
happy if you do.
Plus, there's also the matter that to evaluate the lower-level API, you have
to have something built on top of it with which to evaluate it.
Vinnie Falco
Sep 23, 2016, 5:25:42 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 1:46 PM, Glen Fernandes
<glen.fe...@gmail.com> wrote:
> ...
> 2. You should keep the focus in your discussions on your library and not on
> the Boost review process.
That's good advice. I will table the issue of whether Beast should
offer more high level operations such as a full featured HTTP client
or general purpose asynchronous server.
> [1] If you require more discussion about the interface to reach that level
> of comfort, you've already taken the necessary steps by soliciting feedback
> here.
Yes there are a few points that should be addressed:
So far there has been a lot of talk of HTTP and not a word about
WebSocket. Does this mean that Beast's WebSocket interface is
non-controversial?
What about the stuff in here:
https://github.com/vinniefalco/Beast/tree/master/include/beast/core
These currently are official interfaces, and I would like to keep them
that way. In fact, I think some of them should be part of the
Networking TS (buffer_cat in particular). Is this a problem? What
namespace would they go into? e.g. boost::buffer_cat?
boost::beast::buffer_cat? boost::asio::buffer_cat (that would probably
be the best choice but it injects names into someone else's namespace)
What would the library be called in Boost? Not Boost.Http. Would it be
Boost.Beast? I do plan on adding more to this library such as SOCKS
proxy support (that would be a new protocol). Would we just rename
beast to boost, i.e. boost::http::async_read, boost::http::message?
What happens if someone else puts together a full feature HTTP client
in a new proposed lbrary, what would they call it?
boost::http::client?
HTTP/2 support came up as a blocker in last years reviews. Beast
design does take HTTP/2 into account. Objects which are HTTP/1 only
are clearly marked (e.g. beast::http::message_v1) and HTTP/1 specific
algorithms are overloaded on those specific types. Are these points
obstacles for review?
How do I start the formal review process?
<glen.fe...@gmail.com> wrote:
> ...
> 1. You have a decent library, both in terms of design and implementation.
> Like Paul, I suggest you put it up for review whenever you feel comfortable.
Thank you for the kind words.
> Like Paul, I suggest you put it up for review whenever you feel comfortable.
> 2. You should keep the focus in your discussions on your library and not on
> the Boost review process.
offer more high level operations such as a full featured HTTP client
or general purpose asynchronous server.
> [1] If you require more discussion about the interface to reach that level
> of comfort, you've already taken the necessary steps by soliciting feedback
> here.
So far there has been a lot of talk of HTTP and not a word about
WebSocket. Does this mean that Beast's WebSocket interface is
non-controversial?
What about the stuff in here:
https://github.com/vinniefalco/Beast/tree/master/include/beast/core
These currently are official interfaces, and I would like to keep them
that way. In fact, I think some of them should be part of the
Networking TS (buffer_cat in particular). Is this a problem? What
namespace would they go into? e.g. boost::buffer_cat?
boost::beast::buffer_cat? boost::asio::buffer_cat (that would probably
be the best choice but it injects names into someone else's namespace)
What would the library be called in Boost? Not Boost.Http. Would it be
Boost.Beast? I do plan on adding more to this library such as SOCKS
proxy support (that would be a new protocol). Would we just rename
beast to boost, i.e. boost::http::async_read, boost::http::message?
What happens if someone else puts together a full feature HTTP client
in a new proposed lbrary, what would they call it?
boost::http::client?
HTTP/2 support came up as a blocker in last years reviews. Beast
design does take HTTP/2 into account. Objects which are HTTP/1 only
are clearly marked (e.g. beast::http::message_v1) and HTTP/1 specific
algorithms are overloaded on those specific types. Are these points
obstacles for review?
How do I start the formal review process?
Vladimir Batov
Sep 23, 2016, 6:55:24 PM9/23/16
to bo...@lists.boost.org
On 2016-09-24 04:50, Niall Douglas wrote:
> ...
> ... and very few here would care how it is internally
confronting way but I have to certainly agree with his main point...
Most likely not in as blunt as Nial made it sound manner but
nevertheless.
From my experience Boost reviews indeed often put the cart before the
horse and get into fierce irreconcilable fights and ultimatums over
implementation minutia while totally losing sight or even ignoring the
design and API -- the two fundamental components essential for lib
acceptance and longevity. The design allows the lib to evolve and grow;
and the API... well, it's obvious.
Indeed, other posters indicated that that approach taken to the extreme
may lead to inefficient implementation, etc. However, such criticism can
be applied to probably anything. The "extreme" argument appears to be a
simple/quick/dirty trick to misrepresent and then to discredit an
idea... any idea.
So, nobody is (or should be) expecting a complete, fully working, fully
tested, exhaustively documented proposal on a plate. However, that is
where the wisdom and experience (IMO) of the Boost community must come
in -- to see the potential of a proposal, to see if the proposal has the
right building blocks in the right places and has the room to grow in
the right direction, i.e. that the proposal shows the right design (as,
I think, Nial tried to convey) to eventually grow and to satisfy user
expectations. Indeed, such an evaluation is hard. It is so much more
"fun" to behave like a spoiled child -- I want the proposal to do X for
me, I do not like/accept how Y is implemented/called. IMO such a
reception, attitude and expectations are what drives people away from
contributing to or participating in Boost. IMO in that regard the
experience of Boost.Convert was quite positive when we somehow managed
to stay focused on the design (or so I perceived), the proposal was
accepted in principle and then a group of enthusiasts made sure the
proposal addressed criticism, implemented/accommodated (or had room for)
the requested functionality.
Now back to the actual HTTP-lib proposal. I am merely a user. However, I
do want very much such a functionality in Boost. If Boosters more
knowledgeable in the field might get together, evaluate the proposal in
a realistic and mature manner and, ultimately, manage to plant a seed in
Boost from which a good Boost HTTP lib. might start growing, that be
delightful. Again, let's avoid "I want a full-grown tree or nothing".
Let's evaluate (and accept) if that's a good seed that something good
might grow from... and make sure it does...
Or alternatively we might try to realistically evaluate the situation
and, maybe, to accept that Boost is somewhat late to the HTTP "party"
and not to go down that HTTP road due to existing, say, Poco HTTP
library.
> ...
> Time to be mildly harsh.
> ...
> ... and very few here would care how it is internally
> implemented so long as the public API design is solid - THEN the
> refinement of implementation can begin.
>
> ... the solid design of the top level API. Once we have a
> refinement of implementation can begin.
>
> working implementation of that which has passed Boost peer review,
> everything else will follow naturally.
Niall might have expressed his view in a somewhat extreme and
> everything else will follow naturally.
confronting way but I have to certainly agree with his main point...
Most likely not in as blunt as Nial made it sound manner but
nevertheless.
From my experience Boost reviews indeed often put the cart before the
horse and get into fierce irreconcilable fights and ultimatums over
implementation minutia while totally losing sight or even ignoring the
design and API -- the two fundamental components essential for lib
acceptance and longevity. The design allows the lib to evolve and grow;
and the API... well, it's obvious.
Indeed, other posters indicated that that approach taken to the extreme
may lead to inefficient implementation, etc. However, such criticism can
be applied to probably anything. The "extreme" argument appears to be a
simple/quick/dirty trick to misrepresent and then to discredit an
idea... any idea.
So, nobody is (or should be) expecting a complete, fully working, fully
tested, exhaustively documented proposal on a plate. However, that is
where the wisdom and experience (IMO) of the Boost community must come
in -- to see the potential of a proposal, to see if the proposal has the
right building blocks in the right places and has the room to grow in
the right direction, i.e. that the proposal shows the right design (as,
I think, Nial tried to convey) to eventually grow and to satisfy user
expectations. Indeed, such an evaluation is hard. It is so much more
"fun" to behave like a spoiled child -- I want the proposal to do X for
me, I do not like/accept how Y is implemented/called. IMO such a
reception, attitude and expectations are what drives people away from
contributing to or participating in Boost. IMO in that regard the
experience of Boost.Convert was quite positive when we somehow managed
to stay focused on the design (or so I perceived), the proposal was
accepted in principle and then a group of enthusiasts made sure the
proposal addressed criticism, implemented/accommodated (or had room for)
the requested functionality.
Now back to the actual HTTP-lib proposal. I am merely a user. However, I
do want very much such a functionality in Boost. If Boosters more
knowledgeable in the field might get together, evaluate the proposal in
a realistic and mature manner and, ultimately, manage to plant a seed in
Boost from which a good Boost HTTP lib. might start growing, that be
delightful. Again, let's avoid "I want a full-grown tree or nothing".
Let's evaluate (and accept) if that's a good seed that something good
might grow from... and make sure it does...
Or alternatively we might try to realistically evaluate the situation
and, maybe, to accept that Boost is somewhat late to the HTTP "party"
and not to go down that HTTP road due to existing, say, Poco HTTP
library.
Edward Diener
Sep 23, 2016, 7:32:39 PM9/23/16
to bo...@lists.boost.org
process has become". There are numerous Boost libraries which serve as
building blocks for higher level libraries and these low-level libraries
are just as important for Boost as a higher-level library which might
appeal more to end-users in application development.
>
> The modern consensus is that C++ libraries need to become more focused
> and smaller, performing a single task and doing it really well. And
> that is exactly the design principle of Beast - model the HTTP
> message, serialize and deserialize HTTP messages synchronously or
> asynchronously. This might not satisfy the majority of use cases but
> it gives interested parties something they can work with. Who are the
> interested parties? Anyone who wants to write a generic web server. Or
> a full featured HTTP client. You want those features right, and you
> want them in Boost? Then why would we reject a library that offers the
> primitives for other people to create such high level implementations?
library. Niall's opinion about your library is his own. Don't push that
opinion on other Boost developers who may feel differently.
Vinnie Falco
Sep 23, 2016, 7:58:59 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 4:31 PM, Edward Diener <eldi...@tropicsoft.com> wrote:
> I really object to your characterization of "how broken the Boost review
> process has become".
> ...
> I really object to your characterization of "how broken the Boost review
> process has become".
> Niall's opinion about your library is his own. Don't push that opinion on
> other Boost developers who may feel differently.
Yup, and my apologies for misreading the situation. When I first
> other Boost developers who may feel differently.
announced Beast, there were months of radio silence. But the great
support and advice that I've received in this thread has been very
helpful.
> We don't reject such a library if we find it useful as a low-level library.
recall, you participated in last year's review of Boost.Http.
Thanks
Glen Fernandes
Sep 23, 2016, 8:52:44 PM9/23/16
to bo...@lists.boost.org
On Fri, Sep 23, 2016 at 5:25 PM, Vinnie Falco wrote:
> So far there has been a lot of talk of HTTP and not a word about
> WebSocket. Does this mean that Beast's WebSocket interface is
> non-controversial?
Not necessarily; I would take away from that only that HTTP is the
> So far there has been a lot of talk of HTTP and not a word about
> WebSocket. Does this mean that Beast's WebSocket interface is
> non-controversial?
area of interest enough to evoke initial discussion. i.e. During the
review, anticipate/prepare for any portion, be it your WebSocket class
template, or even the implementation of types in your detail and impl
namespaces to be scrutinized.
> What would the library be called in Boost? Not Boost.Http. Would it be
> Boost.Beast?
something named Boost.Http; e.g. Boost.Http.Core.
If WebSockets and Http both live under the same Boost library, which I
believe is your intention, then your current organization of
Boost.<Library>.Http and Boost.<Library>.WebSocket makes a lot of
sense. I'd prefer that <Library> identifier should be something less
abstract than "Beast".
> I do plan on adding more to this library such as SOCKS
> proxy support (that would be a new protocol). Would we just rename
> beast to boost, i.e. boost::http::async_read, boost::http::message?
encompass both your lower level functionality, as well as higher level
utilities. Good to hear about the desire to provide a SOCKS
implementation.
> What happens if someone else puts together a full feature HTTP client
> in a new proposed lbrary, what would they call it?
> boost::http::client?
> HTTP/2 support came up as a blocker in last years reviews. Beast
> design does take HTTP/2 into account. Objects which are HTTP/1 only
> are clearly marked (e.g. beast::http::message_v1) and HTTP/1 specific
> algorithms are overloaded on those specific types. Are these points
> obstacles for review?
during review it would be a point raised against acceptance, or
conditional-acceptance, I wouldn't want to make that prediction.
You've already done the right thing by designing the library for
users, not designing the library to pass the Boost formal review
process, so the review will just give you an opportunity to:
- Determine whether there is interest in the library as it is for
inclusion into Boost
- If conditionally, or if not, identify what the community that
participates does want in your library
- Decide whether you feel it is in the best interest of the library to
accommodate those requests (either now, or over time)
> How do I start the formal review process?
to help you with this.
2. Find a someone from the community (e.g. Boost developers)
interested in managing the review
Then just work with the review manager to schedule the review.
Glen
Niall Douglas
Sep 24, 2016, 11:23:33 AM9/24/16
to bo...@lists.boost.org
On 23 Sep 2016 at 13:22, Vinnie Falco wrote:
> ]> Almost everyone here and in the wider C++ community wants somebody to
> > take the HTTP client API from the Python Requests module
> > (http://docs.python-requests.org/en/master/) and very closely
> > replicate it in C++
> [snip]
> ]> Almost everyone here and in the wider C++ community wants somebody to
> > take the HTTP client API from the Python Requests module
> > (http://docs.python-requests.org/en/master/) and very closely
> > replicate it in C++
> Unfortunately, even a reduced subset of these features goes far beyond
> my resources to implement. I also do not feel confident in my
> knowledge of the domain or my abilities to provide a robust C++
> interface for all of this.
As I mentioned to you yesterday, wrapping the Python implementation
> my resources to implement. I also do not feel confident in my
> knowledge of the domain or my abilities to provide a robust C++
> interface for all of this.
in Boost.Python and exporting it as C++ solves the cost of
implementation problem. I'd even consider a more exciting method of
transducing implementation say via static conversion of Python to C
or C++ via adapting one of the Python compilers.
Most Boost library authors want to build cathedrals their own way,
and that's fine, it gets you up in the morning to write code you're
not getting paid for. However from an end user's perspective, they
*really do not care* how a library is implemented so long as it
works, has reasonable documentation and has reasonable performance.
Naval gazing or "purity" by library authors is fairly unimportant to
almost all of them. Given the very high popularity of Python
Requests, I do not doubt a very similar C++ library would be
immensely popular in the C++ world, even if its implementation is
actually the Python code itself.
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model). I
> am sure that someone or some group with expert knowledge in creating
> robust HTTP clients could come along and build the C++ equivalent of
> Python Requests on top of Beast. It should not be controversial when I
> say that Beast offers useful functionality today.
definition.
I also don't think it controversial that a HTTP utilities library
would be a valuable addition to Boost. What I did say to you, and
others strongly agreed with me at CppCon if you remember, is that
it's very hard to get something like a HTTP low level library
usefully past a Boost peer review. There are umpteen ways of
implementing it, everyone has their own opinion, and a lot of people
otherwise intelligent will argue and comment from a profoundly
ignorant base of understanding. It'll be hard to decide between the
really valuable feedback from those who really understand this niche
domain from the rest who really don't know what they're talking
about.
I don't think this means the Boost peer review is broken, just that
it fits badly to really niche libraries, and a well designed low
level HTTP client library is a very niche domain very few will ever
use. We've already seen some reviews having only three or four
reviewers because the topic is such a niche domain that most of us
here can't usefully comment. For the heavy maths libraries, most here
realise their opinions are close to worthless and they stay away. For
low level multi purpose libraries like HTTP ones, that tends to not
happen because people think themselves competent which makes the
review process not as valuable nor work as well as one would wish
for.
> The modern consensus is that C++ libraries need to become more focused
> and smaller, performing a single task and doing it really well. And
> that is exactly the design principle of Beast - model the HTTP
> message, serialize and deserialize HTTP messages synchronously or
> asynchronously. This might not satisfy the majority of use cases but
> it gives interested parties something they can work with. Who are the
> interested parties? Anyone who wants to write a generic web server. Or
> a full featured HTTP client. You want those features right, and you
> want them in Boost? Then why would we reject a library that offers the
> primitives for other people to create such high level implementations?
cathedrals. It's what most of us are here to do.
However you have to be realistic. If it seems to take as long for an
end user to learn off your low level HTTP framework as to quickly
hack something bespoke together, they are going to choose the latter.
That outcome is *extremely* likely for any low level HTTP framework.
End users are simply not going to invest the effort to master the
framework you've invested years of your life into unless they are
highly and immediately convinced it will save them a lot of time and
effort.
A high level HTTP client library has a very visible and clear cost
benefit ratio. Any semi decent implementation is going to see rapid
and very widespread adoption because most people just want to go
fetch the bloody HTTP page already.
A low level HTTP client, or server library is always going to have to
fight the not-invented-here syndrome and the hard reality that the
majority of end users will roll their own half baked solution rather
than take the time to master your inevitably complex and brain
hurting framework.
Again, that's fine. Plenty of Boost libraries have been usefully
inspirational rather than actually used. The biggest contribution
Boost made to the C++ community past the standardised facilities was
a source code implementation study reference manual. Your cathedral
won't necessarily be used much, but a lot of people will derive
benefit from it once it's into Boost by studying it when writing
their own code.
In the end you'll need to decide, as all of us have done, whether you
want to write a library end users really want to use and will adopt
in droves and will really solve a big pressing problem for the C++
community, or whether you want to write a library which is a
beautiful and pure cathedral stunning and inspirational to all who
behold it, but rarely enter nor use it. That's absolutely a personal
decision. I'll support you whichever you choose to the best of my
capabilities.
Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/
http://ie.linkedin.com/in/nialldouglas/
Edward Diener
Sep 24, 2016, 12:22:36 PM9/24/16
to bo...@lists.boost.org
On 9/24/2016 11:22 AM, Niall Douglas wrote:
> On 23 Sep 2016 at 13:22, Vinnie Falco wrote:
>
>> ]> Almost everyone here and in the wider C++ community wants somebody to
>>> take the HTTP client API from the Python Requests module
>>> (http://docs.python-requests.org/en/master/) and very closely
>>> replicate it in C++
>> [snip]
>> Unfortunately, even a reduced subset of these features goes far beyond
>> my resources to implement. I also do not feel confident in my
>> knowledge of the domain or my abilities to provide a robust C++
>> interface for all of this.
>
> As I mentioned to you yesterday, wrapping the Python implementation
> in Boost.Python and exporting it as C++ solves the cost of
> implementation problem. I'd even consider a more exciting method of
> transducing implementation say via static conversion of Python to C
> or C++ via adapting one of the Python compilers.
>
> Most Boost library authors want to build cathedrals their own way,
> and that's fine, it gets you up in the morning to write code you're
> not getting paid for. However from an end user's perspective, they
> *really do not care* how a library is implemented so long as it
> works, has reasonable documentation and has reasonable performance.
I agree with this but there is nothing which prevents a low-level
> On 23 Sep 2016 at 13:22, Vinnie Falco wrote:
>
>> ]> Almost everyone here and in the wider C++ community wants somebody to
>>> take the HTTP client API from the Python Requests module
>>> (http://docs.python-requests.org/en/master/) and very closely
>>> replicate it in C++
>> [snip]
>> Unfortunately, even a reduced subset of these features goes far beyond
>> my resources to implement. I also do not feel confident in my
>> knowledge of the domain or my abilities to provide a robust C++
>> interface for all of this.
>
> As I mentioned to you yesterday, wrapping the Python implementation
> in Boost.Python and exporting it as C++ solves the cost of
> implementation problem. I'd even consider a more exciting method of
> transducing implementation say via static conversion of Python to C
> or C++ via adapting one of the Python compilers.
>
> Most Boost library authors want to build cathedrals their own way,
> and that's fine, it gets you up in the morning to write code you're
> not getting paid for. However from an end user's perspective, they
> *really do not care* how a library is implemented so long as it
> works, has reasonable documentation and has reasonable performance.
library from fulfilling another programmer's needs. In other words it is
just as important that a low-level library provides a good design for
others to build upon as it is for a higher-level library to provide a
good design for end-user's to use in their applications.
been immensely used by other Boost developers. MPL is the most obvious
candidate for that, but many developers use other low-level Boost
libraries in their own library.
>
> In the end you'll need to decide, as all of us have done, whether you
> want to write a library end users really want to use and will adopt
> in droves and will really solve a big pressing problem for the C++
> community, or whether you want to write a library which is a
> beautiful and pure cathedral stunning and inspirational to all who
> behold it, but rarely enter nor use it. That's absolutely a personal
> decision. I'll support you whichever you choose to the best of my
> capabilities.
may be enough for that library to interest others but if the low-level
library is not seen as practically useful as something other developers
can build upon it will probably not become part of Boost.
Michael Caisse
Sep 24, 2016, 2:18:18 PM9/24/16
to bo...@lists.boost.org
On 9/23/16 13:22, Vinnie Falco wrote:
> On Fri, Sep 23, 2016 at 11:50 AM, Niall Douglas
> <s_sour...@nedprod.com> wrote:
> ]> Almost everyone here and in the wider C++ community wants somebody to
>> take the HTTP client API from the Python Requests module
>> (http://docs.python-requests.org/en/master/) and very closely
>> replicate it in C++
>
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model). I
> am sure that someone or some group with expert knowledge in creating
> robust HTTP clients could come along and build the C++ equivalent of
> Python Requests on top of Beast. It should not be controversial when I
> say that Beast offers useful functionality today.
>
> And yet, there are strong opinions that Beast as a low level HTTP
> building block is insufficient to be considered as part of a general
> purpose library Boost. Once again I must ask, if Boost.Asio were
> proposed today would it receive the same critique? Would an absence of
> FTP and HTTP implementations make Christopher Kohlhoff's Asio library
> get rejected in a formal review?
review process works. Focused libraries that do something well and can
be used as building blocks for other libraries and applications is
exactly what we need more.
michael
--
Michael Caisse
Ciere Consulting
ciere.com
Paul A. Bristow
Sep 25, 2016, 4:22:28 AM9/25/16
to bo...@lists.boost.org
> -----Original Message-----
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Michael Caisse
> Sent: 24 September 2016 19:18
> To: bo...@lists.boost.org
> Subject: Re: [boost] [beast] Request for Discussion
>
> Don't let a few distractors get you down. That isn't how the boost
> review process works. Focused libraries that do something well and can
> be used as building blocks for other libraries and applications is
> exactly what we need more.
+1
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Michael Caisse
> Sent: 24 September 2016 19:18
> To: bo...@lists.boost.org
> Subject: Re: [boost] [beast] Request for Discussion
>
> Don't let a few distractors get you down. That isn't how the boost
> review process works. Focused libraries that do something well and can
> be used as building blocks for other libraries and applications is
> exactly what we need more.
Bjorn Reese
Sep 25, 2016, 6:57:02 AM9/25/16
to bo...@lists.boost.org
On 09/23/2016 06:25 PM, Vinnie Falco wrote:
> I would like to hear from as many of these individuals as possible,
> about their experience looking at Beast and taking it for a test
> drive, before starting the formal review process.
Apart from usual difficulties, your library is facing an additional
> I would like to hear from as many of these individuals as possible,
> about their experience looking at Beast and taking it for a test
> drive, before starting the formal review process.
complication that as not been brought up yet.
Boost is more than a set of high-quality libraries and a review process.
We are also trying to nurture promising student developers, mainly
through GSoC.
Your library happens to coincide with one of those projects:
Boost.Http. This project started as a GSoC project about two-three
years ago, and this year it was part of the Boost sponsored summer of
code (BSoC), which resulted in a HTTP parser.
If we had simply wanted a HTTP library, I could easily have written
that myself (I am one of the original curl developers), but I chose
to support the nuture program and serve as mentor instead.
Your library unintentionally puts us in a precarious situation with
regards to this nuture program. That is one of the reasons why I
initially suggested that you and Vinicius collaborated instead of
seeing your respective libraries as competitors.
Vinnie Falco
Sep 25, 2016, 8:27:09 AM9/25/16
to bo...@lists.boost.org
On Sun, Sep 25, 2016 at 7:01 AM, Bjorn Reese <bre...@mail1.stofanet.dk> wrote:
> ...I initially suggested that you and Vinicius collaborated instead
Me and Vinicius spent several hours talking about our respective
libraries on Skype. I reported back to the group on this:
http://lists.boost.org/Archives/boost/2016/04/229337.php
I noted that Boost.Http and Beast had minimal overlap. Beast provides
low level HTTP protocol functionality while Boost.Http offers a
turn-key HTTP server. From that discussion came some action items, I
made changes to the Beast message model. I delivered a header-only
parser with no dependencies on the rest of Beast so it could be
re-used in Boost.Http, and informed Vinicius. I started a discussion
on further refinements to the message model
(https://github.com/vinniefalco/Beast/issues/13). I have seen no
evidence of Beast integration in Boost.Http.
> ...I initially suggested that you and Vinicius collaborated instead
> of seeing your respective libraries as competitors.
Absolutely, which is why I reached out to you and Vinicius in April.
Me and Vinicius spent several hours talking about our respective
libraries on Skype. I reported back to the group on this:
http://lists.boost.org/Archives/boost/2016/04/229337.php
I noted that Boost.Http and Beast had minimal overlap. Beast provides
low level HTTP protocol functionality while Boost.Http offers a
turn-key HTTP server. From that discussion came some action items, I
made changes to the Beast message model. I delivered a header-only
parser with no dependencies on the rest of Beast so it could be
re-used in Boost.Http, and informed Vinicius. I started a discussion
on further refinements to the message model
(https://github.com/vinniefalco/Beast/issues/13). I have seen no
evidence of Beast integration in Boost.Http.
Rob Stewart
Sep 25, 2016, 8:38:04 AM9/25/16
to bo...@lists.boost.org, Peter Dimov
On September 23, 2016 5:17:08 PM EDT, Peter Dimov <li...@pdimov.com> wrote:
> Vinnie Falco wrote:
> ...
> > This is the lowest level of functionality a library can offer with
> respect
> > to HTTP. All higher level operations can be expressed in terms of
> these
> > primitives. The corollary is all higher level operations are
> impossible
> > without any of these operations.
[snip important points about expectations and focus]
> Vinnie Falco wrote:
> ...
> > This is the lowest level of functionality a library can offer with
> respect
> > to HTTP. All higher level operations can be expressed in terms of
> these
> > primitives. The corollary is all higher level operations are
> impossible
> > without any of these operations.
> Plus, there's also the matter that to evaluate the lower-level API,
> you have
> to have something built on top of it with which to evaluate it.
--
Rob
[1] https://tools.ietf.org/html/rfc3439#section-3
[2] https://github.com/real-logic/Aeron
(Sent from my portable computation device.)
Thomas Trummer
Sep 25, 2016, 9:39:38 AM9/25/16
to bo...@lists.boost.org
> On 23 Sep 2016, at 20:50, Niall Douglas <s_sour...@nedprod.com> wrote:
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++ (it need not be Python Requests specifically,
> just some well established highly regarded high level HTTP API battle
> tested over many years - the key part here is DO NOT try reinventing
> the wheel over well established practice).
>
> Almost everyone here and in the wider C++ community wants somebody to
> take the HTTP client API from the Python Requests module
> (http://docs.python-requests.org/en/master/) and very closely
> replicate it in C++ (it need not be Python Requests specifically,
> just some well established highly regarded high level HTTP API battle
> tested over many years - the key part here is DO NOT try reinventing
> the wheel over well established practice).
I don’t know if this is true but such a thing actually exists already:
- https://www.youtube.com/watch?v=f_D-wD1EmWk
- https://github.com/whoshuu/cpr
Thomas
Niall Douglas
Sep 27, 2016, 7:40:55 AM9/27/16
to bo...@lists.boost.org
On 25 Sep 2016 at 8:26, Vinnie Falco wrote:
> On Sun, Sep 25, 2016 at 7:01 AM, Bjorn Reese <bre...@mail1.stofanet.dk> wrote:
> > ...I initially suggested that you and Vinicius collaborated instead
> > of seeing your respective libraries as competitors.
>
> Absolutely, which is why I reached out to you and Vinicius in April.
> Me and Vinicius spent several hours talking about our respective
> libraries on Skype. I reported back to the group on this:
> http://lists.boost.org/Archives/boost/2016/04/229337.php
>
> I noted that Boost.Http and Beast had minimal overlap. Beast provides
> low level HTTP protocol functionality while Boost.Http offers a
> turn-key HTTP server. From that discussion came some action items, I
> made changes to the Beast message model. I delivered a header-only
> parser with no dependencies on the rest of Beast so it could be
> re-used in Boost.Http, and informed Vinicius. I started a discussion
> on further refinements to the message model
> (https://github.com/vinniefalco/Beast/issues/13). I have seen no
> evidence of Beast integration in Boost.Http.
My thanks to Bjorn for raising the GSoC point. I should clarify that
> On Sun, Sep 25, 2016 at 7:01 AM, Bjorn Reese <bre...@mail1.stofanet.dk> wrote:
> > ...I initially suggested that you and Vinicius collaborated instead
> > of seeing your respective libraries as competitors.
>
> Absolutely, which is why I reached out to you and Vinicius in April.
> Me and Vinicius spent several hours talking about our respective
> libraries on Skype. I reported back to the group on this:
> http://lists.boost.org/Archives/boost/2016/04/229337.php
>
> I noted that Boost.Http and Beast had minimal overlap. Beast provides
> low level HTTP protocol functionality while Boost.Http offers a
> turn-key HTTP server. From that discussion came some action items, I
> made changes to the Beast message model. I delivered a header-only
> parser with no dependencies on the rest of Beast so it could be
> re-used in Boost.Http, and informed Vinicius. I started a discussion
> on further refinements to the message model
> (https://github.com/vinniefalco/Beast/issues/13). I have seen no
> evidence of Beast integration in Boost.Http.
whilst a unified library would be preferred by all, in the end there
is nothing stopping BOTH libraries entering Boost even if they cover
similar ground.
By the way, Bjorn is one of those here on boost-dev whose feedback on
a HTTP library I would weigh very heavily in any peer review. Bjorn
knows his stuff here, and as he mentioned, he would be highly likely
to write a very high quality HTTP library on his first attempt had he
the time to do so.
Regarding persuading Vinicius to use Beast in Http, it's not like
anyone can wave a magic wand here. Plenty Boost library authors
refuse to use other Boost libraries and instead reinvent the wheel.
The deliberate and intentional refusal to reuse existing library code
is taken into consideration during any peer review, and I have noted
it has generally not been weighed heavily when deciding to accept.
Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/
http://ie.linkedin.com/in/nialldouglas/
David Sankel
Sep 28, 2016, 8:40:41 PM9/28/16
to bo...@lists.boost.org
Hi Vinnie!
On Fri, Sep 23, 2016 at 10:58 AM, Vinnie Falco <vinnie...@gmail.com>
wrote:
> Beast provides low level HTTP and WebSocket interfaces built on
> Boost.Asio, the project is here:
> https://github.com/vinniefalco/Beast
> The plan is to solicit feedback from the Boost community before
> submitting a request for a formal review.
> provide every possible feature or handle every use case. It just
> needed to handle the common use-cases.
>
> For the record, I am against this higher level functionality for a
> number of reasons.
> Despite my objection I have started work on these features anyway.
> Here are the beginnings of a HTTP client:
> https://github.com/vinniefalco/Beast/blob/88b485df7e6216282842f40cf99cc7
> 5dee1b82d4/test/http/get.cpp#L123
>
> Here is some work on a generic server:
> https://github.com/vinniefalco/Beast/blob/server/extras/beast/http/async_
> server.hpp#L84
Your investigation into creating a higher level interface, despite your own
reluctance to do so, is very much appreciated. I have a few ideas wrt. what
a good higher level interface would look like building off your work.
First, I think that the io_service object should not be hidden with a
higher-level interface. Asynchronous applications, once the Networking TS
becomes available, are generally going to start with the creation of an
io_service (executor), followed by the creation of some objects (or calling
of functions) which use it, and then complete with a 'ioService.run()'
call. Folks who want to do something more sophisticated with a thread pool
will have facilities to do so, but this will be the "Hello World" for
asynchronous applications.
Related to this, I don't think a synchronous interface is necessary. The
'get' function should take in a URL and a callback which has 'error_code'
and 'response' parameters. The callback would have the ability to query the
response's headers and body (as a string). Streaming the result of a get, I
would say, is out-of-scope for the high level interface.
A slightly lower-level interface (which may be good enough to replace a
special 'get' function) would take in a URL and a request<std::string>. We
could include some helpers, like 'buildSimpleGetRequest', that make this
easy to use for common scenarios where users don't want to mess around with
setting 'User-Agent' headers and such.
I'd be happy to work with you on creating a proposed higher level interface
that I think would stand a chance at WG21. It may improve consensus for the
Boost review, but if it has the opposite effect we could just rip it out.
What do you think?
-- David Sankel
On Fri, Sep 23, 2016 at 10:58 AM, Vinnie Falco <vinnie...@gmail.com>
wrote:
> Beast provides low level HTTP and WebSocket interfaces built on
> Boost.Asio, the project is here:
> https://github.com/vinniefalco/Beast
> The plan is to solicit feedback from the Boost community before
> submitting a request for a formal review.
>
> One point brought up during the conference is that Beast does not
> offer sufficient higher level HTTP functionality. This omission is
> intentional.
> The requestor informed me that this functionality did not need to
> One point brought up during the conference is that Beast does not
> offer sufficient higher level HTTP functionality. This omission is
> intentional.
> provide every possible feature or handle every use case. It just
> needed to handle the common use-cases.
>
> For the record, I am against this higher level functionality for a
> number of reasons.
> Despite my objection I have started work on these features anyway.
> Here are the beginnings of a HTTP client:
> https://github.com/vinniefalco/Beast/blob/88b485df7e6216282842f40cf99cc7
> 5dee1b82d4/test/http/get.cpp#L123
>
> Here is some work on a generic server:
> https://github.com/vinniefalco/Beast/blob/server/extras/beast/http/async_
> server.hpp#L84
Your investigation into creating a higher level interface, despite your own
reluctance to do so, is very much appreciated. I have a few ideas wrt. what
a good higher level interface would look like building off your work.
First, I think that the io_service object should not be hidden with a
higher-level interface. Asynchronous applications, once the Networking TS
becomes available, are generally going to start with the creation of an
io_service (executor), followed by the creation of some objects (or calling
of functions) which use it, and then complete with a 'ioService.run()'
call. Folks who want to do something more sophisticated with a thread pool
will have facilities to do so, but this will be the "Hello World" for
asynchronous applications.
Related to this, I don't think a synchronous interface is necessary. The
'get' function should take in a URL and a callback which has 'error_code'
and 'response' parameters. The callback would have the ability to query the
response's headers and body (as a string). Streaming the result of a get, I
would say, is out-of-scope for the high level interface.
A slightly lower-level interface (which may be good enough to replace a
special 'get' function) would take in a URL and a request<std::string>. We
could include some helpers, like 'buildSimpleGetRequest', that make this
easy to use for common scenarios where users don't want to mess around with
setting 'User-Agent' headers and such.
I'd be happy to work with you on creating a proposed higher level interface
that I think would stand a chance at WG21. It may improve consensus for the
Boost review, but if it has the opposite effect we could just rip it out.
What do you think?
-- David Sankel
Vinnie Falco
Sep 28, 2016, 9:06:29 PM9/28/16
to bo...@lists.boost.org
On Wed, Sep 28, 2016 at 8:39 PM, David Sankel <cam...@gmail.com> wrote:
> Your investigation into creating a higher level interface, despite your own
> reluctance to do so, is very much appreciated.
Thanks!
> Your investigation into creating a higher level interface, despite your own
> reluctance to do so, is very much appreciated.
> I think that the io_service object should not be hidden with a
> higher-level interface.
a server, it should be possible for:
1. The caller to provide an io_service with its own threads if they want
2. A choice of using an implicit strand: just one thread in the
io_service pool), or an explicit strand (e.g. io_service::strand
wrapper equivalent).
3.Some struct tag or construction option that provides an io_service
with some reasonable default threading policy, so the caller doesn't
have to bother with it at all.
> I don't think a synchronous interface is necessary.
> I'd be happy to work with you on creating a proposed higher level interface
> that I think would stand a chance at WG21. It may improve consensus for the
> Boost review, but if it has the opposite effect we could just rip it out.
> What do you think?
However, I'm not sure that I want to broaden the scope of Beast's
current interfaces. Other individuals have tried to create these high
level interfaces, and they have all failed review. There was a recent
claim in the mailing list that some individual could have written a
high quality HTTP client if they wanted to - then why didn't they?
We've certainly gone long enough without basic HTTP functionality
available with a sensible interface. I think Beast has a better chance
if it keeps its scope limited to just the low level protocol. Despite
this limited scope, it still provides a useful building block - and
one that we need.
With respect to a high level client interface, I have to be honest,
reading the Python Requests documentation really spoiled me. In a way,
its a good thing because I now realize that my own individual effort
and resources at producing a client would fall far short of Python
Requests - so I'm not wasting any of my energy on that anymore. When I
imagine what the most awesome C++ HTTP client library could look like,
it looks like Python Requests.
I don't think we should reinvent the wheel here, when some folks have
already done most of the heavy lifting for us. A C++, full featured
HTTP client implementation should be closely modeled after Python
Requests. The cool thing about this, is that someone has already
created such an interface. It is Huu Nguyen, the author of "C++
Requests: Curl for People". And the repository is here:
https://github.com/whoshuu/cpr
I reached out to the author and he related to me that his ambition is
to iterate on this library to produce a version that does not have
curl as a dependency. I think the most productive use of development
energies with respect to building a high level HTTP client would go
towards supporting the refinement of this library, or if the author
does not have the resources then to examine his interface and use it
as a model for creating such a library (using Beast for the protocol
level support).
On the subject of a turn-key generic HTTP server I think there are
more possibilities that I could put something together. We have a use
case for it at Ripple, and the amount of work to write a server is far
less than writing a full featured client. One caveat, when I talk
about writing a server I am talking about a low level server. I don't
have the knowledge or resources to include features like server side
multi-part encoding, cookie jars, redirection, caching of local
content, authentication, etc... I am talking about a server class
template which lets you write something like this:
struct handler
{
template<class Body, class Headers>
response_v1<string_body>
operator()(request_v1<Body, Headers> const& req)
{
// your code here
}
};
int main()
{
using beast::http;
beast::http::async_server<handler> s{"0.0.0.0", 80};
// Block until SIGINT or SIGTERM is received.
sig_wait();
}
Like the current Beast interfaces, this would be a low level server.
it manages the connections, correctly handles exiting the app cleanly,
reads the HTTP requests and buffers the responses. But that's it, the
rest is up to the caller. I imagine we would have the same debate
about low level versus high level server interface, and my answer
would be that someone can use this generic server as a building block
to add components to it and make it more full featured. For example by
writing a cgi module, or adding a file serving handler, etc..
I've already started some of this work but there are some design
issues that I haven't solved yet (I'm open to suggestions):
https://github.com/vinniefalco/Beast/blob/6f794e80fd4fad3a4777e68f7fc732ecbbdb94e7/extras/beast/http/async_server.hpp#L84.
I think conversations about these subjects are a good idea, thanks for
bringing it to the list.
Regards
Christof Donat
Sep 29, 2016, 5:24:01 AM9/29/16
to bo...@lists.boost.org
Hi,
Am 29.09.2016 03:06, schrieb Vinnie Falco:
>> I don't think a synchronous interface is necessary.
>
> Probably true. A synchronous interface would require a redundant code
> path.
I second that.
>> I'd be happy to work with you on creating a proposed higher level
>> interface
>> that I think would stand a chance at WG21. It may improve consensus
>> for the
>> Boost review, but if it has the opposite effect we could just rip it
>> out.
>> What do you think?
>
> There's a lot of merit to a team effort for building a good design.
> However, I'm not sure that I want to broaden the scope of Beast's
> current interfaces.
I agree, that there is much value in having one library for the low
level stuff, which beast is very well suited. I also agree, that higher
level APIs should be built on top of that instead of putting everything
in the same project.
> With respect to a high level client interface, [...]
> On the subject of a turn-key generic HTTP server I think there are
> more possibilities that I could put something together. [...]
I think frameworks for HTTP Servers should be separated in various
parts:
1. Low Level HTTP stuff - that is what beast is perfect for
2. A Request Router
3. A Database Access Layer - most Web applications are good off to use a
database
4. A way to create the HTML output without mixing it with the code
For 3 and 4 I'd propose Roland Bocks sqlpp11
(https://github.com/rbock/sqlpp11) and kiss-templates
(https://github.com/rbock/kiss-templates)
So some weeks ago I started a request router as hobby project based on
beast v1. It does not even remotely work yet and of course the interface
is not at all fixed now. As usuall hobby projects might get a lot of
love, but never enough time, so the advance is very slow at the moment.
The routing looks like this:
auto appRoute = route_url(route("^/foo/([0-9]+)"s,
app::foo{url_match_cast<int>{"$1"s}}),
route("^/bar/([a-fA-F0-9]*)"s,
app::bar{url_match_cast<std::string>{"$1"s}}),
route("^/baz/([0-9]*([.][0-9]+)?)",
app::baz{url_match_cast<double>{"$1"}}));
auto r = route_url(route("^/oldAppBase(.*)$"s,
redirect("/newAppBase$1"s)),
route("^/newAppBase(.*)$"s, appRoute("$1"s)),
route("^/webSockets(.*)$"s, sockHandler{"$1"s}),
route("^/otherWebSockets(.*)$"s,
otherSockHandler{"$1"s}),
route("^/static(.*)$"s,
static_file_handler{boost::filesystem::current_path(), "$1"s}));
The idea here is, that you can create independent components, like
"appRoute", and mount them together to build a complete application. The
Components themself can be build with multiple handlers, that can be
components themselves as well. This is inspired by Django.
The handlers might look like this:
namespace app {
class foo: public base_http_handler<foo> {
using base_http_handler<foo>::operator();
url_match_cast<int> get_foo_;
public:
foo(const url_match_cast<int>& get_foo):
get_foo_{get_foo} {};
template<typename Request>
beast::http::response_v1<beast::http::string_body> operator
() (const std::smatch& match, const Request& req) {
auto resp =
beast::http::response_v1<beast::http::string_body>{};
resp.status = 200;
resp.reason = beast::http::reason_string(200);
resp.headers.replace("Content-Type", "text/html");
auto foo_text =
boost::lexical_cast<std::string>(get_foo_(match));
resp.body = "<html><head><title>foo: "s + foo_text +
"</title></head><body>foo: "s + foo_text + "</body></html>"s;
resp.headers.replace("Content-Length",
resp.body.size());
return resp;
};
};
}
But they can also just be lambdas with the signature like this call
operator. With generalized captures, this could also be writen as:
route("^/foo/([0-9]+)"s,
[get_foo_=url_match_cast<int>{"$1"s}] (const std::smatch& match,
const Request& req)
-> beast::http::response_v1<beast::http::string_body> operator
() {
// ...
auto foo = get_foo_(match);
// ...
})
The parameter match is the match of the regular expression on the path.
With get_foo_(match) we get the first parameter ("$1") from that match
as an integer (url_match_cast<int>).
I think, the main loop should not be part of the framework. My current
idea would look like to use std::future<boost::system::error_code> as
return type for the call operator of a rout_url:
std::future<boost::system::error_code> success_async = r(url, req,
my_socket);
What do you think about such an interface?
Christof
Am 29.09.2016 03:06, schrieb Vinnie Falco:
>> I don't think a synchronous interface is necessary.
>
> Probably true. A synchronous interface would require a redundant code
> path.
>> I'd be happy to work with you on creating a proposed higher level
>> interface
>> that I think would stand a chance at WG21. It may improve consensus
>> for the
>> Boost review, but if it has the opposite effect we could just rip it
>> out.
>> What do you think?
>
> There's a lot of merit to a team effort for building a good design.
> However, I'm not sure that I want to broaden the scope of Beast's
> current interfaces.
level stuff, which beast is very well suited. I also agree, that higher
level APIs should be built on top of that instead of putting everything
in the same project.
> With respect to a high level client interface, [...]
> The cool thing about this, is that someone has already
> created such an interface. It is Huu Nguyen, the author of "C++
> Requests: Curl for People". And the repository is here:
> https://github.com/whoshuu/cpr
> [...]
> created such an interface. It is Huu Nguyen, the author of "C++
> Requests: Curl for People". And the repository is here:
> https://github.com/whoshuu/cpr
> I reached out to the author and he related to me that his ambition is
> to iterate on this library to produce a version that does not have
> curl as a dependency. I think the most productive use of development
> energies with respect to building a high level HTTP client would go
> towards supporting the refinement of this library, or if the author
> does not have the resources then to examine his interface and use it
> as a model for creating such a library (using Beast for the protocol
> level support).
Really great.
> to iterate on this library to produce a version that does not have
> curl as a dependency. I think the most productive use of development
> energies with respect to building a high level HTTP client would go
> towards supporting the refinement of this library, or if the author
> does not have the resources then to examine his interface and use it
> as a model for creating such a library (using Beast for the protocol
> level support).
> On the subject of a turn-key generic HTTP server I think there are
I think frameworks for HTTP Servers should be separated in various
parts:
1. Low Level HTTP stuff - that is what beast is perfect for
2. A Request Router
3. A Database Access Layer - most Web applications are good off to use a
database
4. A way to create the HTML output without mixing it with the code
For 3 and 4 I'd propose Roland Bocks sqlpp11
(https://github.com/rbock/sqlpp11) and kiss-templates
(https://github.com/rbock/kiss-templates)
So some weeks ago I started a request router as hobby project based on
beast v1. It does not even remotely work yet and of course the interface
is not at all fixed now. As usuall hobby projects might get a lot of
love, but never enough time, so the advance is very slow at the moment.
The routing looks like this:
auto appRoute = route_url(route("^/foo/([0-9]+)"s,
app::foo{url_match_cast<int>{"$1"s}}),
route("^/bar/([a-fA-F0-9]*)"s,
app::bar{url_match_cast<std::string>{"$1"s}}),
route("^/baz/([0-9]*([.][0-9]+)?)",
app::baz{url_match_cast<double>{"$1"}}));
auto r = route_url(route("^/oldAppBase(.*)$"s,
redirect("/newAppBase$1"s)),
route("^/newAppBase(.*)$"s, appRoute("$1"s)),
route("^/webSockets(.*)$"s, sockHandler{"$1"s}),
route("^/otherWebSockets(.*)$"s,
otherSockHandler{"$1"s}),
route("^/static(.*)$"s,
static_file_handler{boost::filesystem::current_path(), "$1"s}));
The idea here is, that you can create independent components, like
"appRoute", and mount them together to build a complete application. The
Components themself can be build with multiple handlers, that can be
components themselves as well. This is inspired by Django.
The handlers might look like this:
namespace app {
class foo: public base_http_handler<foo> {
using base_http_handler<foo>::operator();
url_match_cast<int> get_foo_;
public:
foo(const url_match_cast<int>& get_foo):
get_foo_{get_foo} {};
template<typename Request>
beast::http::response_v1<beast::http::string_body> operator
() (const std::smatch& match, const Request& req) {
auto resp =
beast::http::response_v1<beast::http::string_body>{};
resp.status = 200;
resp.reason = beast::http::reason_string(200);
resp.headers.replace("Content-Type", "text/html");
auto foo_text =
boost::lexical_cast<std::string>(get_foo_(match));
resp.body = "<html><head><title>foo: "s + foo_text +
"</title></head><body>foo: "s + foo_text + "</body></html>"s;
resp.headers.replace("Content-Length",
resp.body.size());
return resp;
};
};
}
But they can also just be lambdas with the signature like this call
operator. With generalized captures, this could also be writen as:
route("^/foo/([0-9]+)"s,
[get_foo_=url_match_cast<int>{"$1"s}] (const std::smatch& match,
const Request& req)
-> beast::http::response_v1<beast::http::string_body> operator
() {
// ...
auto foo = get_foo_(match);
// ...
})
The parameter match is the match of the regular expression on the path.
With get_foo_(match) we get the first parameter ("$1") from that match
as an integer (url_match_cast<int>).
I think, the main loop should not be part of the framework. My current
idea would look like to use std::future<boost::system::error_code> as
return type for the call operator of a rout_url:
std::future<boost::system::error_code> success_async = r(url, req,
my_socket);
What do you think about such an interface?
Christof
Vinnie Falco
Sep 29, 2016, 9:36:02 AM9/29/16
to bo...@lists.boost.org
On Thu, Sep 29, 2016 at 5:23 AM, Christof Donat <c...@okunah.de> wrote:
> I think frameworks for HTTP Servers should be separated in various parts:
>
> 1. Low Level HTTP stuff - that is what beast is perfect for
> 2. A Request Router
> 3. A Database Access Layer - most Web applications are good off to use a
> database
> 4. A way to create the HTML output without mixing it with the code
I'm proposing that there is a server layer that is even lower than the
> I think frameworks for HTTP Servers should be separated in various parts:
>
> 1. Low Level HTTP stuff - that is what beast is perfect for
> 2. A Request Router
> 3. A Database Access Layer - most Web applications are good off to use a
> database
> 4. A way to create the HTML output without mixing it with the code
items listed above. Specifically a generic server with these features:
1. An object modeling the entire server
2. An object to model an acceptor (listening port)
3. Server methods to add and remove acceptors dynamically
4. For acceptors supporting TLS, control over certificates
5. Acceptor supports plain, TLS, or auto-detected plain/TLS
6. An object to model a session (active connection)
7. Customization of the server with caller-provided behavior and state
8. Customization of the acceptor with caller-provided behavior and state
9. Customization of the session with caller-provided behavior and state
10. Control over the associated io_service and its threads
Explanation of each of these items:
1. A type that represents the server, its acceptors, its active
sessions, and caller provided state. This allows the caller to declare
one or more variables of server type.
2. Each listening port is represented by an object with its own
settings such as the address to bind to, port number, and whether it
supports plain, TLS, or auto-detect. By auto-detect, I mean that the
acceptor can detect whether the remote is attempting to perform a TLS
handshake or not, and allow either type of connection on the active
socket.
3. Allow callers to modify the listening ports on the fly. For
example, to stop allowing new connections, remove the port, or add a
new port.
4. For a port that supports TLS, an interface giving control of
certificates. It should also be possible to require client
certificates thus permitting development of a middleware server.
6. The active connection is represented by an object in memory with
suitable member functions.
7. The caller can associate the server with state. One example, is the
root directory from which to serve files. This could be done using
CRTP, or a caller supplied template argument. Acceptor and session
objects need access to this state so they can make decisions. For
example, the server could have an IP whitelist that each acceptor can
look at.
8. The caller can associate the acceptor with state. For example, the
domain specific permissions to give sessions associated with that
acceptor.
9. The caller can associate the session with state, access the state
of the associated acceptor from which the connection was established,
and access the state of the owning server.
10. Handling of the io_service should not constrain users' choice of
design patterns.
> For 3 and 4 I'd propose Roland Bocks sqlpp11
> (https://github.com/rbock/sqlpp11) and kiss-templates
> (https://github.com/rbock/kiss-templates)
>
> So some weeks ago I started a request router as hobby project based on beast
> v1. It does not even remotely work yet and of course the interface is not at
> all fixed now. As usuall hobby projects might get a lot of love, but never
> enough time, so the advance is very slow at the moment.
> The handlers might look like this:
>
> namespace app {
> class foo: public base_http_handler<foo> {
What you have done here, and what you should be congratulated for, is
to become the first stakeholder to provide an example of code (I
think). Code samples are a universally understood way to get your
point across regarding design decisions.
The interface to the router is a bit high level for me. I would be
interested to see your ideas about how your class base_http_handler is
instantiated. Is it created when the socket is created? How does the
handler gain access to session specific information, such as the
remote IP address? Or caller provided state such as the filesystem
root or permissions? Did you start with Beast's http_async_server and
modify it? Or do you have your own implementation that manages
connections?
> template<typename Request>
> beast::http::response_v1<beast::http::string_body>
> ...
> What do you think about such an interface?
I love that you have provided a function signature for everyone to
consider, I wish more people did this.
With respect to your signature, I think there a problem with using the
return type in that fashion. What if different routers want to use
different types for the HTTP body? A key feature of Beast is that
callers can get control over the algorithms used to parse and
serialize the body. For a server, this ability is critical - its the
intention that streaming from a coroutine, progressive
download/upload, handling of the Content-Encoding, are all done using
user-defined body customizations. C++ only allows for one return type
so this would complicate things.
A robust server will almost never use beast::http::string_body or
beast::http::streambuf_body except for error messages like 404 Not
Found. More likely, someone will develop a rich Body type that
supports a wide variety of features such as the encoding as I
mentioned above, multi-part/mime extensions, and the like.
Another approach, instead of using the return type, is to give the
handler an object which when invoked with a message, queues or sends
the message. This allows the caller to use any choices of body type
(or headers type) depending on what the router selects. A modified
version of your handler might look like this:
struct http_handler
{
/** Provides the HTTP response given a request.
SendFunction will have this equivalent signature:
@code
template<class Body, Class Headers>
f(response_v1<Body, Headers>&& res)
@endcode
If res is movable, the send function will acquire ownership. Otherwise, if
res is copyable then a copy will be made.
SendFunction Requirements:
res must be MoveConstructible or CopyConstructible.
*/
template<class SendFunction, class Body, class Headers>
void
operator()(request_v1<Body, Headers> const& req, SendFunction&& f);
};
Thank you for sharing your ideas, I think this is going to really help
the community make forward progress on HTTP.
Regards
Christof Donat
Sep 30, 2016, 10:35:15 AM9/30/16
to bo...@lists.boost.org
Hi,
Am 29.09.2016 15:34, schrieb Vinnie Falco:
> On Thu, Sep 29, 2016 at 5:23 AM, Christof Donat <c...@okunah.de> wrote:
>> I think frameworks for HTTP Servers should be separated in various
>> parts:
>>
>> 1. Low Level HTTP stuff - that is what beast is perfect for
>> 2. A Request Router
>> 3. A Database Access Layer - most Web applications are good off to use
>> a
>> database
>> 4. A way to create the HTML output without mixing it with the code
>
> I'm proposing that there is a server layer that is even lower than the
> items listed above. Specifically a generic server with these features:
> [...]
So that would be between the low level HTTP stuff and the request
router. Looks like a good idea and probably will make the request router
simpler.
> 6. The active connection is represented by an object in memory with
> suitable member functions.
We could try define a kind of concept for the session store and the
session objects, so that the session could also be persistent on disk,
or in e.g. memcached. The session handling code then should be a
template that is instantiated with these types. You'll probably like to
be able to define the session objects class anyway.
One important thing, I think is, that we should also be able to have no
session tracking at all without having to pay for that, e.g. we don't
want a session cookie then. So maybe we define a session object class,
that triggers the corresponding meta program to cut out all the session
management code all together.
> 7. The caller can associate the server with state. One example, is the
> root directory from which to serve files.
When you look back at my examples, my approach to serve static files was
to use a special handler:
static_file_handler{boost::filesystem::current_path(), "$1"s} That will
instantiate a handler that works on the directory, the application has
been started in (why-ever you might want that). It will take the first
sub-expression of the match as path to find the file to serve in that
directory.
I don't think, that serving static files should be implemented on a
lower level. You'd have to twiddle around with the routing system on a
lower level then in order to know, which URLs are served by the
application and which should be served by files.
>> The handlers might look like this:
>>
>> namespace app {
>> class foo: public base_http_handler<foo> {
>> ...
>
> What you have done here, and what you should be congratulated for, is
> to become the first stakeholder to provide an example of code (I
> think). Code samples are a universally understood way to get your
> point across regarding design decisions.
:-) that was the idea.
> The interface to the router is a bit high level for me. I would be
> interested to see your ideas about how your class base_http_handler is
> instantiated.
Base_http_handler is a service class for other http handlers. Handlers
don't have to be derived from base_http_handler, it just provides some
stuff, that many handlers will probably need.
> Is it created when the socket is created?
The handlers are only instantiated, when the route is defined. So in a
typical application, there will be only one instance of each handler.
The constructor parameters are global to all requests to that handler.
E.g. url_match_cast<int>{"$1"s} will simply provide a type, that can be
used to extract the first parameter of any regular expression match as
int. The default implementation uses boost::lexical_cast<>, but of
course you can create specializations for your own types.
When the request is processed, I only call the call operator of that
handler. That is, why it can also be a lambda.
> How does the
> handler gain access to session specific information, such as the
> remote IP address?
The current idea is, to get that from the request object in future.
There might be better options, of course.
> Or caller provided state such as the filesystem
> root or permissions?
Those should be passed in as constructor parameters, like I did with the
base path for the static_file_handler.
> Did you start with Beast's http_async_server and
> modify it? Or do you have your own implementation that manages
> connections?
I tried using the code in http_async_server, but at the moment I'd be
happy to get the unit tests running, before I even hope to soon have a
working example server.
> With respect to your signature, I think there a problem with using the
> return type in that fashion. What if different routers want to use
> different types for the HTTP body? A key feature of Beast is that
> callers can get control over the algorithms used to parse and
> serialize the body. For a server, this ability is critical - its the
> intention that streaming from a coroutine, progressive
> download/upload, handling of the Content-Encoding, are all done using
> user-defined body customizations. C++ only allows for one return type
> so this would complicate things.
>
> A robust server will almost never use beast::http::string_body or
> beast::http::streambuf_body except for error messages like 404 Not
> Found. More likely, someone will develop a rich Body type that
> supports a wide variety of features such as the encoding as I
> mentioned above, multi-part/mime extensions, and the like.
Actually I obviously misunderstood that part of beast then. I guess,
that is the reason, why I have searched for a reasonable body object and
then resorted to beast::http::string_body. I thought, having different
body types would be for some rare cornercases, that we usually will not
have in a high level framework.
So am I correct to assume, that the idea is to have a more or less
arbitrary body object. In the end that will have to serialized to a
string of bytes. Did you intend to have that implemented in the body
object? If so, that would be a great place to plug in e.g. kiss
templates. My current idea was to call the template inside the handler
and write the result to the body.
I think, the router can be made to handle different return types, as
long as they implement a yet to define concept. I also have some
thoughts to make it more flexible on the parameters. For some handlers
you'll be happy to just get the URL match, others will maybe need a
stream for websocket interaction. I think, that can be achieved using
boost call traits, std::result_of, std::is_callable, or similar things,
and a little bit of not too difficult meta programming magic. I hope to
get some time this weekend to look into that.
> Another approach, instead of using the return type, is to give the
> handler an object which when invoked with a message, queues or sends
> the message.
I also thought about that. For the time being, I chose to return the
response object for semantic reasons. For me that is the result of the
operation. With the approach of passing in a kind of response object,
the operation would, as a side effect, respond to the request, and
return something else, like an error code, or even void.
> Thank you for sharing your ideas, I think this is going to really help
> the community make forward progress on HTTP.
I hope so :-)
Christof
Am 29.09.2016 15:34, schrieb Vinnie Falco:
> On Thu, Sep 29, 2016 at 5:23 AM, Christof Donat <c...@okunah.de> wrote:
>> I think frameworks for HTTP Servers should be separated in various
>> parts:
>>
>> 1. Low Level HTTP stuff - that is what beast is perfect for
>> 2. A Request Router
>> 3. A Database Access Layer - most Web applications are good off to use
>> a
>> database
>> 4. A way to create the HTML output without mixing it with the code
>
> I'm proposing that there is a server layer that is even lower than the
> items listed above. Specifically a generic server with these features:
So that would be between the low level HTTP stuff and the request
router. Looks like a good idea and probably will make the request router
simpler.
> 6. The active connection is represented by an object in memory with
> suitable member functions.
session objects, so that the session could also be persistent on disk,
or in e.g. memcached. The session handling code then should be a
template that is instantiated with these types. You'll probably like to
be able to define the session objects class anyway.
One important thing, I think is, that we should also be able to have no
session tracking at all without having to pay for that, e.g. we don't
want a session cookie then. So maybe we define a session object class,
that triggers the corresponding meta program to cut out all the session
management code all together.
> 7. The caller can associate the server with state. One example, is the
> root directory from which to serve files.
to use a special handler:
static_file_handler{boost::filesystem::current_path(), "$1"s} That will
instantiate a handler that works on the directory, the application has
been started in (why-ever you might want that). It will take the first
sub-expression of the match as path to find the file to serve in that
directory.
I don't think, that serving static files should be implemented on a
lower level. You'd have to twiddle around with the routing system on a
lower level then in order to know, which URLs are served by the
application and which should be served by files.
>> The handlers might look like this:
>>
>> namespace app {
>> class foo: public base_http_handler<foo> {
>> ...
>
> What you have done here, and what you should be congratulated for, is
> to become the first stakeholder to provide an example of code (I
> think). Code samples are a universally understood way to get your
> point across regarding design decisions.
> The interface to the router is a bit high level for me. I would be
> interested to see your ideas about how your class base_http_handler is
> instantiated.
don't have to be derived from base_http_handler, it just provides some
stuff, that many handlers will probably need.
> Is it created when the socket is created?
typical application, there will be only one instance of each handler.
The constructor parameters are global to all requests to that handler.
E.g. url_match_cast<int>{"$1"s} will simply provide a type, that can be
used to extract the first parameter of any regular expression match as
int. The default implementation uses boost::lexical_cast<>, but of
course you can create specializations for your own types.
When the request is processed, I only call the call operator of that
handler. That is, why it can also be a lambda.
> How does the
> handler gain access to session specific information, such as the
> remote IP address?
There might be better options, of course.
> Or caller provided state such as the filesystem
> root or permissions?
base path for the static_file_handler.
> Did you start with Beast's http_async_server and
> modify it? Or do you have your own implementation that manages
> connections?
happy to get the unit tests running, before I even hope to soon have a
working example server.
> With respect to your signature, I think there a problem with using the
> return type in that fashion. What if different routers want to use
> different types for the HTTP body? A key feature of Beast is that
> callers can get control over the algorithms used to parse and
> serialize the body. For a server, this ability is critical - its the
> intention that streaming from a coroutine, progressive
> download/upload, handling of the Content-Encoding, are all done using
> user-defined body customizations. C++ only allows for one return type
> so this would complicate things.
>
> A robust server will almost never use beast::http::string_body or
> beast::http::streambuf_body except for error messages like 404 Not
> Found. More likely, someone will develop a rich Body type that
> supports a wide variety of features such as the encoding as I
> mentioned above, multi-part/mime extensions, and the like.
that is the reason, why I have searched for a reasonable body object and
then resorted to beast::http::string_body. I thought, having different
body types would be for some rare cornercases, that we usually will not
have in a high level framework.
So am I correct to assume, that the idea is to have a more or less
arbitrary body object. In the end that will have to serialized to a
string of bytes. Did you intend to have that implemented in the body
object? If so, that would be a great place to plug in e.g. kiss
templates. My current idea was to call the template inside the handler
and write the result to the body.
I think, the router can be made to handle different return types, as
long as they implement a yet to define concept. I also have some
thoughts to make it more flexible on the parameters. For some handlers
you'll be happy to just get the URL match, others will maybe need a
stream for websocket interaction. I think, that can be achieved using
boost call traits, std::result_of, std::is_callable, or similar things,
and a little bit of not too difficult meta programming magic. I hope to
get some time this weekend to look into that.
> Another approach, instead of using the return type, is to give the
> handler an object which when invoked with a message, queues or sends
> the message.
response object for semantic reasons. For me that is the result of the
operation. With the approach of passing in a kind of response object,
the operation would, as a side effect, respond to the request, and
return something else, like an error code, or even void.
> Thank you for sharing your ideas, I think this is going to really help
> the community make forward progress on HTTP.
Christof
Vinnie Falco
Sep 30, 2016, 11:23:49 AM9/30/16
to bo...@lists.boost.org
On Fri, Sep 30, 2016 at 10:34 AM, Christof Donat <c...@okunah.de> wrote:
>> I'm proposing that there is a server layer that is even lower than the
>> items listed above.
>
>> I'm proposing that there is a server layer that is even lower than the
>> items listed above.
>
> So that would be between the low level HTTP stuff and the request router.
Right. My approach is to work from the bottom up, since the number of
degrees of freedom with respect to designing an interface increases as
you go to higher levels (some would disagree with that statement). No
one can dispute that an HTTP application needs to send and receive
HTTP messages (the lowest level). But there would be debate on how to
handle cookies or how they should be persisted (high level functions).
Once you have read/write HTTP message primitives available, then on
the server side the next logical step up is to have a low-level server
framework.
> We could try define a kind of concept for the session store and the session
> objects, so that the session could also be persistent on disk
in the lowest level sense. Pretty much the
boost::asio::ip::tcp::socket and a few state variables (such as a
reference to the server, and a reference to the acceptor). I wasn't
talking about a high level HTTP session such as persistent data that
survives an individual TCP/IP connection.
If it seems that I'm trying to steer the discussion towards more
low-level concepts, well... I am :) For me to get from point A
(Beast's interfaces) to point B (what real users want, such as a
robust client or server template) I think of small steps that build on
what's already there.
> One important thing, I think is, that we should also be able to have no
> session tracking at all without having to pay for that
> So am I correct to assume, that the idea is to have a more or less arbitrary
> body object.
> In the end that will have to serialized to a string of bytes.
type to be Serializable, the Body must provide the nested type which
meets the requirements of Writer:
http://vinniefalco.github.io/beast/beast/ref/Writer.html
> Did you intend to have that implemented in the body object?
both the container used to represent the body (this becomes a data
member in the message object) and the algorithms used to serialize and
parse the body. The choice of container is entirely up to you and
doesn't need to actually hold the data. In the Beast HTTP server
example, the file_body uses a std::string representing the path to the
file:
https://github.com/vinniefalco/Beast/blob/70b8555cdca69b9c5777db02115d30d1c1aad631/examples/file_body.hpp#L22
https://github.com/vinniefalco/Beast/blob/70b8555cdca69b9c5777db02115d30d1c1aad631/examples/file_body.hpp#L52
> My current idea was to call the template inside the handler
> and write the result to the body.
large files, I think you will get better results if you implement your
own custom Body type. Remember that the calculation of HTTP responses
happens on an io_service thread. Things work better if you block on
O(1) instead of O(n). So rather than reading in the entire file into
memory when building the response, its better to just send fixed size
pieces of the file from a custom writer. This also consumes less
memory, allowing the server to scale to more connections with the same
amount of resources.
The Beast file_body sends in chunks of 4K:
https://github.com/vinniefalco/Beast/blob/70b8555cdca69b9c5777db02115d30d1c1aad631/examples/file_body.hpp#L30
> I think, the router can be made to handle different return types
> I also have some thoughts to make it
> more flexible on the parameters. For some handlers you'll be happy to just
> get the URL match, others will maybe need a stream for websocket
> interaction. I think, that can be achieved using boost call traits,
> std::result_of, std::is_callable, or similar things, and a little bit of not
> too difficult meta programming magic.
>> Another approach, instead of using the return type, is to give the
>> handler an object which when invoked with a message, queues or sends
>> the message.
> I also thought about that. For the time being, I chose to return the
> response object for semantic reasons. For me that is the result of the
> operation. With the approach of passing in a kind of response object, the
> operation would, as a side effect, respond to the request, and return
> something else, like an error code, or even void.
Adhering to language "purity" certainly has its merits but when
> response object for semantic reasons. For me that is the result of the
> operation. With the approach of passing in a kind of response object, the
> operation would, as a side effect, respond to the request, and return
> something else, like an error code, or even void.
building general purpose code I think we need to consider all the
possibilities.
For returning the http::response_v1, using the indirect interface I
described above will result in smaller, easier to understand code.
Creating a system where functions return response objects of different
types (varying by their Body parameter) only creates complexity for
calling code. And the only advantage is that it is emotionally
satisfying - this will be hard to justify to users of the library.
For error_code, passing it as a non-const reference is a clear winner.
If you use error_code as the return value you are paying for a default
construction in every function on the call chain.
Robert Ramey
Oct 1, 2016, 5:43:12 PM10/1/16
to bo...@lists.boost.org
I've followed a little the discussion here.
I've got your cool little brochure from CPPCon
I've browsed the documentation
I don't really know anything about networking
I like what I've seen.
I think one thing the would help me a lot would be if the documentation
included one or two simple examples. Ideally that would be something
simple from the python request library. Hopefully this would
illustrate how your library can be of help in creating functionality
we don't yet have.
Given the work you've obviously already invested, I think adding
two examples with good narrative would be well worth the effort.
Robert Ramey
I've got your cool little brochure from CPPCon
I've browsed the documentation
I don't really know anything about networking
I like what I've seen.
I think one thing the would help me a lot would be if the documentation
included one or two simple examples. Ideally that would be something
simple from the python request library. Hopefully this would
illustrate how your library can be of help in creating functionality
we don't yet have.
Given the work you've obviously already invested, I think adding
two examples with good narrative would be well worth the effort.
Robert Ramey
Paul Fultz II
Oct 1, 2016, 8:20:52 PM10/1/16
to bo...@lists.boost.org
> On Sep 23, 2016, at 9:58 AM, Vinnie Falco <vinnie...@gmail.com> wrote:
>
> Beast provides low level HTTP and WebSocket interfaces built on
> Boost.Asio, the project is here:
> https://github.com/vinniefalco/Beast
> The plan is to solicit feedback from the Boost community before
> submitting a request for a formal review.
Also, looking at the cmake, it looks like you use coroutines. I would prefer not to have an HTTP library that depends on Boost.Context due to portability issues(the author refuses to support mingw).
Thanks,
Paul
Klemens Morgenstern
Oct 1, 2016, 8:25:31 PM10/1/16
to bo...@lists.boost.org
Am 02.10.2016 um 02:20 schrieb Paul Fultz II:
>> On Sep 23, 2016, at 9:58 AM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>
>> Beast provides low level HTTP and WebSocket interfaces built on
>> Boost.Asio, the project is here:
>> https://github.com/vinniefalco/Beast
>> The plan is to solicit feedback from the Boost community before
>> submitting a request for a formal review.
> The library looks nice, I tried to run the tests and install it using cmake and it both failed.
>
> Also, looking at the cmake, it looks like you use coroutines. I would prefer not to have an HTTP library that depends on Boost.Context due to portability issues(the author refuses to support mingw).
>
> Thanks,
> Paul
I'm using boost.coroutine through boost.asio in a mingw project, which
>> On Sep 23, 2016, at 9:58 AM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>
>> Beast provides low level HTTP and WebSocket interfaces built on
>> Boost.Asio, the project is here:
>> https://github.com/vinniefalco/Beast
>> The plan is to solicit feedback from the Boost community before
>> submitting a request for a formal review.
> The library looks nice, I tried to run the tests and install it using cmake and it both failed.
>
> Also, looking at the cmake, it looks like you use coroutines. I would prefer not to have an HTTP library that depends on Boost.Context due to portability issues(the author refuses to support mingw).
>
> Thanks,
> Paul
would then be dependent on boost.context. So either I'm doing something
unsupported or you're wrong here. Works fine anyway.
Vinnie Falco
Oct 1, 2016, 8:39:40 PM10/1/16
to bo...@lists.boost.org
On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
> The library looks nice
Thanks!
> I tried to run the tests and install it using cmake and it both failed.
If you could please be so kind as to create an issue on the Beast
GitHub repository page with the details, I will try to address it
right away:
https://github.com/vinniefalco/Beast/issues
> Also, looking at the cmake, it looks like you use coroutines.
Beast is header-only, the Jamfile and CMakeLists.txt are only for
building the unit tests and the examples. As far as linking goes, you
should only need Boost.System (for error_code). Some of the unit tests
use coroutines, to make sure that they work and also as a tool to
assist reaching a high level of code coverage.
You shouldn't need either the Jamfile or the CMakeLists.txt found in
the repository in order to use Beast.
> I would prefer not to have an HTTP library that depends on Boost.Context
> due to portability issues(the author refuses to support mingw).
I have no plans to add Boost.Context as a dependency, Beast uses the
Universal Asynchronous Model which abstracts the completion token (see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3747.pdf).
There is at least one mingw Beast user, they kindly provided minor
fixes to make Beast work there:
https://github.com/vinniefalco/Beast/commit/38f0d950b06059f5c671b3e381344fc782fac499
If you have any other problems or questions please don't hesitate to
open a GitHub issue, that's the best way to get things fixed or
answered.
Hope you enjoy Beast!
> The library looks nice
Thanks!
> I tried to run the tests and install it using cmake and it both failed.
GitHub repository page with the details, I will try to address it
right away:
https://github.com/vinniefalco/Beast/issues
> Also, looking at the cmake, it looks like you use coroutines.
building the unit tests and the examples. As far as linking goes, you
should only need Boost.System (for error_code). Some of the unit tests
use coroutines, to make sure that they work and also as a tool to
assist reaching a high level of code coverage.
You shouldn't need either the Jamfile or the CMakeLists.txt found in
the repository in order to use Beast.
> I would prefer not to have an HTTP library that depends on Boost.Context
> due to portability issues(the author refuses to support mingw).
Universal Asynchronous Model which abstracts the completion token (see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3747.pdf).
There is at least one mingw Beast user, they kindly provided minor
fixes to make Beast work there:
https://github.com/vinniefalco/Beast/commit/38f0d950b06059f5c671b3e381344fc782fac499
If you have any other problems or questions please don't hesitate to
open a GitHub issue, that's the best way to get things fixed or
answered.
Hope you enjoy Beast!
Paul Fultz II
Oct 1, 2016, 8:46:42 PM10/1/16
to bo...@lists.boost.org
> On Oct 1, 2016, at 7:24 PM, Klemens Morgenstern <klemens.m...@gmx.net> wrote:
>
> I'm using boost.coroutine through boost.asio in a mingw project, which would then be dependent on boost.context. So either I'm doing something unsupported or you're wrong here. Works fine anyway.
Boost.Context only builds for mingw if you have a MSVC assembler. Now, RedHat patches it, so Boost.Context can be built on RedHat(or CentOS). However, the author has refused to accept those patches unfortunately. When I build boost on ubuntu for mingw, I disable Boost.Context, which would mean disabling Boost.Beast as well.
Also, Boost.Asio works fine as it doesn’t use Boost.Coroutine nor Boost.Context. I think it provides its own stackless-based coroutines.
Paul
Klemens Morgenstern
Oct 1, 2016, 8:52:27 PM10/1/16
to bo...@lists.boost.org
Am 02.10.2016 um 02:45 schrieb Paul Fultz II:
>> On Oct 1, 2016, at 7:24 PM, Klemens Morgenstern <klemens.m...@gmx.net> wrote:
>>
>> I'm using boost.coroutine through boost.asio in a mingw project, which would then be dependent on boost.context. So either I'm doing something unsupported or you're wrong here. Works fine anyway.
> Boost.Context only builds for mingw if you have a MSVC assembler. Now, RedHat patches it, so Boost.Context can be built on RedHat(or CentOS). However, the author has refused to accept those patches unfortunately. When I build boost on ubuntu for mingw, I disable Boost.Context, which would mean disabling Boost.Beast as well.
>
> Also, Boost.Asio works fine as it doesn’t use Boost.Coroutine nor Boost.Context. I think it provides its own stackless-based coroutines.
>
> Paul
Oh, I didn't know that, thanks for the info - but since I'm not >> On Oct 1, 2016, at 7:24 PM, Klemens Morgenstern <klemens.m...@gmx.net> wrote:
>>
>> I'm using boost.coroutine through boost.asio in a mingw project, which would then be dependent on boost.context. So either I'm doing something unsupported or you're wrong here. Works fine anyway.
> Boost.Context only builds for mingw if you have a MSVC assembler. Now, RedHat patches it, so Boost.Context can be built on RedHat(or CentOS). However, the author has refused to accept those patches unfortunately. When I build boost on ubuntu for mingw, I disable Boost.Context, which would mean disabling Boost.Beast as well.
>
> Also, Boost.Asio works fine as it doesn’t use Boost.Coroutine nor Boost.Context. I think it provides its own stackless-based coroutines.
>
> Paul
cross-compiling I didn't notive that. I'm using stackful coroutines, and
those are implemented with boost.coroutine.
Paul Fultz II
Oct 1, 2016, 8:59:48 PM10/1/16
to bo...@lists.boost.org
> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>
> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>> The library looks nice
>
> Thanks!
>
>> I tried to run the tests and install it using cmake and it both failed.
>
> If you could please be so kind as to create an issue on the Beast
> GitHub repository page with the details, I will try to address it
> right away:
> https://github.com/vinniefalco/Beast/issues
Ok, will do.
>
>> Also, looking at the cmake, it looks like you use coroutines.
>
> Beast is header-only, the Jamfile and CMakeLists.txt are only for
> building the unit tests and the examples.
Well, cmake should install the library, even its header-only. It should also provide cmake packaging and/or pkg-config to help with linking against the library and its dependencies. I do this in my libraries and could send you a pull request.
But I do wonder if it should be header-only. Is it that heavily templated that `.cpp` files don’t make sense?
> As far as linking goes, you
> should only need Boost.System (for error_code).
Oh, so it doesn’t use Boost.Asio either?
> Some of the unit tests
> use coroutines, to make sure that they work and also as a tool to
> assist reaching a high level of code coverage.
>
> You shouldn't need either the Jamfile or the CMakeLists.txt found in
> the repository in order to use Beast.
>
>> I would prefer not to have an HTTP library that depends on Boost.Context
>> due to portability issues(the author refuses to support mingw).
>
> I have no plans to add Boost.Context as a dependency, Beast uses the
> Universal Asynchronous Model which abstracts the completion token (see
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3747.pdf).
Ok awesome.
Vinnie Falco
Oct 1, 2016, 9:07:32 PM10/1/16
to bo...@lists.boost.org
On Sat, Oct 1, 2016 at 8:58 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
> Well, cmake should install the library, even its header-only. It should also provide
> cmake packaging and/or pkg-config to help with linking against the library and its
> dependencies. I do this in my libraries and could send you a pull request.
> Well, cmake should install the library, even its header-only. It should also provide
> cmake packaging and/or pkg-config to help with linking against the library and its
> dependencies. I do this in my libraries and could send you a pull request.
Uhh...err... okay, I have to admit - I've been a Windows user for
quite a long time. And Windows was born with a handicap - there's no
well defined place to put libraries and header files. So I'm behind
the curve on having CMake "install" the library (what would that even
mean for Windows?)
But, its an interesting coincidence, someone already raised this point:
https://github.com/vinniefalco/Beast/pull/98
If you want to submit a pull request for the install functionality, I
think that would be great.
> But I do wonder if it should be header-only.
> Is it that heavily templated that `.cpp` files don’t make sense?
Well, yes. The algorithms for serializing and parsing messages are
generic, accepting any object that meets the type requirements. The
WebSocket stream can wrap any object that meets the requirements. This
lets the same code work both for plain connections and SSL
connections. Or use your own socket class if you want.
On a related point, header-only libraries are more convenient. They
require less configuration and mostly "just work." The first thing
people ask about non-header only libraries in a formal review is "Why
isn't this header-only?"
>> As far as linking goes, you should only need Boost.System (for error_code).
>
> Oh, so it doesn’t use Boost.Asio either?
It "uses" Boost.Asio, but since Asio is also header-only, there is
nothing to link with. However, Boost.Asio also requires Boost.System
for error_code (unless you fiddle with some macros).
Thanks
Oliver Kowalke
Oct 2, 2016, 7:01:41 AM10/2/16
to boost
2016-10-02 2:20 GMT+02:00 Paul Fultz II <pfu...@yahoo.com>:
> I would prefer not to have an HTTP library that depends on Boost.Context
> due to portability issues(the author refuses to support mingw).
>
boost.context (and boost.coroutine) support mingw - you should take a look
> I would prefer not to have an HTTP library that depends on Boost.Context
> due to portability issues(the author refuses to support mingw).
>
at boost's regression tests
Oliver Kowalke
Oct 2, 2016, 7:22:37 AM10/2/16
to boost
2016-10-02 2:45 GMT+02:00 Paul Fultz II <pfu...@yahoo.com>:
> Boost.Context only builds for mingw if you have a MSVC assembler.
no - if you build boost.context on Windows you can use MSCV or MinGW. The
library selects the assembler tool provided by the compiler, e.g. MSVC ->
MASM, MinGW -> GNU as
please take a look at boost's regression tests - maybe your informations
are up-to-date
> However, the author has refused to accept those patches unfortunately.
the patches were incorrect
> When I build boost on ubuntu for mingw, I disable Boost.Context, which
> would mean disabling Boost.Beast as well.
>
should not be a problem as long as you provide the correct properties at b2
command line (architecture, address-model, binary-format, abi ...) for
cross compiling.
it is nearly impossible to provide assembler implementations for all
combinations of architecture + address-model + abi + binary format +
compiler/assembler (especially to have those systems to develop on)?
but you are welcome to provide a correct/valid support for your
combination/needs
> Also, Boost.Asio works fine as it doesn’t use Boost.Coroutine nor
> Boost.Context.
>
boost.asio's spawn/yield uses boost.coroutine and works well too
Bjorn Reese
Oct 2, 2016, 10:26:26 AM10/2/16
to bo...@lists.boost.org
On 09/25/2016 02:26 PM, Vinnie Falco wrote:
> I noted that Boost.Http and Beast had minimal overlap. Beast provides
> low level HTTP protocol functionality while Boost.Http offers a
> turn-key HTTP server. From that discussion came some action items, I
That is a mischaracterization of the differences between Boost.Http
> I noted that Boost.Http and Beast had minimal overlap. Beast provides
> low level HTTP protocol functionality while Boost.Http offers a
> turn-key HTTP server. From that discussion came some action items, I
and Beast. Both provide low-level HTTP functionality.
Beast uses an Asio socket together with free functions to read/write
HTTP messages, whereas Boost.Http uses its own basic_socket that
contains the Asio socket. Those are just two different approaches that
provide the same functionality.
In addition to the low-level basic_socket, Boost.Http also has some
adaptors for a lower-level server side. That was done because we
recognized that the two main audiences for a C++ HTTP library would be
(1) developers writing small embedded HTTP servers, and (2) developers
using HTTP to get stuff. The latter are not particularly interested in
low level APIs, so we initially focused on the former.
> made changes to the Beast message model. I delivered a header-only
> parser with no dependencies on the rest of Beast so it could be
> re-used in Boost.Http, and informed Vinicius. I started a discussion
project was already accepted and announced by the time you announced
your intension to write a HTTP parser (which escaped my attention.)
Looking briefly at your HTTP parser I can see that it is a push parser,
which may serve your current purposes, but not those of higher-level
use cases.
The purpose of the BSoC project was to investigate the more flexible
design of pull parsers that extends more easily to higher-level
interfaces.
Vinnie Falco
Oct 2, 2016, 11:09:20 AM10/2/16
to bo...@lists.boost.org
On Sun, Oct 2, 2016 at 10:31 AM, Bjorn Reese <bre...@mail1.stofanet.dk> wrote:
> Looking briefly at your HTTP parser I can see that it is a push parser,
> which may serve your current purposes, but not those of higher-level
> use cases.
Beast offers parsing free functions that are not advertised loudly
> Looking briefly at your HTTP parser I can see that it is a push parser,
> which may serve your current purposes, but not those of higher-level
> use cases.
(because they are more complicated to use, but give callers more
control):
http://vinniefalco.github.io/beast/beast/ref/http__parse.html
http://vinniefalco.github.io/beast/beast/ref/http__async_parse.html
These functions accept any object meeting the Parser type requirements:
http://vinniefalco.github.io/beast/beast/ref/Parser.html
Although, this interface might still have as an unstated requirement,
that such a parser must be of the "push" variety.
I did some Internet searching about push versus pull parsing, both now
and a bit ago when there was chatter about Boost.Http adopting a pull
parser. All I could find was push versus pull parsing in the context
of processing XML and HTML content. I was unable to find anything
having to do with pull parsing and HTTP, with the exception of some
Rust code:
https://github.com/vinipsmaker/http-pull-parser-rs
I'm having a difficult time understanding how a pull parser would be
suitable for an algorithm that can operate on a network socket -
perhaps you can enlighten me?
> The purpose of the BSoC project was to investigate the more flexible
> design of pull parsers that extends more easily to higher-level
> interfaces.
there a discussion or some kind of example that I can study? Or can
you provide more details? If there's some sort of design flaw in Beast
that prevents users from important implementations, I would like to
find out and address it.
Regards
Vinnie Falco
Oct 2, 2016, 11:29:59 AM10/2/16
to bo...@lists.boost.org
On Sun, Oct 2, 2016 at 10:31 AM, Bjorn Reese <bre...@mail1.stofanet.dk> wrote:
> On 09/25/2016 02:26 PM, Vinnie Falco wrote:
>> Beast provides low level HTTP protocol functionality while Boost.Http offers
>> a turn-key HTTP server.
Beast originally started with a WebSocket implementation. In the
>> Beast provides low level HTTP protocol functionality while Boost.Http offers
>> a turn-key HTTP server.
WebSocket protocol, every session starts out with a HTTP
request/response sequence. The client sends a HTTP Upgrade request and
upon success the server sends a 101 Switching Protocols response. See:
https://en.wikipedia.org/wiki/HTTP/1.1_Upgrade_header#Use_with_WebSockets
I went looking for a suitable low-level HTTP library to use as a
modular component in building beast::websocket::stream. I wanted users
of Beast to be able to customize the initial handshake, including the
ability to write servers that support not only WebSocket but also HTTP
(features not present in websocketpp). I surveyed these libraries:
Boost.Http
https://github.com/BoostGSoC14/boost.http
cpp-netlib:
https://github.com/cpp-netlib/cpp-netlib
Microsoft/cpprestsdk
https://github.com/Microsoft/cpprestsdk
POCO
https://github.com/pocoproject/poco
I found that all existing libraries were unsuitable for use as a
building block for a WebSocket implementation to use for the HTTP
handshake portion of session establishment. All of them forced callers
to use a particular container concept for storing message bodies. Most
of them did not have a correct separation of concerns (for example, by
declaring the message object to also include the socket or remote IP).
There was missing client functionality (e.g. parsing responses or
sending requests). And incorrect usage of Boost.Asio idioms (for
example std::function<> instead of completion tokens). I can offer to
review those libraries publicly if any parties are interested...
This is just my opinion, but a library that claims to offer low-level
HTTP functionality should offer a good message model that includes
requests and responses, and a clean, well factored interface to send
and receive those messages. I did not find that functionality in any
of the libraries that I reviewed.
> ...
> That is a mischaracterization of the differences between Boost.Http
> and Beast. Both provide low-level HTTP functionality.
The description on https://github.com/BoostGSoC14/boost.http reads:
> and Beast. Both provide low-level HTTP functionality.
"Boost.Http"
"Embeddable C++ HTTP server"
"This library implements a core HTTP server for Boost that can be used
from resource-constrained devices to powerful machines that have
plenty of resources to make use of and can speed up the server (such
as extra ram available to pools and extra cpus available to
multithreaded servers)."
It seems we are using different definitions of low-level here. Is
there another definition that I should be using instead?
Regards
Edward Diener
Oct 2, 2016, 1:02:10 PM10/2/16
to bo...@lists.boost.org
On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>
>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>
>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>>> The library looks nice
>>
>> Thanks!
>>
>>> I tried to run the tests and install it using cmake and it both failed.
>>
>> If you could please be so kind as to create an issue on the Beast
>> GitHub repository page with the details, I will try to address it
>> right away:
>> https://github.com/vinniefalco/Beast/issues
>
> Ok, will do.
>
>>
>>> Also, looking at the cmake, it looks like you use coroutines.
>>
>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>> building the unit tests and the examples.
>
> Well, cmake should install the library, even its header-only.
Why should CMake "install" a header-only library ? Boost does not use
>
>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>
>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>>> The library looks nice
>>
>> Thanks!
>>
>>> I tried to run the tests and install it using cmake and it both failed.
>>
>> If you could please be so kind as to create an issue on the Beast
>> GitHub repository page with the details, I will try to address it
>> right away:
>> https://github.com/vinniefalco/Beast/issues
>
> Ok, will do.
>
>>
>>> Also, looking at the cmake, it looks like you use coroutines.
>>
>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>> building the unit tests and the examples.
>
> Well, cmake should install the library, even its header-only.
CMake and header-only libraries do not need to be "installed".
Oswin Krause
Oct 2, 2016, 3:24:32 PM10/2/16
to bo...@lists.boost.org, Edward Diener
Hi,
On 2016-10-02 19:01, Edward Diener wrote:
> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>
>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com>
>>> wrote:
>>>
>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com>
>>> wrote:
>>>> The library looks nice
>>>
>>> Thanks!
>>>
>>>> I tried to run the tests and install it using cmake and it both
>>>> failed.
>>>
>>> If you could please be so kind as to create an issue on the Beast
>>> GitHub repository page with the details, I will try to address it
>>> right away:
>>> https://github.com/vinniefalco/Beast/issues
>>
>> Ok, will do.
>>
>>>
>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>
>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>> building the unit tests and the examples.
>>
>> Well, cmake should install the library, even its header-only.
>
> Why should CMake "install" a header-only library ? Boost does not use
> CMake and header-only libraries do not need to be "installed".
>
to install the header files into the final location of the boost
libraries
On 2016-10-02 19:01, Edward Diener wrote:
> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>
>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com>
>>> wrote:
>>>
>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com>
>>> wrote:
>>>> The library looks nice
>>>
>>> Thanks!
>>>
>>>> I tried to run the tests and install it using cmake and it both
>>>> failed.
>>>
>>> If you could please be so kind as to create an issue on the Beast
>>> GitHub repository page with the details, I will try to address it
>>> right away:
>>> https://github.com/vinniefalco/Beast/issues
>>
>> Ok, will do.
>>
>>>
>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>
>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>> building the unit tests and the examples.
>>
>> Well, cmake should install the library, even its header-only.
>
> Why should CMake "install" a header-only library ? Boost does not use
> CMake and header-only libraries do not need to be "installed".
>
libraries
Edward Diener
Oct 2, 2016, 6:08:54 PM10/2/16
to bo...@lists.boost.org
On 10/2/2016 3:24 PM, Oswin Krause wrote:
> Hi,
>
> On 2016-10-02 19:01, Edward Diener wrote:
>> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>>
>>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com>
>>>> wrote:
>>>>
>>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com>
>>>> wrote:
>>>>> The library looks nice
>>>>
>>>> Thanks!
>>>>
>>>>> I tried to run the tests and install it using cmake and it both
>>>>> failed.
>>>>
>>>> If you could please be so kind as to create an issue on the Beast
>>>> GitHub repository page with the details, I will try to address it
>>>> right away:
>>>> https://github.com/vinniefalco/Beast/issues
>>>
>>> Ok, will do.
>>>
>>>>
>>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>>
>>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>>> building the unit tests and the examples.
>>>
>>> Well, cmake should install the library, even its header-only.
>>
>> Why should CMake "install" a header-only library ? Boost does not use
>> CMake and header-only libraries do not need to be "installed".
>>
>
> to install the header files into the final location of the boost libraries
If you follow the Boost directory structure your header files are in the
> Hi,
>
> On 2016-10-02 19:01, Edward Diener wrote:
>> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>>
>>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com>
>>>> wrote:
>>>>
>>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com>
>>>> wrote:
>>>>> The library looks nice
>>>>
>>>> Thanks!
>>>>
>>>>> I tried to run the tests and install it using cmake and it both
>>>>> failed.
>>>>
>>>> If you could please be so kind as to create an issue on the Beast
>>>> GitHub repository page with the details, I will try to address it
>>>> right away:
>>>> https://github.com/vinniefalco/Beast/issues
>>>
>>> Ok, will do.
>>>
>>>>
>>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>>
>>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>>> building the unit tests and the examples.
>>>
>>> Well, cmake should install the library, even its header-only.
>>
>> Why should CMake "install" a header-only library ? Boost does not use
>> CMake and header-only libraries do not need to be "installed".
>>
>
> to install the header files into the final location of the boost libraries
right place as soon as you unzip your library beneath the Boost libs
subdirectory. Afterward running 'b2 headers' from the Boost top-level
directory creates the links to your header files under the Boost boost
subdirectory. There is absolutely no need for anything else for a
header-only library, nor is there a need to make things more complicated
than they need to be. Of course if you are not going to follow the Boost
directory structure you will make things needlessly complicated.
Paul Fultz II
Oct 2, 2016, 7:50:49 PM10/2/16
to bo...@lists.boost.org
> On Oct 2, 2016, at 6:21 AM, Oliver Kowalke <oliver....@gmail.com> wrote:
>
> 2016-10-02 2:45 GMT+02:00 Paul Fultz II <pfu...@yahoo.com>:
>
>> Boost.Context only builds for mingw if you have a MSVC assembler.
>
>
> no - if you build boost.context on Windows you can use MSCV or MinGW. The
> library selects the assembler tool provided by the compiler, e.g. MSVC ->
> MASM, MinGW -> GNU as
> please take a look at boost's regression tests - maybe your informations
> are up-to-date
I did look at the assembly files. As there was only x86 `.S` files for elf and mach-o, I assumed they were missing, but the windows assembler are named like `*_gas.asm`. The different extension through me off. Sorry for the confusion.
>
>
>> However, the author has refused to accept those patches unfortunately.
>
>
> the patches were incorrect
>
>
>> When I build boost on ubuntu for mingw, I disable Boost.Context, which
>> would mean disabling Boost.Beast as well.
>>
>
> should not be a problem as long as you provide the correct properties at b2
> command line (architecture, address-model, binary-format, abi ...) for
> cross compiling.
Awesome!
>
> it is nearly impossible to provide assembler implementations for all
> combinations of architecture + address-model + abi + binary format +
> compiler/assembler (especially to have those systems to develop on)?
> but you are welcome to provide a correct/valid support for your
> combination/needs
Yes, I understand. I don’t expect it to support every combination out there, but mingw on linux is common.
I do wonder, if on C++11, it could have an option to fallback on threads when its on an unsupported platform. It would work, just not efficiently.
>
>
>> Also, Boost.Asio works fine as it doesn’t use Boost.Coroutine nor
>> Boost.Context.
>>
>
> boost.asio's spawn/yield uses boost.coroutine and works well too
I was unaware of this. I guess being header-only it depends on what features are used.
Thanks,
Paul
Paul Fultz II
Oct 2, 2016, 7:56:40 PM10/2/16
to bo...@lists.boost.org
> On Oct 2, 2016, at 12:01 PM, Edward Diener <eldi...@tropicsoft.com> wrote:
>
> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>
>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>>
>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>>>> The library looks nice
>>>
>>> Thanks!
>>>
>>>> I tried to run the tests and install it using cmake and it both failed.
>>>
>>> If you could please be so kind as to create an issue on the Beast
>>> GitHub repository page with the details, I will try to address it
>>> right away:
>>> https://github.com/vinniefalco/Beast/issues
>>
>> Ok, will do.
>>
>>>
>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>
>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>> building the unit tests and the examples.
>>
>> Well, cmake should install the library, even its header-only.
>
> Why should CMake "install" a header-only library ?
There are a lot of reasons for this:
1) Cmake can install configured headers in the library(such as a version or config header).
2) Cmake can install packaging information to make it easier to use and link the library. Just because it is header-only doesn’t mean it doesn’t need to link against other libraries.
> Boost does not use CMake and header-only libraries do not need to be "installed”.
Boost.Hana does use cmake and is header-only that is installed with cmake.
Paul
Edward Diener
Oct 2, 2016, 8:39:04 PM10/2/16
to bo...@lists.boost.org
On 10/2/2016 7:55 PM, Paul Fultz II wrote:
>
>> On Oct 2, 2016, at 12:01 PM, Edward Diener <eldi...@tropicsoft.com> wrote:
>>
>> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>>
>>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>>>
>>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>>>>> The library looks nice
>>>>
>>>> Thanks!
>>>>
>>>>> I tried to run the tests and install it using cmake and it both failed.
>>>>
>>>> If you could please be so kind as to create an issue on the Beast
>>>> GitHub repository page with the details, I will try to address it
>>>> right away:
>>>> https://github.com/vinniefalco/Beast/issues
>>>
>>> Ok, will do.
>>>
>>>>
>>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>>
>>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>>> building the unit tests and the examples.
>>>
>>> Well, cmake should install the library, even its header-only.
>>
>> Why should CMake "install" a header-only library ?
>
> There are a lot of reasons for this:
>
> 1) Cmake can install configured headers in the library(such as a version or config header).
>
> 2) Cmake can install packaging information to make it easier to use and link the library. Just because it is header-only doesn’t mean it doesn’t need to link against other libraries.
>
>
>> On Oct 2, 2016, at 12:01 PM, Edward Diener <eldi...@tropicsoft.com> wrote:
>>
>> On 10/1/2016 8:58 PM, Paul Fultz II wrote:
>>>
>>>> On Oct 1, 2016, at 7:39 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
>>>>
>>>> On Sat, Oct 1, 2016 at 8:20 PM, Paul Fultz II <pfu...@yahoo.com> wrote:
>>>>> The library looks nice
>>>>
>>>> Thanks!
>>>>
>>>>> I tried to run the tests and install it using cmake and it both failed.
>>>>
>>>> If you could please be so kind as to create an issue on the Beast
>>>> GitHub repository page with the details, I will try to address it
>>>> right away:
>>>> https://github.com/vinniefalco/Beast/issues
>>>
>>> Ok, will do.
>>>
>>>>
>>>>> Also, looking at the cmake, it looks like you use coroutines.
>>>>
>>>> Beast is header-only, the Jamfile and CMakeLists.txt are only for
>>>> building the unit tests and the examples.
>>>
>>> Well, cmake should install the library, even its header-only.
>>
>> Why should CMake "install" a header-only library ?
>
> There are a lot of reasons for this:
>
> 1) Cmake can install configured headers in the library(such as a version or config header).
>
> 2) Cmake can install packaging information to make it easier to use and link the library. Just because it is header-only doesn’t mean it doesn’t need to link against other libraries.
>
Whatever CMake can or cannot do, Boost should not require its use for
anything right now since their is no official support for it.
>> Boost does not use CMake and header-only libraries do not need to be "installed”.
>
> Boost.Hana does use cmake and is header-only that is installed with cmake.
Boost.Hana is part of Boost now and is "installed" when you install
Boost. In Github Boost Hana is installed when I get the Hana submodule.
If Boost Hana needs CMake in order to be installed or in order to work
that is news to me, and I am sure it would also be news to other Boost
developers.
I have no official stake in whatever debate may exist regarding Boost
Build and CMake but I think that, even if you favor CMake, you should
not be saying that CMake is necessary for Boost when it is not.
Paul Fultz II
Oct 3, 2016, 1:11:57 AM10/3/16
to bo...@lists.boost.org
> On Oct 2, 2016, at 7:38 PM, Edward Diener <eldi...@tropicsoft.com> wrote:
>
> Whatever CMake can or cannot do, Boost should not require its use for anything right now since their is no official support for it.
But it would be nice for it to support it now, as it will make it easier to use the library(especially before its accepted in boost).
>
>>> Boost does not use CMake and header-only libraries do not need to be "installed”.
>>
>> Boost.Hana does use cmake and is header-only that is installed with cmake.
>
> Boost.Hana is part of Boost now and is "installed" when you install Boost.
Its also installed with cmake.
> In Github Boost Hana is installed when I get the Hana submodule. If Boost Hana needs CMake in order to be installed or in order to work that is news to me, and I am sure it would also be news to other Boost developers.
>
> I have no official stake in whatever debate may exist regarding Boost Build and CMake but I think that, even if you favor CMake, you should not be saying that CMake is necessary for Boost when it is not.
I am not saying its required, but by having Boost.Beast support cmake makes it easier to use, and give real feedback for the review.
Paul
Oswin Krause
Oct 3, 2016, 2:25:14 AM10/3/16
to bo...@lists.boost.org
Hi,
would not move your library to the final ${prefix}/include/*? I would
find it confusing having a library that supposedly "builds" with cmake
as well as bjam but does only install with one of them.
>>
>> to install the header files into the final location of the boost
>> libraries
>
> If you follow the Boost directory structure your header files are in
> the right place as soon as you unzip your library beneath the Boost
> libs subdirectory. Afterward running 'b2 headers' from the Boost
> top-level directory creates the links to your header files under the
> Boost boost subdirectory. There is absolutely no need for anything
> else for a header-only library, nor is there a need to make things
> more complicated than they need to be. Of course if you are not going
> to follow the Boost directory structure you will make things
> needlessly complicated.
>
Does it matter what bjam would correctly be doing when "cmake install"
>> to install the header files into the final location of the boost
>> libraries
>
> If you follow the Boost directory structure your header files are in
> the right place as soon as you unzip your library beneath the Boost
> libs subdirectory. Afterward running 'b2 headers' from the Boost
> top-level directory creates the links to your header files under the
> Boost boost subdirectory. There is absolutely no need for anything
> else for a header-only library, nor is there a need to make things
> more complicated than they need to be. Of course if you are not going
> to follow the Boost directory structure you will make things
> needlessly complicated.
>
would not move your library to the final ${prefix}/include/*? I would
find it confusing having a library that supposedly "builds" with cmake
as well as bjam but does only install with one of them.
Vinnie Falco
Oct 3, 2016, 8:30:21 AM10/3/16
to bo...@lists.boost.org
It seems that we've gone a bit off-topic. The CMake discussion
certainly has value, but if I could kindly ask for that conversation
to take place in a new thread, it would be greatly appreciated.
certainly has value, but if I could kindly ask for that conversation
to take place in a new thread, it would be greatly appreciated.
Vinnie Falco
Oct 6, 2016, 2:22:51 PM10/6/16
to bo...@lists.boost.org
To get this topic hopping again, I'll share a bit of news. With help
from Howard Hinnant I've opened an issue against the Networking TS to
make sure that its algorithms do not place unnecessary requirements on
user-defined types:
http://cplusplus.github.io/LWG/lwg-active.html#2779
This will affect Beast's buffer sequence adapters (such as buffer_cat).
from Howard Hinnant I've opened an issue against the Networking TS to
make sure that its algorithms do not place unnecessary requirements on
user-defined types:
http://cplusplus.github.io/LWG/lwg-active.html#2779
This will affect Beast's buffer sequence adapters (such as buffer_cat).
Vinnie Falco
Oct 9, 2016, 7:33:19 AM10/9/16
to bo...@lists.boost.org
News from CppCon:
The video of the Beast 15 minute Lightning Talk is up:
https://www.youtube.com/watch?v=uJZgRcvPFwI
Slides for the presentation:
https://raw.githubusercontent.com/vinniefalco/Beast/master/doc/images/CppCon2016.pdf
On Thu, Oct 6, 2016 at 2:22 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
> To get this topic hopping again, I'll share a bit of news. With help
> from Howard Hinnant I've opened an issue against the Networking TS to
> make sure that its algorithms do not place unnecessary requirements on
> user-defined types:
> http://cplusplus.github.io/LWG/lwg-active.html#2779
>
> This will affect Beast's buffer sequence adapters (such as buffer_cat).
--
Follow me on Github: https://github.com/vinniefalco
The video of the Beast 15 minute Lightning Talk is up:
https://www.youtube.com/watch?v=uJZgRcvPFwI
Slides for the presentation:
https://raw.githubusercontent.com/vinniefalco/Beast/master/doc/images/CppCon2016.pdf
On Thu, Oct 6, 2016 at 2:22 PM, Vinnie Falco <vinnie...@gmail.com> wrote:
> To get this topic hopping again, I'll share a bit of news. With help
> from Howard Hinnant I've opened an issue against the Networking TS to
> make sure that its algorithms do not place unnecessary requirements on
> user-defined types:
> http://cplusplus.github.io/LWG/lwg-active.html#2779
>
> This will affect Beast's buffer sequence adapters (such as buffer_cat).
Follow me on Github: https://github.com/vinniefalco
Vinnie Falco
Oct 10, 2016, 5:31:47 PM10/10/16
to bo...@lists.boost.org
There's some great programming and design discussion related to Beast
and HTTP going on in the GitHub issues for the project. And there are
some interface changes looming. Stakeholders and interested parties
should consider participating in these discussions if they would like
to influence the outcome:
Some of the discussions:
HTTP 1 versus 2 namespacing
https://github.com/vinniefalco/Beast/issues/121
HTTP/2 design
https://github.com/vinniefalco/Beast/issues/122
Expect: 100-continue
https://github.com/vinniefalco/Beast/issues/123
`basic_headers` to make better use of allocators and string_ref
https://github.com/vinniefalco/Beast/issues/124
and HTTP going on in the GitHub issues for the project. And there are
some interface changes looming. Stakeholders and interested parties
should consider participating in these discussions if they would like
to influence the outcome:
Some of the discussions:
HTTP 1 versus 2 namespacing
https://github.com/vinniefalco/Beast/issues/121
HTTP/2 design
https://github.com/vinniefalco/Beast/issues/122
Expect: 100-continue
https://github.com/vinniefalco/Beast/issues/123
`basic_headers` to make better use of allocators and string_ref
https://github.com/vinniefalco/Beast/issues/124
Vinnie Falco
Nov 7, 2016, 11:50:30 AM11/7/16
to bo...@lists.boost.org
Recently I've added a container "class message_headers" to model the
HTTP message excluding the body. This provides more flexibility for
callers to handle the body themselves, or for more complex scenarios
such as initiating or responding to Expect: 100-continue.
Now that we have message_headers to refer to a HTTP message without
the body, the term "headers" has become ambiguous. Specifically class
basic_headers, and the Headers template argument name can be confused
with message_headers. I am strongly considering the following changes:
* Define "fields" to mean to a collection of HTTP field name / value pairs
* Define "headers" to mean a container holding "fields" plus the
request or response line specific items (method, uri, status code, and
reason phrase)
* Rename class template basic_headers to basic_fields
* Rename class template alias headers to fields
* Rename Headers in template parameter lists to Fields
* Change documentation to be consistent with these terms
This is a fairly big API change and I'm interested in hearing from
any/all stakeholders with feedback.
Very happy to hear from anyone here, or if you want please comment on
the corresponding GitHub issue:
https://github.com/vinniefalco/Beast/issues/171
Thanks
HTTP message excluding the body. This provides more flexibility for
callers to handle the body themselves, or for more complex scenarios
such as initiating or responding to Expect: 100-continue.
Now that we have message_headers to refer to a HTTP message without
the body, the term "headers" has become ambiguous. Specifically class
basic_headers, and the Headers template argument name can be confused
with message_headers. I am strongly considering the following changes:
* Define "fields" to mean to a collection of HTTP field name / value pairs
* Define "headers" to mean a container holding "fields" plus the
request or response line specific items (method, uri, status code, and
reason phrase)
* Rename class template basic_headers to basic_fields
* Rename class template alias headers to fields
* Rename Headers in template parameter lists to Fields
* Change documentation to be consistent with these terms
This is a fairly big API change and I'm interested in hearing from
any/all stakeholders with feedback.
Very happy to hear from anyone here, or if you want please comment on
the corresponding GitHub issue:
https://github.com/vinniefalco/Beast/issues/171
Thanks
Vinnie Falco via Boost
Jun 9, 2017, 10:28:20 PM6/9/17
to Boost, Vinnie Falco
Reanimating an old topic about Beast HTTP
On Wed, Sep 28, 2016 at 5:39 PM, David Sankel <cam...@gmail.com> wrote:
> I don't think a synchronous interface is necessary.
I disagree. Synchronous interface used with the non-blocking setting,
and a reactor style wait operation
(http://www.boost.org/doc/libs/1_64_0/doc/html/boost_asio/overview/core/reactor.html)
present a strong alternative to the traditional proactor model.
Specifically, using the reactor style wait operation means that no
memory buffers need to be allocated while a connection is idle /
waiting for bytes to become available. A robust reactor style
implementation can service tens of thousands of connections using
memory proportional to the number of threads rather than the number of
connections.
The technique is overlooked because of the assumption that the
asynchronous interfaces should be preferred, and the synchronous
interfaces deprecated. This impressive performance feat is not
theoretical, it is already happening in the wild, for example, this
library uses the reactor model and gets impressive performance
figures:
https://github.com/uNetworking/uWebSockets
I will be investigating these techniques in Beast, most likely as an
example server for resource-constrained devices, you can follow the
progress here:
https://github.com/vinniefalco/Beast/issues/445
On Wed, Sep 28, 2016 at 5:39 PM, David Sankel <cam...@gmail.com> wrote:
> I don't think a synchronous interface is necessary.
I disagree. Synchronous interface used with the non-blocking setting,
and a reactor style wait operation
(http://www.boost.org/doc/libs/1_64_0/doc/html/boost_asio/overview/core/reactor.html)
present a strong alternative to the traditional proactor model.
Specifically, using the reactor style wait operation means that no
memory buffers need to be allocated while a connection is idle /
waiting for bytes to become available. A robust reactor style
implementation can service tens of thousands of connections using
memory proportional to the number of threads rather than the number of
connections.
The technique is overlooked because of the assumption that the
asynchronous interfaces should be preferred, and the synchronous
interfaces deprecated. This impressive performance feat is not
theoretical, it is already happening in the wild, for example, this
library uses the reactor model and gets impressive performance
figures:
https://github.com/uNetworking/uWebSockets
I will be investigating these techniques in Beast, most likely as an
example server for resource-constrained devices, you can follow the
progress here:
https://github.com/vinniefalco/Beast/issues/445
Andrzej Krzemienski via Boost
Jun 12, 2017, 11:04:27 AM6/12/17
to bo...@lists.boost.org, Andrzej Krzemienski
2016-09-23 22:22 GMT+02:00 Vinnie Falco <vinnie...@gmail.com>:
> On Fri, Sep 23, 2016 at 11:50 AM, Niall Douglas
> <s_sour...@nedprod.com> wrote:
> ]> Almost everyone here and in the wider C++ community wants somebody to
> > take the HTTP client API from the Python Requests module
> > (http://docs.python-requests.org/en/master/) and very closely
> > replicate it in C++
>
> Hello! I just had a look at this Requests library and it is AMAZING!
> C++ most definitely needs something like this! Look at these features
> it has:
>
> International Domains and URLs
> Keep-Alive & Connection Pooling
> Sessions with Cookie Persistence
> Browser-style SSL Verification
> Basic/Digest Authentication
> Elegant Key/Value Cookies
> Automatic Decompression
> Automatic Content Decoding
> Unicode Response Bodies
> Multipart File Uploads
> HTTP(S) Proxy Support
> Connection Timeouts
> Streaming Downloads
> .netrc Support
> Chunked Requests
> Thread-safety
> Custom Body Content Workflow
> Streaming Uploads
> POST Multiple Multipart-Encoded Files
> Event Hooks
> Custom Authentication
> Streaming Requests
> SOCKS proxies
> Link Headers
> Transport Adapters
>
Hi Vinnie,
I am not well familiar with details of http, and I have not even read all
the responses to this thread, so what I say may not fully apply, but...
> Just look at that awesome list of features. The Python Requests
> library is something we should hold up as a model of how we want a
> full-featured HTTP client library to look and act.
>
I believe you when you say that the Python library is amazing.
>
> Unfortunately, even a reduced subset of these features goes far beyond
> my resources to implement. I also do not feel confident in my
> knowledge of the domain or my abilities to provide a robust C++
> interface for all of this.
>
That's ok. You do not have to do all of this. It does not have to be one
man's, one-time effort.
>
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model). I
> am sure that someone or some group with expert knowledge in creating
> robust HTTP clients could come along and build the C++ equivalent of
> Python Requests on top of Beast. It should not be controversial when I
> say that Beast offers useful functionality today.
>
Exactly. If I ever get to write some of these features, I would like to
have your low-level components, tested and ready.
I am apparently missing something. You are saying as though Boost community
were already against your low-level design. To me this is the correct
approach: start with building block, let someone else write the high level
API. I did not record much objections to this approach. Some potential
users might be disappointed they do not get more, but this does not imply
they are against your http building blocks.
>
> And yet, there are strong opinions that Beast as a low level HTTP
> building block is insufficient to be considered as part of a general
> purpose library Boost.
Really? Can anybody confirm that they strongly object to having a well
designed http building blocks library in Boost?
Maybe this is just about the name? Maybe if this was called "Http building
blocks", it would get a consensus?
> Once again I must ask, if Boost.Asio were
> proposed today would it receive the same critique? Would an absence of
> FTP and HTTP implementations make Christopher Kohlhoff's Asio library
> get rejected in a formal review?
>
To me both ASIO and having a low-level http is fine. In fact, I had
problems using Boost.HTTP because it is so coupled with ASIO. In my project
we are using a custom asynchronous task framework, and I would really
appreciate something that does not impose on me any particular framework or
programming model.
>
> The modern consensus is that C++ libraries need to become more focused
> and smaller, performing a single task and doing it really well. And
> that is exactly the design principle of Beast - model the HTTP
> message, serialize and deserialize HTTP messages synchronously or
> asynchronously. This might not satisfy the majority of use cases but
> it gives interested parties something they can work with. Who are the
> interested parties? Anyone who wants to write a generic web server. Or
> a full featured HTTP client. You want those features right, and you
> want them in Boost? Then why would we reject a library that offers the
> primitives for other people to create such high level implementations?
>
I agree with you. I just cannot see with whom you are arguing. You need
low-level libraries in order to build higher-level libraries. Is anyone
questioning this?
Does anyone think that accepting Http building blocks library into Boost
prevents adding a Python-like library in the future?
Regards,
&rzej;
> On Fri, Sep 23, 2016 at 11:50 AM, Niall Douglas
> <s_sour...@nedprod.com> wrote:
> ]> Almost everyone here and in the wider C++ community wants somebody to
> > take the HTTP client API from the Python Requests module
> > (http://docs.python-requests.org/en/master/) and very closely
> > replicate it in C++
>
> Hello! I just had a look at this Requests library and it is AMAZING!
> C++ most definitely needs something like this! Look at these features
> it has:
>
> International Domains and URLs
> Keep-Alive & Connection Pooling
> Sessions with Cookie Persistence
> Browser-style SSL Verification
> Basic/Digest Authentication
> Elegant Key/Value Cookies
> Automatic Decompression
> Automatic Content Decoding
> Unicode Response Bodies
> Multipart File Uploads
> HTTP(S) Proxy Support
> Connection Timeouts
> Streaming Downloads
> .netrc Support
> Chunked Requests
> Thread-safety
> Custom Body Content Workflow
> Streaming Uploads
> POST Multiple Multipart-Encoded Files
> Event Hooks
> Custom Authentication
> Streaming Requests
> SOCKS proxies
> Link Headers
> Transport Adapters
>
Hi Vinnie,
I am not well familiar with details of http, and I have not even read all
the responses to this thread, so what I say may not fully apply, but...
> Just look at that awesome list of features. The Python Requests
> library is something we should hold up as a model of how we want a
> full-featured HTTP client library to look and act.
>
I believe you when you say that the Python library is amazing.
>
> Unfortunately, even a reduced subset of these features goes far beyond
> my resources to implement. I also do not feel confident in my
> knowledge of the domain or my abilities to provide a robust C++
> interface for all of this.
>
That's ok. You do not have to do all of this. It does not have to be one
man's, one-time effort.
>
> I think that what Niall has done, probably inadvertently, is to
> demonstrate just how broken the Boost review process has become. We
> have Beast, which provides a great implementation of the low level
> HTTP operations (read and write a message, provide a message model). I
> am sure that someone or some group with expert knowledge in creating
> robust HTTP clients could come along and build the C++ equivalent of
> Python Requests on top of Beast. It should not be controversial when I
> say that Beast offers useful functionality today.
>
Exactly. If I ever get to write some of these features, I would like to
have your low-level components, tested and ready.
I am apparently missing something. You are saying as though Boost community
were already against your low-level design. To me this is the correct
approach: start with building block, let someone else write the high level
API. I did not record much objections to this approach. Some potential
users might be disappointed they do not get more, but this does not imply
they are against your http building blocks.
>
> And yet, there are strong opinions that Beast as a low level HTTP
> building block is insufficient to be considered as part of a general
> purpose library Boost.
Really? Can anybody confirm that they strongly object to having a well
designed http building blocks library in Boost?
Maybe this is just about the name? Maybe if this was called "Http building
blocks", it would get a consensus?
> Once again I must ask, if Boost.Asio were
> proposed today would it receive the same critique? Would an absence of
> FTP and HTTP implementations make Christopher Kohlhoff's Asio library
> get rejected in a formal review?
>
To me both ASIO and having a low-level http is fine. In fact, I had
problems using Boost.HTTP because it is so coupled with ASIO. In my project
we are using a custom asynchronous task framework, and I would really
appreciate something that does not impose on me any particular framework or
programming model.
>
> The modern consensus is that C++ libraries need to become more focused
> and smaller, performing a single task and doing it really well. And
> that is exactly the design principle of Beast - model the HTTP
> message, serialize and deserialize HTTP messages synchronously or
> asynchronously. This might not satisfy the majority of use cases but
> it gives interested parties something they can work with. Who are the
> interested parties? Anyone who wants to write a generic web server. Or
> a full featured HTTP client. You want those features right, and you
> want them in Boost? Then why would we reject a library that offers the
> primitives for other people to create such high level implementations?
>
I agree with you. I just cannot see with whom you are arguing. You need
low-level libraries in order to build higher-level libraries. Is anyone
questioning this?
Does anyone think that accepting Http building blocks library into Boost
prevents adding a Python-like library in the future?
Regards,
&rzej;
Vinnie Falco via Boost
Jun 12, 2017, 1:31:11 PM6/12/17
to Boost, Vinnie Falco
On Mon, Jun 12, 2017 at 8:04 AM, Andrzej Krzemienski via Boost
<bo...@lists.boost.org> wrote:
> I agree with you. I just cannot see with whom you are arguing. You need
> low-level libraries in order to build higher-level libraries. Is anyone
> questioning this?
There were some people who objected (I won't say their names) but I
<bo...@lists.boost.org> wrote:
> I agree with you. I just cannot see with whom you are arguing. You need
> low-level libraries in order to build higher-level libraries. Is anyone
> questioning this?
gave their opinions too much weight. I continued to develop my library
anyway.
And now, Beast is done, in the review queue, and has its review
starting July 1st (18 days).
> In my project
> we are using a custom asynchronous task framework, and I would really
> appreciate something that does not impose on me any particular framework or
> programming model.
http://vinniefalco.github.io/beast/beast/http/serializer_buffers.html
http://vinniefalco.github.io/beast/beast/http/parser_buffers.html
Thanks
Reply all
Reply to author
Forward
0 new messages