ANN: pipes-network-tls 0.1.0.0

46 views
Skip to first unread message

Renzo Carbonara

unread,
May 29, 2013, 3:05:50 AM5/29/13
to haskell-pipes
I'm happy to announce the first release of `pipes-network-tls`, a
Haskell library that enables streaming data through TLS-secured
network connections using `pipes` and `pipes-safe`.

In Hackage: http://hackage.haskell.org/package/pipes-network-tls-0.1.0.0

Code repository:
https://github.com/k0001/pipes-network-tls/tree/pipes-network-tls-0.1.0.0

The API exposed in this package is similar to that exposed by
`pipes-network` and builds on top of the `network-simple-tls` package,
which in turn is similar to `network-simple`.

The TLS support is enabled by the `tls` package, Hopefully this
library helps `tls` get more widespread use.

Currently, only TCP network connections are supported.

This is a new library that haven't been tested much, please report any
issues you might find in
https://github.com/k0001/pipes-network-tls/issues


Regards,

Renzo Carbonara.

Renzo Carbonara

unread,
May 29, 2013, 3:07:06 AM5/29/13
to haskell-pipes
On Wed, May 29, 2013 at 4:05 AM, Renzo Carbonara <gnuk...@gmail.com> wrote:
> In Hackage: http://hackage.haskell.org/package/pipes-network-tls-0.1.0.0

You can find the rendered documentation here until Hackage renders them:

http://monoid.k0001.org/haskell/pipes-network-tls/


Regards,

Renzo Carbonara.

Gabriel Gonzalez

unread,
May 29, 2013, 10:54:06 AM5/29/13
to haskel...@googlegroups.com
Wow, this is really nice! The API is really simple and easy to understand.

I tried it out and could not get an https request to succeed using this:

import Control.Proxy
import Control.Proxy.TCP.TLS

main = do
cs <- getDefaultClientSettings
connect cs "www.google.com" "443" $ \(ctx, addr) -> do
print addr
runProxy $ contextReadS ctx >-> printD

The address prints so I assume that means the connection succeeded, but
it doesn't stream bytes from the context and doesn't terminate either.

Is that normal or am I missing a key concept? Remember that I'm a
complete novice when it comes to TLS. :)

Oliver Charles

unread,
May 29, 2013, 11:12:33 AM5/29/13
to haskel...@googlegroups.com
On 05/29/2013 03:54 PM, Gabriel Gonzalez wrote:
> Wow, this is really nice! The API is really simple and easy to
> understand.
>
> I tried it out and could not get an https request to succeed using this:
>
> import Control.Proxy
> import Control.Proxy.TCP.TLS
>
> main = do
> cs <- getDefaultClientSettings
> connect cs "www.google.com" "443" $ \(ctx, addr) -> do
> print addr
> runProxy $ contextReadS ctx >-> printD
>
> The address prints so I assume that means the connection succeeded,
> but it doesn't stream bytes from the context and doesn't terminate
> either.
>
> Is that normal or am I missing a key concept? Remember that I'm a
> complete novice when it comes to TLS. :)
Don't you need to write "GET /" to the context first? You haven't issued
a HTTP request, so Google has nothing to say to you. pipes-network-tls
isn't a HTTP client - it's a TLS socket library.

Or perhaps I've misunderstood your question :)
- Ollie

Gabriel Gonzalez

unread,
May 29, 2013, 11:25:55 AM5/29/13
to haskel...@googlegroups.com

Oops!  You can tell how little I know about HTTP, too! :). That's probably my mistake, although I can't test it at the moment because I'm on my phone.

Either way this library is really useful.  I anticipate this being one of the more widely used libraries in the `pipes` ecosystem.

--
You received this message because you are subscribed to the Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipes+unsubscribe@googlegroups.com.
To post to this group, send email to haskel...@googlegroups.com.


Oliver Charles

unread,
May 29, 2013, 11:34:54 AM5/29/13
to haskel...@googlegroups.com
On 05/29/2013 04:25 PM, Gabriel Gonzalez wrote:

Oops!  You can tell how little I know about HTTP, too! :). That's probably my mistake, although I can't test it at the moment because I'm on my phone.


Don't worry, I started work on pipes-http this morning ;) Which brings up another topic - was anyone already working on this? It seems like an "obvious" gap in the set of pipes libraries, but a quick search on GitHub didn't show anyone even prototyping this.

Gabriel Gonzalez

unread,
May 29, 2013, 11:36:53 AM5/29/13
to haskel...@googlegroups.com

Renzo and Jeremy Shaw are the only people I know that might be working on the same thing.

--
You received this message because you are subscribed to the Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipe...@googlegroups.com.

Renzo Carbonara

unread,
May 29, 2013, 3:30:14 PM5/29/13
to haskel...@googlegroups.com
On Wed, May 29, 2013 at 12:12 PM, Oliver Charles <ol...@ocharles.org.uk> wrote:
> On 05/29/2013 03:54 PM, Gabriel Gonzalez wrote:
>>
>> Wow, this is really nice! The API is really simple and easy to
>> understand.

I'm glad you like it :)


>> I tried it out and could not get an https request to succeed using this:
>>
>> import Control.Proxy
>> import Control.Proxy.TCP.TLS
>>
>> main = do
>> cs <- getDefaultClientSettings
>> connect cs "www.google.com" "443" $ \(ctx, addr) -> do
>> print addr
>> runProxy $ contextReadS ctx >-> printD
>>
>> The address prints so I assume that means the connection succeeded, but it
>> doesn't stream bytes from the context and doesn't terminate either.
>>
>> Is that normal or am I missing a key concept? Remember that I'm a
>> complete novice when it comes to TLS. :)
>
> Don't you need to write "GET /" to the context first? You haven't issued a
> HTTP request, so Google has nothing to say to you. pipes-network-tls isn't a
> HTTP client - it's a TLS socket library.

Yes, this is exactly what is going on.

If `connect` ever performs the given callback, then it means that the
connection has been successfully established and a TLS handshake has
been performed for you. Then is up to you to decide how to use the
connection.

Here's a complete example that issues an HTTPS request to Google:

import qualified Data.ByteString.Char8 as B
import Control.Proxy ((>->))
import qualified Control.Proxy as P
import qualified Control.Proxy.TCP.TLS as T


main = do
cs <- T.getDefaultClientSettings
T.connect cs "www.google.com" "443" $ \(ctx,_) -> do
P.runProxy $ P.fromListS [B.pack "GET / HTTP/1.0\r\n\r\n"]
>-> T.contextWriteD ctx
P.runProxy $ T.contextReadS ctx
>-> P.useD B.putStr

Regards,

Renzo.

Renzo Carbonara

unread,
May 29, 2013, 4:29:45 PM5/29/13
to haskel...@googlegroups.com
I am experimenting in that field, too. I'll tell you what I have so
far, which is mostly just ideas and some bits of code. As many others,
I'm stuck without `pipes-parse` and `pipes-attoparsec`, but luckily
those are coming out really soon :)

My plan currently looks like this:

First, I think that it would be nice if we manage to split the HTTP
problem in three core and reusable packages:

- `pipes-www-client`: This package would be the pipes counterpart to
`http-conduit` and `http-streams`, that is, an HTTP(S) client,
hopefully lightweight. And it would be built on top of
`pipes-attoparsec`, `pipes-network`, `pipes-network-tls`,
`pipes-zlib`, among other non-pipes related packages.

- `pipes-www-server`: This would be an HTTP server built with the same
tools used by `pipes-www-client`. Jeremy Shaw is working in his
`hyperdrive` pipes-based HTTP server, and he has quite some experience
in this field. Hopefully we can work on this together and lay down a
healthy foundation for future HTTP projects, and maybe `hyperdrive`
itself can occupy this place.

- `pipes-www-core`: I'd like to put here as many reusable parts as
possible. Say types and the core API that defines how to deal with a
Request, so that it can be used by other packages different than
`pipes-www-server`. That is, I want this package to be similar to
conduit's WAI, but hopefully covering both server-side and client-side
concerns. I don't know if this makes sense, maybe it doesn't, but I'll
think more about it.

I'm particularly interested in getting `pipes-www-core` and
`pipes-www-client` right now, since I'll need to use them in the real
world soon :)

I've been thinking about the best way to deal with parsing HTTP
Request and HTTP Responses in a compositional manner, and I've
identified problems similar to those you had when writing `pipes-tar`.
My current idea is that within the “incoming HTTP message consumer”
pipeline I'll parse the HTTP message header and handle that header to
an user-provided function that will return an apropiate `Consumer p
ByteString m r` able to parse the rest of the incoming message body.
I've made some experiments with this, but nothing serious until
`pipes-parse` and `pipes-attoparsec` are out.

Given the context of this conversation, I guess that by `pipes-http`
you are referring to “client side HTTP”, right? Let's work together on
this. In this project I started, trying to work my way towards
`pipes-www-server`, I already have some types and attoparsec parsers
that I plan to use: https://github.com/k0001/seldom (most of the
original code there belongs to the Snap and Yesod team, I'm thankful
to them for sharing the code as they do).


Regards,

Renzo.

Renzo Carbonara

unread,
May 29, 2013, 4:37:04 PM5/29/13
to haskel...@googlegroups.com
On Wed, May 29, 2013 at 5:29 PM, Renzo Carbonara <gnuk...@gmail.com> wrote:
> I've been thinking about the best way to deal with parsing HTTP
> Request and HTTP Responses in a compositional manner, and I've
> identified problems similar to those you had when writing `pipes-tar`.
> My current idea is that within the “incoming HTTP message consumer”
> pipeline I'll parse the HTTP message header and handle that header to
> an user-provided function that will return an apropiate `Consumer p
> ByteString m r` able to parse the rest of the incoming message body.

Yes, yes… I know, the `respond` category is a better solution to this problem :)

Gabriel Gonzalez

unread,
May 29, 2013, 4:55:03 PM5/29/13
to haskel...@googlegroups.com
Yeah, I think this is where the `respond` category really shines.  I think the `pipes-tar` solution is really a model for this kind of thing and we should adopt it for cases where you want the user to inject pipes that have access to the same upstream interface.

It's also nice because it behaves like a dual-purpose interface: users can use it just to inspect the headers without parsing the body or they can use the headers to process the body.

I recommend avoid continuation passing style because continuations are not first-class.  They are not monadic (in the pipe monad sense), they aren't 'hoist'able, 'lift'able, or reifiable into a stream without concurrency.  Continuations were precisely the problem that `pipes` was originally designed to avoid where iteratee libraries would have one first-class data type (i.e. the sink) and everything else was continuations built on top of that data type.  Pipes are the only streaming library where the 'respond' continuations are themselves first class streaming objects, so it would be a shame not to use them! :)


--
You received this message because you are subscribed to the Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipe...@googlegroups.com.

Gabriel Gonzalez

unread,
May 29, 2013, 5:16:50 PM5/29/13
to haskel...@googlegroups.com
I got it to work!  In fact, I tested it on Windows, too, and I noticed one thing that you can improve in the documentation: your example should use `withSocketsDo` from the `Network` module so that it also works on Windows.  Without this you get a really unhelpful error message just saying that `getAddrInfo` failed.

`withSocketsDo` is always safe to use, even on Linux, where it just becomes `id`, so there is no harm in telling your users to always use it.  You might even consider re-exporting `withSocketsDo` in your networking libraries.

Anyway, the good news is that once you add  `withSocketsDo` then everything works flawlessly on Windows, too.  I was able to issue an HTTPS request to Google without any problems.


Renzo Carbonara

unread,
May 29, 2013, 5:27:10 PM5/29/13
to haskel...@googlegroups.com
On Wed, May 29, 2013 at 6:16 PM, Gabriel Gonzalez <gabri...@gmail.com> wrote:
> I got it to work! In fact, I tested it on Windows, too, and I noticed one
> thing that you can improve in the documentation: your example should use
> `withSocketsDo` from the `Network` module so that it also works on Windows.
> Without this you get a really unhelpful error message just saying that
> `getAddrInfo` failed.
>
> `withSocketsDo` is always safe to use, even on Linux, where it just becomes
> `id`, so there is no harm in telling your users to always use it. You might
> even consider re-exporting `withSocketsDo` in your networking libraries.
>
> Anyway, the good news is that once you add `withSocketsDo` then everything
> works flawlessly on Windows, too. I was able to issue an HTTPS request to
> Google without any problems.

Great!! This is good to know. Thanks.

I'll update the documentation as you suggest, and export
`withSocketsDo` too. I'm not a Windows user myself so I usually forget
about it :P



Regards,

Renzo.

Renzo Carbonara

unread,
May 29, 2013, 5:29:22 PM5/29/13
to haskel...@googlegroups.com
On Wed, May 29, 2013 at 5:55 PM, Gabriel Gonzalez <gabri...@gmail.com> wrote:
> Yeah, I think this is where the `respond` category really shines. I think
> the `pipes-tar` solution is really a model for this kind of thing and we
> should adopt it for cases where you want the user to inject pipes that have
> access to the same upstream interface.
>
> It's also nice because it behaves like a dual-purpose interface: users can
> use it just to inspect the headers without parsing the body or they can use
> the headers to process the body.
>
> I recommend avoid continuation passing style because continuations are not
> first-class. They are not monadic (in the pipe monad sense), they aren't
> 'hoist'able, 'lift'able, or reifiable into a stream without concurrency.
> Continuations were precisely the problem that `pipes` was originally
> designed to avoid where iteratee libraries would have one first-class data
> type (i.e. the sink) and everything else was continuations built on top of
> that data type. Pipes are the only streaming library where the 'respond'
> continuations are themselves first class streaming objects, so it would be a
> shame not to use them! :)

Yes, you are right, and I'll go the `respond` way. I just mentioned
the continuation approach since that's what I had in mind before the
whole `pipes-tar` enlightening happened, and I haven't yet thought in
detail about this new approach.

I'll update my code and push it to GitHub so we can continue working
openly on it :)


Regards,

Renzo Carbonara.
Reply all
Reply to author
Forward
0 new messages