SSL support in INN 2.5

5 views
Skip to first unread message

Iulius

unread,
Apr 22, 2007, 12:38:48 PM4/22/07
to
Hi,

Everything seems to work fine, except the use of SSL.
I cannot find how to make it work with INN 2.5.
With the 2.4.3 version, I use « nnrpd-ssl » and everything
works well but there is something I must be missing with the
2.5 version.


First of all, I have INN and nnrpd launched on port 119.
Everything is fine.
I want to add a nnrpd on port 563 in order to listen to
SSL connections.

I do:

nnrpd -D -c /home/news/etc/readers-ssl.conf -p 563 -S

Then I am told that innbind cannot bind to that port...
After a while, I understand why: innbind is only allow
to bind to port 119 and 433 and all the ports above 1024.
Why the port 433?!? It is not nntps!


Well, I then do:

nnrpd -D -c /home/news/etc/readers-ssl.conf -p 433 -S

and it binds well. However, I have a little problem:
I cannot connect.


% cat /home/news/etc/sasl.conf
tls_cert_file: /home/news/lib/cert.pem
tls_key_file: /home/news/lib/key.pem
tls_ca_path: /home/news/lib
tls_ca_file: /home/news/lib/cert.pem

I generated these files with « make cert ».
Is « tls_ca_file » useful (and correct, here) if I only have
an auto-generated certificate?
By the way, it is very strange that they are put in /home/news/lib
(not a right place)...

It looks as though that file (in pathetc) were not read:

Apr 22 18:28:11 ks39874 nnrpd[1358]: error initializing TLS: [CA_file: ] [CA_path: ] [cert_file: ] [key_file: ]
Apr 22 18:28:11 ks39874 nnrpd[1358]: times user 0.008 system 0.000 idle 0.000 elapsed 0.008
Apr 22 18:28:11 ks39874 nnrpd[1358]: time 9 nntpwrite 0(1)

Do you have any idea regarding that?

Thanks a lot.

Regards,

--
Iulius

« Aequum est ut cuius participauit lucrum, participet et damnun. »

Russ Allbery

unread,
Apr 22, 2007, 4:04:54 PM4/22/07
to
Iulius <iul...@nom-de-mon-site.com.invalid> writes:

> First of all, I have INN and nnrpd launched on port 119.
> Everything is fine.
> I want to add a nnrpd on port 563 in order to listen to
> SSL connections.

> I do:

> nnrpd -D -c /home/news/etc/readers-ssl.conf -p 563 -S

> Then I am told that innbind cannot bind to that port...
> After a while, I understand why: innbind is only allow
> to bind to port 119 and 433 and all the ports above 1024.
> Why the port 433?!? It is not nntps!

433 is the other transport NNTP port. You're the first person who's tried
to use nnrpd -D with SSL in CURRENT so far as I know, so no one had
thought about letting it bind to that port.

> % cat /home/news/etc/sasl.conf
> tls_cert_file: /home/news/lib/cert.pem
> tls_key_file: /home/news/lib/key.pem
> tls_ca_path: /home/news/lib
> tls_ca_file: /home/news/lib/cert.pem

CURRENT doesn't use sasl.conf any more (really, nothing should have). Set
the options in inn.conf instead. (I think the name changed slightly to
drop the underscores.)

--
Russ Allbery (r...@stanford.edu) <http://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<http://www.eyrie.org/~eagle/faqs/questions.html> explains why.

Iulius

unread,
Apr 22, 2007, 5:19:43 PM4/22/07
to
En réponse à Russ Allbery :

>> Why the port 433?!? It is not nntps!
>
> 433 is the other transport NNTP port.

All right. I didn't know that « NNSP » protocol (news without reading).
Is it used by known news servers?


> You're the first person who's tried
> to use nnrpd -D with SSL in CURRENT so far as I know, so no one had
> thought about letting it bind to that port.

Well, would it be please possible to add the port 563 when you have
time?

/* Make sure that we're allowed to bind to that port. */
- if (port < 1024 && port != 119 && port != 433 && port != INND_PORT)
+ if (port < 1024 && port != 119 && port != 433 && port != 563
+ && port != INND_PORT)
die("cannot bind to restricted port %hu in %s", port, spec);


> CURRENT doesn't use sasl.conf any more (really, nothing should have). Set
> the options in inn.conf instead. (I think the name changed slightly to
> drop the underscores.)

Oh! I did not notice that change.
Would it be please possible to update samples/inn.conf.in, providing
the new options? (it would be easier for people who migrate)
I notice the documentation <http://www.eyrie.org/~eagle/software/inn/docs/inn.conf.html>
mentions that. Thanks for having pointed it to me!
By the way, there is a little mistake: « The default value is pathnews/lib/cert.pem. »
should be « key.pem » for « tlskeyfile ».
But... why is it called a « default value » if it is unset when I run « nnrpd -S »?


Well, only for the pleasure:

Apr 22 22:54:41 trigofacile nnrpd[11274]: starttls: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) no authentication
Apr 22 22:54:41 trigofacile nnrpd[11274]: goret.via.ecp.fr (138.195.145.22) connect
Apr 22 22:54:44 trigofacile nnrpd[11274]: goret.via.ecp.fr user julien
Apr 22 22:54:46 trigofacile nnrpd[11274]: goret.via.ecp.fr group junk 0


It works well! I am glad to see that (and I will peacefully go to sleep!!).
I think everything is fine now for an operational news server. As I consider
CURRENT INN rather stable, I plan to provide a news access to French-speaking
people (only French newsgroups indeed) on a personal server. It will also
be useful for me to better test a working INN when a new CURRENT version
is released :)

--
Iulius

« Il ne faut jamais gifler un sourd : il perd la moitié du plaisir.
Il sent la gifle mais il ne l'entend pas. » (Georges Courteline)

Russ Allbery

unread,
Apr 22, 2007, 7:30:20 PM4/22/07
to
Iulius <iul...@nom-de-mon-site.com.invalid> writes:
> En réponse à Russ Allbery :

>>> Why the port 433?!? It is not nntps!
>>
>> 433 is the other transport NNTP port.

> All right. I didn't know that « NNSP » protocol (news without reading).
> Is it used by known news servers?

Yes, occasionally, when they want to run innd and nnrpd on separate ports.

>> You're the first person who's tried
>> to use nnrpd -D with SSL in CURRENT so far as I know, so no one had
>> thought about letting it bind to that port.

> Well, would it be please possible to add the port 563 when you have
> time?

> /* Make sure that we're allowed to bind to that port. */
> - if (port < 1024 && port != 119 && port != 433 && port != INND_PORT)
> + if (port < 1024 && port != 119 && port != 433 && port != 563
> + && port != INND_PORT)
> die("cannot bind to restricted port %hu in %s", port, spec);

Yup. Will do.

>> CURRENT doesn't use sasl.conf any more (really, nothing should have). Set
>> the options in inn.conf instead. (I think the name changed slightly to
>> drop the underscores.)

> Oh! I did not notice that change.
> Would it be please possible to update samples/inn.conf.in, providing
> the new options? (it would be easier for people who migrate)

Huh, I could have sworn they were. Yes, they should be added.

> I notice the documentation <http://www.eyrie.org/~eagle/software/inn/docs/inn.conf.html>
> mentions that. Thanks for having pointed it to me!
> By the way, there is a little mistake: « The default value is pathnews/lib/cert.pem. »
> should be « key.pem » for « tlskeyfile ».
> But... why is it called a « default value » if it is unset when I run « nnrpd -S »?

No idea. I'll try to investigate if I can find some time. (Really,
really hoping that I can find some time to spend on INN soon, since it's
been far too long. Things are getting more hopeful in that area, though.)

Thank you very much for your testing and reports!

Iulius

unread,
Apr 23, 2007, 9:10:41 AM4/23/07
to
En réponse à Russ Allbery :
>> Well, would it be please possible to add the port 563 when you have
>> time?
>
> Yup. Will do.

Thanks!


>> Would it be please possible to update samples/inn.conf.in, providing
>> the new options? (it would be easier for people who migrate)
>
> Huh, I could have sworn they were. Yes, they should be added.

--- trunk/samples/inn.conf.in (revision 7606)
+++ trunk/samples/inn.conf.in (working copy)
@@ -61,13 +61,13 @@
enableoverview: true
groupbaseexpiry: true
mergetogroups: false
+nfswriter: false
overcachesize: 15
#ovgrouppat:
storeonxref: true
useoverchan: false
wireformat: false
xrefslave: false
-nfswriter: false

# Reading

@@ -76,14 +76,18 @@
clienttimeout: 600
initialtimeout: 10
msgidcachesize: 10000
+nfsreader: false
+nfsreaderdelay: 60
nnrpdcheckart: true
+nnrpdloadlimit: 16
noreader: false
readerswhenstopped: false
readertrack: false
-nfsreader: false
-nfsreaderdelay: 60
+#tlscafile:
+#tlscapath:
+#tlscertfile: @prefix@/lib/cert.pem
+#tlskeyfile: @prefix@/lib/key.pem
tradindexedmmap: true
-nnrpdloadlimit: 16

# Reading -- Keyword Support
#
@@ -143,6 +147,7 @@
nnrpdoverstats: false
nntpactsync: 200
nntplinklog: false
+#stathist: @LOGDIR@
status: 0
timer: 0

Morevover:

* dontrejectfiltered should be between bindaddress6
and hiscachesize in inn.conf manpage.

* The following options should be in the reading section
(and not in article storage) in inn.conf manpage,
and at their right alphabetical place:
nfsreader
nfsreaderdelay
msgidcachesize
tradindexedmmap

* You can also review the alphabetical sorting in article
storage and readin (inn.conf manpage).


>> I notice the documentation <http://www.eyrie.org/~eagle/software/inn/docs/inn.conf.html>
>> mentions that. Thanks for having pointed it to me!
>> By the way, there is a little mistake: « The default value is pathnews/lib/cert.pem. »
>> should be « key.pem » for « tlskeyfile ».

In innconf.c:

if (innconf->tlscertfile == NULL)
innconf->tlscertfile = concatpath(innconf->pathnews, "lib/cert.pem");
if (innconf->tlskeyfile == NULL)
- innconf->tlskeyfile = concatpath(innconf->pathnews, "lib/cert.pem");
+ innconf->tlskeyfile = concatpath(innconf->pathnews, "lib/key.pem");

It should be better (it is more consistent with the behaviour of « make cert »).
And also change inn.conf manpage to tell that.


>> But... why is it called a « default value » if it is unset when I run « nnrpd -S »?
>
> No idea. I'll try to investigate if I can find some time.

Found it:

#ifdef HAVE_SSL
{ K(tlscafile), STRING ("") },
{ K(tlscapath), STRING ("") },
{ K(tlscertfile), STRING ("") },
{ K(tlskeyfile), STRING ("") },
#endif /* HAVE_SSL */

They should be STRING (NULL)...


> Thank you very much for your testing and reports!

You're welcome.


By the way, I understand why lots of INN do not have control articles enabled
by default even though they have a pgp/gpg implementation.
I also have @DO_PGPVERIFY@=false by default.
Yet, I have gpg installed (binary /usr/bin/gpgv).
I believe that $PATH_GPGV is unset; that is why @DO_PGPVERIFY@=false.
What can be done instead of that? `where gpgv`? (and also pgp/gpg)

--
Iulius

« Ad iura renunciata, non datur regressus. »

Kachun Lee

unread,
May 3, 2007, 7:06:49 PM5/3/07
to
"Iulius" <iul...@nom-de-mon-site.com.invalid> wrote in message
news:f0g32k$83v$1...@guepard.ecp.fr...

> Hi,
>
> Everything seems to work fine, except the use of SSL.
> I cannot find how to make it work with INN 2.5.
> With the 2.4.3 version, I use « nnrpd-ssl » and everything
> works well but there is something I must be missing with the
> 2.5 version.
> [snip]

Hi,

This may not be exactly the same issue started this thread... I think there
is a dead lock issue with the current SSL code in NNRPD. I meant to bring
this up several times, but always missed the opportunity.

Basically, the SSL_read cannot be mixed with select(...) as in the current
code. SSL communicates in its own data blocks and hand shaking. The
do_readline using SSL_read could return, but still with a partial line in
the SSL_read buffer. Then the server SSL routine would sit there waiting for
completion of that data block while NNRPD sat at the select(...) routine
waiting for more data from the server.

OE as NNTP client caused easy dead locks with multiple return lines commands
such as XOV. Agent seemed OK... they probably coordinated their lines with
the SSL blocks. And, using any SSL wrappers should definitely has problem,
since they would not know to align their data blocks with line breaks.

The correct fix probably was to make sure the full SSL block was read from
SSL_read before going into the select(...) wait. Nevertheless, for general
release, it would be very hard to guarantee the interaction between the SSL
IO code and the raw select IO, especially with so many different platforms.

I took the short route of just bypass the select(...) wait (see patch
below). Unlike INND with multiple threads, the select on NNRPD just waiting
on a single file descriptor, so it is not really essential with blocked read
like SSL_read. Using an alarm signal around SSL_read for non active timeout,
we has been running a patched version of NNRPD in SSL without the dead lock
condition for quite many months. Of course, without the select(...) wait,
the IDLE timer stat won't be collected..

Kachun

*** line.c.orig Thu May 3 02:07:33 2007
--- line.c Thu May 3 15:01:39 2007
***************
*** 141,146 ****
--- 141,150 ----
line->allocated = newsize;
}
}
+
+ #ifdef HAVE_SSL
+ if (tls_conn == NULL) {
+ #endif
/* wait for activity on stdin, updating timer stats as we
* go */
do {
***************
*** 157,162 ****
--- 161,169 ----
syswarn("%s can't select", ClientHost);
return RTtimeout;
}
+ #ifdef HAVE_SSL
+ }
+ #endif


Julien ÉLIE

unread,
May 4, 2007, 9:46:53 AM5/4/07
to
En réponse à Kachun Lee :

> This may not be exactly the same issue started this thread... I think there
> is a dead lock issue with the current SSL code in NNRPD. I meant to bring
> this up several times, but always missed the opportunity.

Do you think it could possibly be linked to:
http://groups.google.fr/group/news.software.nntp/browse_frm/thread/de6269dab752204c
https://bugzilla.mozilla.org/show_bug.cgi?id=253523


> OE as NNTP client caused easy dead locks with multiple return lines commands
> such as XOV. Agent seemed OK... they probably coordinated their lines with
> the SSL blocks.

And Thunderbird too?

--
Julien ÉLIE

« Il vaut mieux un tapis persan volé qu'un tapis volant percé ! » (Astérix)

Julien ÉLIE

unread,
May 4, 2007, 2:50:31 PM5/4/07
to
En réponse à Kachun Lee :
> *** line.c.orig Thu May 3 02:07:33 2007
> --- line.c Thu May 3 15:01:39 2007

Well, I tried to patch my INN CURRENT version but it failed.
Indeed, nntp worked very well, but not nntp/ssl: I couldn't
connect any longer and timeouted.

May 4 20:38:06 news nnrpd[28088]: starttls: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) no authentication
May 4 20:38:06 news nnrpd[28088]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr (90.27.33.202) connect
May 4 20:38:06 news nnrpd[28088]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr timeout short
May 4 20:38:06 news nnrpd[28088]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr times user 0.044 system 0.012 idle 0.000 elapsed 0.272
May 4 20:38:06 news nnrpd[28088]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr time 274 nntpwrite 0(2)
May 4 20:38:07 news nnrpd[28089]: starttls: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) no authentication
May 4 20:38:07 news nnrpd[28089]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr (90.27.33.202) connect
May 4 20:38:07 news nnrpd[28089]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr timeout short
May 4 20:38:07 news nnrpd[28089]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr times user 0.040 system 0.016 idle 0.000 elapsed 0.188
May 4 20:38:07 news nnrpd[28089]: amontpellier-157-1-178-202.w90-27.abo.wanadoo.fr time 189 nntpwrite 0(2)
May 4 20:38:08 news nnrpd[28090]: starttls: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) no authentication
[...]

> ***************
> *** 141,146 ****
> --- 141,150 ----
> line->allocated = newsize;
> }
> }
> +
> + #ifdef HAVE_SSL
> + if (tls_conn == NULL) {
> + #endif
> /* wait for activity on stdin, updating timer stats as we
> * go */
> do {

All right for that part.


> ***************
> *** 157,162 ****
> --- 161,169 ----
> syswarn("%s can't select", ClientHost);
> return RTtimeout;
> }
> + #ifdef HAVE_SSL
> + }
> + #endif

I do not see that. Which version of INN do you use?
In both 2.4.x and 2.5, I see:

/* wait for activity on stdin, updating timer stats as we
* go */
do {

struct timeval t;

FD_ZERO(&rmask);
FD_SET(STDIN_FILENO, &rmask);
t.tv_sec = timeout;
t.tv_usec = 0;
TMRstart(TMR_IDLE);
i = select(STDIN_FILENO + 1, &rmask, NULL, NULL, &t);
TMRstop(TMR_IDLE);
if (i == -1 && errno != EINTR) {
syswarn("%s can't select", Client.host);
return RTtimeout;
}
} while (i == -1);

/* if stdin didn't select, we must have timed out */
if (i == 0 || !FD_ISSET(STDIN_FILENO, &rmask))
return RTtimeout;
count = line_doread(where,
line->allocated - (where - line->start));

I did that:

> return RTtimeout;
> }
> } while (i == -1);


> + #ifdef HAVE_SSL
> + }
> + #endif

But I believe it is not what you did.
So where is the end of your do/while statement?

--
Julien ÉLIE

« Ta vie ne tient qu'à un fil, Téléféric ! » (Astérix)

Kachun Lee

unread,
May 4, 2007, 7:00:17 PM5/4/07
to
"Julien ÉLIE" <iul...@nom-de-mon-site.com.invalid> wrote in message
news:f1fdge$62j$1...@news.trigofacile.com...

Hi,

Since the problem mentioned in your link was related to 'posting', I don't
think it was related to the partial line condition. At the time, I could
clearly trace the problem when XOV list or article returning were larger
than 32K, which was the SSL block size.

However, I believe it could be related to the second point I mentioned.
Underneath the SSL_write and SSL_read was a whole layer of creating and
transfering the SSL data blocks and ack'ing implemented by raw IO calls. I
definitely can imagine interference by outside select(...) calls.

It has been quite many months, but I recalled Thunderbird was one of the
clients reported with the hanging problems. And we have not had reported
issue with Thunderbird after the patch.

Kachun


Kachun Lee

unread,
May 4, 2007, 7:09:39 PM5/4/07
to
Hi,

We ran a heavy modified INN branched out before v2. However, I browered the
SSL code from v2.4.3. Before I posted, I did a quick check and verify the
code in v2.5 was similar. I was trying to create the diff with v2.5, but I
put the closing bracket into the wrong line. These are the lines in my
source...

#ifdef HAVE_SSL


/* wait for activity on stdin, updating timer stats as we
* go */

if (tls_conn == NULL) {
#endif
do {
struct timeval t;

FD_ZERO(&rmask);
FD_SET(STDIN_FILENO, &rmask);
t.tv_sec = timeout;
t.tv_usec = 0;

/*TMRstart(TMR_IDLE);*/


i = select(STDIN_FILENO + 1, &rmask, NULL, NULL, &t);

/*TMRstop(TMR_IDLE);*/


if (i == -1 && errno != EINTR) {

syslog(LOG_WARNING, "%s can't select", ClientHost);
return RTtimeout;
}
} while (i == -1);

/* if stdin didn't select, we must have timed out */
if (i == 0 || !FD_ISSET(STDIN_FILENO, &rmask))
return RTtimeout;

#ifdef HAVE_SSL
}
#endif


"Julien ÉLIE" <iul...@nom-de-mon-site.com.invalid> wrote in message
news:f1fv9o$v5h$1...@news.trigofacile.com...


> En réponse à Kachun Lee :
>> *** line.c.orig Thu May 3 02:07:33 2007
>> --- line.c Thu May 3 15:01:39 2007
>
> Well, I tried to patch my INN CURRENT version but it failed.
> Indeed, nntp worked very well, but not nntp/ssl: I couldn't
> connect any longer and timeouted.

Hi,

We ran a heavy modified INN branched out before v2. However, I browered the
SSL code from v2.4.3. Before I posted, I did a quick check and verify the
code in v2.5 was similar. I was trying to create the diff with v2.5, and put
the closing bracket into the wrong line... sorry about that.

You need to move the closing bracket few lines down. I copy and pasted the
loop from my working source code below.

Kachun

#ifdef HAVE_SSL
if (tls_conn == NULL) {


#endif
/* wait for activity on stdin, updating timer stats as we
* go */
do {

struct timeval t;

FD_ZERO(&rmask);
FD_SET(STDIN_FILENO, &rmask);
t.tv_sec = timeout;
t.tv_usec = 0;

/*TMRstart(TMR_IDLE);*/


i = select(STDIN_FILENO + 1, &rmask, NULL, NULL, &t);

/*TMRstop(TMR_IDLE);*/


if (i == -1 && errno != EINTR) {

syslog(LOG_WARNING, "%s can't select", ClientHost);
return RTtimeout;
}
} while (i == -1);

/* if stdin didn't select, we must have timed out */
if (i == 0 || !FD_ISSET(STDIN_FILENO, &rmask))
return RTtimeout;

#ifdef HAVE_SSL
}
#endif

[...]


Julien ÉLIE

unread,
May 5, 2007, 6:58:25 AM5/5/07
to
En réponse à Kachun Lee :
> I was trying to create the diff with v2.5, but I
> put the closing bracket into the wrong line.

Don't worry.


> These are the lines in my source...

Thanks. INN compiles and nnrpd/ssl seems to work.
I will tell you if it fixes the hanging problems I have
always had with nnrpd/ssl.

--
Julien ÉLIE

« -- Tu crois qu'ils auront du sanglier, dis ?
-- Faut pas te faire d'idées ; plus les armées sont puissantes,
plus la nourriture est mauvaise. Ça maintient les guerriers de mauvaise humeur.
-- ...
-- Je ne pensais pas que l'armée romaine était aussi puissante ! » (Astérix)

Julien ÉLIE

unread,
May 5, 2007, 1:57:54 PM5/5/07
to
En réponse à Kachun Lee :
> Since the problem mentioned in your link was related to 'posting', I don't
> think it was related to the partial line condition.
> However, I believe it could be related to the second point I mentioned.

And it was!


> Underneath the SSL_write and SSL_read was a whole layer of creating and
> transfering the SSL data blocks and ack'ing implemented by raw IO calls. I
> definitely can imagine interference by outside select(...) calls.

I think you're right since I no longer have problems of hangings
when posting with Thunderbird over NNTP/SSL.


> It has been quite many months, but I recalled Thunderbird was one of the
> clients reported with the hanging problems. And we have not had reported
> issue with Thunderbird after the patch.

That's great. And I can tell you it works well with me too.

Thanks a lot for having taken the time to provide that patch!
I can now open my news server to people only with secured NNTP connections
(I was indeed reluctant to only provide NNTP/SSL because of these very
nasty hangings; with that patch, I know it works well -- and I do not
worry for the idle timer -- so I can only provide NNTP/SSL to the
future users of my public news server.)

--
Julien ÉLIE

« Gynécologue, c'est un métier accessible aux sourds. En effet,
il n'y a rien à entendre et on peut lire sur les lèvres. » (Coluche)

Julien ÉLIE

unread,
May 6, 2007, 6:19:24 AM5/6/07
to
En réponse à Kachun Lee :
> We ran a heavy modified INN branched out before v2.

If not indiscreet, what kind of modifications not CURRENTly
implemented in INN did you make?

--
Julien ÉLIE

« Nemo auditur turpitudinem allegans. »

Kachun Lee

unread,
May 7, 2007, 12:54:03 PM5/7/07
to
"Julien ÉLIE" <iul...@nom-de-mon-site.com.invalid> wrote in message
news:f1ka3e$l90$1...@news.trigofacile.com...

> En réponse à Kachun Lee :
>> We ran a heavy modified INN branched out before v2.
>
> If not indiscreet, what kind of modifications not CURRENTly
> implemented in INN did you make?
>

Hi,

The multi-thread implementation of INND blocks on disk write, so it loses a
lot of the advantage of using multiple disk spindles. We modified INND so
multiple INNDs can be started at different ports and sharing a single copy
of history and active files... a poor man version of multi-process INND. We
also got the article assignment runs on a single separate server, so we can
have same article numbers in the server farm without every article
propagating through a single machine. And, we have a lot of spam management
code added.

On the NNRPD side, it is mostly related to user controls, such as user
authentication, data tally and connection controls.

Kachun


Julien ÉLIE

unread,
May 7, 2007, 2:13:48 PM5/7/07
to
En réponse à Kachun Lee :
>> If not indiscreet, what kind of modifications not CURRENTly
>> implemented in INN did you make?
>
> The multi-thread implementation of INND blocks on disk write
[...]

> On the NNRPD side, it is mostly related to user controls, such as user
> authentication, data tally and connection controls.

All right. Thanks for the answer.
It must have given a lot of work to do all these things!

--
Julien ÉLIE

« Quand tu auras lu ces lignes, le papyrus s'autodétruira. » (Astérix)

Julien ÉLIE

unread,
Jun 3, 2007, 11:40:22 AM6/3/07
to
En réponse à Julien ÉLIE :

> En réponse à Kachun Lee :
>> Underneath the SSL_write and SSL_read was a whole layer of creating and
>> transfering the SSL data blocks and ack'ing implemented by raw IO calls. I
>> definitely can imagine interference by outside select(...) calls.
>
> I think you're right since I no longer have problems of hangings
> when posting with Thunderbird over NNTP/SSL.

I do not know whether it can be of help, but I see:

https://bugzilla.mozilla.org/show_bug.cgi?id=247226

%%
I experienced this problem as well. It seems the SSL implementation of INN
doesn't play well with NSS. If I setup connections (on the server side) to go
through stunnel instead it all works perfect.

I stepped through the thunderbird code. The message is net all the way through
to the final '.' line. Thunderbird then waits for a response but somehow misses
the servers response (I also couldn't verify that the server actually sent a
response). The next response from the server that is read and parsed is the '500
Timeout' response that comes minutes later.
%%


Unfortunately, I do not know how to check that.
ssldump does not seem to work after the handshake (only lines like:
1 220 1.7827 (0.0108) C>SV3.1(48) application_data
1 221 1.7830 (0.0002) S>CV3.1(48) application_data
1 222 1.7935 (0.0105) C>SV3.1(64) application_data
1 223 1.7937 (0.0002) S>CV3.1(80) application_data
appear).
Perhaps a non-static RSA communication?

--
Julien ÉLIE

« O tempora ! O mores ! » (Cicéron)

Julien ÉLIE

unread,
Jun 6, 2007, 11:16:05 AM6/6/07
to
En réponse à Julien ÉLIE :
> I stepped through the thunderbird code. The message is net all the way through
> to the final '.' line. Thunderbird then waits for a response but somehow misses
> the servers response (I also couldn't verify that the server actually sent a
> response). The next response from the server that is read and parsed is the '500
> Timeout' response that comes minutes later.
>
> Unfortunately, I do not know how to check that.
> ssldump does not seem to work after the handshake (only lines like:
> 1 220 1.7827 (0.0108) C>SV3.1(48) application_data
> 1 221 1.7830 (0.0002) S>CV3.1(48) application_data
> 1 222 1.7935 (0.0105) C>SV3.1(64) application_data
> 1 223 1.7937 (0.0002) S>CV3.1(80) application_data
> appear).
> Perhaps a non-static RSA communication?

Well, that's it.
Thunderbird and INN2 communicate through a cipher 0x37 (DHE-RSA-AES256-SHA),
which means that neither ssldump nor Wireshark can decipher the communication...
Indeed, an Ephemeral Diffie Hellman is used and provides Perfect Forward Secrecy
to the established TLS connection. The server key is only used to sign the DH
parameters during the Server Key Exchange.

I see in « tls.c » that the DH parameters are hardcoded but obviously the
DH keys are... ephemeral!
So I believe the easiest thing to do is to log somewhere the buffers used
in SSL_read() and SSL_write() and then see what happens. I shall try.

--
Julien ÉLIE

« -- Que lui est-il arrivé ? Un choc moral ?
-- Oui, avec un menhir. » (Astérix)

Julien ÉLIE

unread,
Jun 6, 2007, 12:09:27 PM6/6/07
to
En réponse à Julien ÉLIE :
> I see in « tls.c » that the DH parameters are hardcoded but obviously the
> DH keys are... ephemeral!
> So I believe the easiest thing to do is to log somewhere the buffers used
> in SSL_read() and SSL_write() and then see what happens. I shall try.

All right. I got it.
Well, I do not post logs for what is good (I did lots of tests) but only
an example of what happens with STABLE and CURRENT INN2.


## SSL connection

Jun 6 17:34:35 news nnrpd[7791]: 200 news.trigofacile.com InterNetNews NNRP server INN 2.5.0 (20070421 snapshot for TrigoFACILE) ready (posting ok).
Jun 6 17:34:35 news nnrpd[7791]: MODE READER
Jun 6 17:34:35 news nnrpd[7791]: 200 news.trigofacile.com InterNetNews NNRP server INN 2.5.0 (20070421 snapshot for TrigoFACILE) ready (posting ok).
Jun 6 17:34:35 news nnrpd[7791]: AUTHINFO user xxx
Jun 6 17:34:35 news nnrpd[7791]: 381 PASS required
Jun 6 17:34:35 news nnrpd[7791]: AUTHINFO pass xxx
Jun 6 17:34:35 news nnrpd[7791]: 281 Ok

### Some stuff (read and post)
### The last post:

Jun 6 17:37:15 news nnrpd[7791]: POST
Jun 6 17:37:15 news nnrpd[7791]: 340 Ok, recommended ID <f46kbb$7jf$1...@news.trigofacile.com>
Jun 6 17:37:15 news nnrpd[7791]: Date: Wed, 06 Jun 2007 17:37:15 +0200 From: =?ISO-8859-15?Q?Julien_=C9LIE?= <iul...@nom-de-mon-site.com.invalid> Organization: Gens Iulia -- http://www.trigofacile.com/ User-Agent: Thunderbird 2.0.0.0 (Windows/20070326) MIME-Version: 1.0 Newsgroups: trigofacile.test Subject: hi Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit A. D. VIII ID. IVN. A. MMDXVI P. E. R. Le huitième jour avant les Ides de juin -- Julien ÉLIE « -- Que lui est-il arrivé ?
Jun 6 17:37:15 news nnrpd[7791]: Un choc moral ? -- Oui, avec un menhir. » (Astérix)
Jun 6 17:37:15 news nnrpd[7791]: .
Jun 6 17:37:15 news nnrpd[7791]: 240 Article posted <f46kbb$7jf$1...@news.trigofacile.com>

### I post another one (it follows the last 17:37:15 line).
### Note that the connection is closed for 7791 and opened for 8251.

Jun 6 17:42:32 news nnrpd[7791]: QUIT
Jun 6 17:42:32 news nnrpd[7791]: 205 .
Jun 6 17:42:32 news nnrpd[8251]: 200 news.trigofacile.com InterNetNews NNRP server INN 2.5.0 (20070421 snapshot for TrigoFACILE) ready (posting ok).
Jun 6 17:42:32 news nnrpd[8251]: MODE READER
Jun 6 17:42:32 news nnrpd[8251]: 200 news.trigofacile.com InterNetNews NNRP server INN 2.5.0 (20070421 snapshot for TrigoFACILE) ready (posting ok).
Jun 6 17:42:32 news nnrpd[8251]: AUTHINFO user xxx
Jun 6 17:42:32 news nnrpd[8251]: 381 PASS required
Jun 6 17:42:32 news nnrpd[8251]: AUTHINFO pass xxx
Jun 6 17:42:32 news nnrpd[8251]: 281 Ok
Jun 6 17:42:32 news nnrpd[8251]: POST
Jun 6 17:42:32 news nnrpd[8251]: 340 Ok, recommended ID <f46kl8$81r$1...@news.trigofacile.com>
Jun 6 17:42:32 news nnrpd[8251]: Date: Wed, 06 Jun 2007 17:42:32 +0200 From: =?ISO-8859-15?Q?Julien_=C9LIE?= <iul...@nom-de-mon-site.com.invalid> Organization: Gens Iulia -- http://www.trigofacile.com/ User-Agent: Thunderbird 2.0.0.0 (Windows/20070326) MIME-Version: 1.0 Newsgroups: trigofacile.test Subject: testee Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit A. D. VIII ID. IVN. A. MMDXVI P. E. R. Le huitième jour avant les Ides de juin -- Julien ÉLIE « -- Que lui est-il arriv

### Thunderbird hangs (one minute).
### A dialog box appears, telling me the article could not be posted. I click « OK ».

Jun 6 17:43:33 news nnrpd[8251]: é ? Un choc moral ? -- Oui, avec un menhir. » (Astérix) . arrivivJulien ÉLIE « -- Que lui est-il arrivivivÉLIE « -- Que lui est-il arriv -- Julien ÉLIE « -- Que lui est-il arriv lui est-il arrivE « -- Que lui est-il arrivue lui est-il arriv -- Julien ÉLIE « -- Que lui est-il arrivin -- Julien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arriv
Jun 6 17:43:33 news nnrpd[8251]: 240 Article posted <f46kl8$81r$1...@news.trigofacile.com>
Jun 6 17:43:33 news nnrpd[8251]: .
Jun 6 17:43:33 news nnrpd[8251]: 205 .

### And, obviously, the article was correctly posted.

Well, I do not know how to interpret the last four lines...

The first is obtained thanks to line.c:


static ssize_t
line_doread(void *p, size_t len)
{
ssize_t n;

do {
#ifdef HAVE_SSL
if (tls_conn) {
int err;
do {
n = SSL_read(tls_conn, p, len);
+ syslog(L_ERROR, "%s", p);


I wonder what happened to the line (is the part after the « . » really sent
by Thunderbird? maybe it is an artefact from my writing the buffer p).

And I do not know why « . » in the third line. And also why the
answer « 205 . » to such a command.

If you have any idea regarding that problem (a INN one or a Thunderbird one?
why does it occur?), it would be great.

Regards,

Julien ÉLIE

unread,
Jun 6, 2007, 12:22:39 PM6/6/07
to
En réponse à Julien ÉLIE :
> I wonder what happened to the line (is the part after the « . » really sent
> by Thunderbird? maybe it is an artefact from my writing the buffer p).

Well, since my last post was in the same case, here are new logs:

### Normal posting until:

Jun 6 18:09:27 news nnrpd[9842]: lui est-il arriv ### Thunderbird hangs (one minute). ### A dialog box appears, telling me the article could not be posted. I click « OK ». Jun 6 17:43:33 news nnrpd[8251]: é ? Un choc moral ? -- Oui, avec un menhir. » (Astérix) . arrivivJulien ÉLIE « -- Que lui est-il arrivivivÉLIE « -- Que lui est-il arriv -- Julien ÉLIE « -- Que lui est-il arriv lui est-il arrivE « -- Que lui est-il arrivue lui est-il arriv -- Julien ÉLIE « -- Que lui est-il arriv

### Hang during a minute, and then the end of my message arrives:

Jun 6 18:10:28 news nnrpd[9842]: in -- Julien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arrivulien ÉLIE « -- Que lui est-il arriv Jun 6 17:43:33 news nnrpd[8251]: 240 Article posted <f46kl8$81r$1...@news.trigofacile.com> Jun 6 17:43:33 news nnrpd[8251]: . Jun 6 17:43:33 news nnrpd[8251]: 205 . ### And, obviously, the article was correctly posted. Well, I do not know how to interpret the last four lines... The first is obtained thanks to line.c: static ssize_t line_doread(void *p, size_t len) { ssize_t n; do { #ifdef HAVE_SSL if (tls_conn) { int err; do { n =
Jun 6 18:10:28 news nnrpd[9842]: SSL_read(tls_conn, p, len); + syslog(L_ERROR, "%s", p); I wonder what happened to the line (is the part after the « . » really sent by Thunderbird? maybe it is an artefact from my writing the buffer p). And I do not know why « . » in the third line. And also why the answer « 205 . » to such a command. If you have any idea regarding that problem (a INN one or a Thunderbird one? why does it occur?), it would be great. Regards, -- Julien ÉLIE « -- Que lui est-il arrivé ? Un choc moral ? -- Oui, avec un menhir. » (Astérix) . kl8$81r$1...@news.trigofacile.com> Jun 6 17:43:33 news nnrpd[8251]: . Jun 6 17:43:33 news nnrpd[8251]: 205 . ### And, obviously, the article was correctly posted. Well, I do not know how to interpret the last four lines... The first is obtained thanks to line.c: static ssize_t line_doread(void *p, size_t len) { ssize_t n; do { #ifdef HAVE_SSL if

(tls_conn) { int err; do { n =

### I am still not sure whether the end of the previous line is an artefact (after « . »).

Jun 6 18:10:28 news nnrpd[9842]: 240 Article posted <f46m7n$9ji$1...@news.trigofacile.com>
Jun 6 18:10:28 news nnrpd[9842]: .
Jun 6 18:10:28 news nnrpd[9842]: 205 .


Still that strange « . » answer.


By the way, couldn't that issue be related to « line.c »:

** This code implements a infinitely (well size_t) long single line
** read routine, to protect against eating all available memory it
** actually starts discarding characters if you try to send more than
** the maximum article size in a single line.


I also wish to recall that no problem occurs with:

+ #ifdef HAVE_SSL


/* wait for activity on stdin, updating timer stats as we
* go */
+ if (tls_conn == NULL) {
+ #endif

do {
struct timeval t;

FD_ZERO(&rmask);
FD_SET(STDIN_FILENO, &rmask);
t.tv_sec = timeout;
t.tv_usec = 0;
/*TMRstart(TMR_IDLE);*/
i = select(STDIN_FILENO + 1, &rmask, NULL, NULL, &t);
/*TMRstop(TMR_IDLE);*/
if (i == -1 && errno != EINTR) {

syslog(LOG_WARNING, "%s can't select", ClientHost);
return RTtimeout;
}
} while (i == -1);

/* if stdin didn't select, we must have timed out */
if (i == 0 || !FD_ISSET(STDIN_FILENO, &rmask))

return RTtimeout;
+ #ifdef HAVE_SSL
+ }
+ #endif

I hope someone will understand what is happening.

--
Julien ÉLIE

« Savoir c'est pouvoir. » (Francis Bacon)

Reply all
Reply to author
Forward
0 new messages