One more project using ispdb

[Andreas Nilsson]

Hi!
As some of you may know, the Evolution project recently started to make
use of ISPDB [1] and the other day I learned that Kmail, the primary
e-mailing app for KDE project is going to make use of it as well through
the Akonadi framework [2].
Just wanted to let you know, as someone mentioned it could affect
Thunderbirds server statistics data.

1. http://git.gnome.org/browse/api-web/tree/evolution/autoconfig
2.
http://www.omat.nl/2010/05/15/akonadi-meeting-day-2-hacking-accountwizard-continued/

- Andreas
_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

[David Ascher]

On 5/17/10 3:06 AM, Andreas Nilsson wrote:
> Hi!
> As some of you may know, the Evolution project recently started to
> make use of ISPDB [1] and the other day I learned that Kmail, the
> primary e-mailing app for KDE project is going to make use of it as
> well through the Akonadi framework [2].
> Just wanted to let you know, as someone mentioned it could affect
> Thunderbirds server statistics data.

Gee, it'd be nice if these projects talked to us about it.

--da

[Ben Bucksch]

On 17.05.2010 17:25, David Ascher wrote:
> On 5/17/10 3:06 AM, Andreas Nilsson wrote:
>> Hi!
>> As some of you may know, the Evolution project recently started to
>> make use of ISPDB [1] and the other day I learned that Kmail, the
>> primary e-mailing app for KDE project is going to make use of it as
>> well through the Akonadi framework [2].
>> Just wanted to let you know, as someone mentioned it could affect
>> Thunderbirds server statistics data.
>
> Gee, it'd be nice if these projects talked to us about it.

Why? They blogged about it! </irony>
Thanks, Andreas, for the note!

I'm in contact with the guy.

Ben

[Andreas Nilsson]

On 05/17/2010 05:25 PM, David Ascher wrote:
> On 5/17/10 3:06 AM, Andreas Nilsson wrote:
>> Hi!
>> As some of you may know, the Evolution project recently started to
>> make use of ISPDB [1] and the other day I learned that Kmail, the
>> primary e-mailing app for KDE project is going to make use of it as
>> well through the Akonadi framework [2].
>> Just wanted to let you know, as someone mentioned it could affect
>> Thunderbirds server statistics data.
>
> Gee, it'd be nice if these projects talked to us about it.

Blake have already been in contact with Michael Meeks that did the
implementation in Evolution (I can forward the e-mails to you if you
want to), and I referred the Kmail guy to the ispdb list if he had any
questions.
Is there anything specific we need to talk to them about regarding ispdb?
If so, I can try to get in e-mail contact with the developers involved.
- Andreas

[Mark Banner]

On 17/05/2010 16:25, David Ascher wrote:
> On 5/17/10 3:06 AM, Andreas Nilsson wrote:
>> Hi!
>> As some of you may know, the Evolution project recently started to
>> make use of ISPDB [1] and the other day I learned that Kmail, the
>> primary e-mailing app for KDE project is going to make use of it as
>> well through the Akonadi framework [2].
>> Just wanted to let you know, as someone mentioned it could affect
>> Thunderbirds server statistics data.
>
> Gee, it'd be nice if these projects talked to us about it.

Maybe just add a note on the wiki page asking them to let us know....

Mark.

[Blake Winton]

On 10-05-17 11:41 AM, Andreas Nilsson wrote:
> On 05/17/2010 05:25 PM, David Ascher wrote:
>> On 5/17/10 3:06 AM, Andreas Nilsson wrote:
>>> Hi!
>>> As some of you may know, the Evolution project recently started to
>>> make use of ISPDB [1] and the other day I learned that Kmail, the
>>> primary e-mailing app for KDE project is going to make use of it as
>>> well through the Akonadi framework [2].
>>> Just wanted to let you know, as someone mentioned it could affect
>>> Thunderbirds server statistics data.
>>
>> Gee, it'd be nice if these projects talked to us about it.
> Blake have already been in contact with Michael Meeks that did the
> implementation in Evolution (I can forward the e-mails to you if you
> want to), and I referred the Kmail guy to the ispdb list if he had any
> questions.
> Is there anything specific we need to talk to them about regarding ispdb?
> If so, I can try to get in e-mail contact with the developers involved.

One big thing is that they should mirror our svn version [0], and use
the script [1] to generate the configs, instead of running against the
ispdb directly. (For a number of reasons, including the
non-reviewed-ness of the ispdb configs.)

Later,
Blake.
--
[0]
http://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/
[1]
http://svn.mozilla.org/mozillamessaging.com/sites/ispdb.mozillamessaging.com/trunk/tools/convert.py

[David Ascher]

On 2010-05-17, at 8:53 AM, Blake Winton <bwi...@mozillamessaging.com>
wrote:

> On 10-05-17 11:41 AM, Andreas Nilsson wrote:
>> On 05/17/2010 05:25 PM, David Ascher wrote:
>>> On 5/17/10 3:06 AM, Andreas Nilsson wrote:
>>>> Hi!
>>>> As some of you may know, the Evolution project recently started to
>>>> make use of ISPDB [1] and the other day I learned that Kmail, the
>>>> primary e-mailing app for KDE project is going to make use of it as
>>>> well through the Akonadi framework [2].
>>>> Just wanted to let you know, as someone mentioned it could affect
>>>> Thunderbirds server statistics data.
>>>
>>> Gee, it'd be nice if these projects talked to us about it.
>> Blake have already been in contact with Michael Meeks that did the
>> implementation in Evolution (I can forward the e-mails to you if you
>> want to), and I referred the Kmail guy to the ispdb list if he had
>> any
>> questions.
>> Is there anything specific we need to talk to them about regarding
>> ispdb?
>> If so, I can try to get in e-mail contact with the developers
>> involved.
>
> One big thing is that they should mirror our svn version [0], and
> use the script [1] to generate the configs, instead of running
> against the ispdb directly. (For a number of reasons, including the
> non-reviewed-ness of the ispdb configs.)

They certainly shouldn't talk to the django app directly, but I'm
wondering what te downside of talking to the front-end http server is,
if they use a user-agent string to identify themselves? I'm fine with
them "freeloading" off of what I see as a public benefit service.

Clearly the above assumes some mechanism for us communicating changes
to the API, and we don't want to have to support third parties beyond
our own usage of a particular endpoint, but that seems solvable.

--da

[Blake Winton]

On 10-05-17 12:12 PM, David Ascher wrote:
>>> Is there anything specific we need to talk to them about regarding
>>> ispdb?

>> One big thing is that they should mirror our svn version [0], and use
>> the script [1] to generate the configs, instead of running against the
>> ispdb directly. (For a number of reasons, including the
>> non-reviewed-ness of the ispdb configs.)
> They certainly shouldn't talk to the django app directly, but I'm
> wondering what te downside of talking to the front-end http server is,
> if they use a user-agent string to identify themselves? I'm fine with
> them "freeloading" off of what I see as a public benefit service.

Well, the Evolution developers had a problem with the speed and
connectivity of the website we have it running on. (So they put it on
their distributed cluster, with machines all over the world.)

I'm as happy as anyone else to have them point at our configs, but that
does mean that we might want to test any changes with the various
clients, at least to let them know if they might break. (Or possibly
get them some bugzilla accounts, so that they can help verify configs.)

> Clearly the above assumes some mechanism for us communicating changes to
> the API, and we don't want to have to support third parties beyond our
> own usage of a particular endpoint, but that seems solvable.

is...@googlegroups.com seems like a reasonable way for us to communicate
changes.

Later,
Blake.

[David Ascher]

On 5/17/10 10:14 AM, Blake Winton wrote:
> Well, the Evolution developers had a problem with the speed and
> connectivity of the website we have it running on. (So they put it on
> their distributed cluster, with machines all over the world.)

Interesting. Are they cloning the MX lookups etc as well?

> I'm as happy as anyone else to have them point at our configs, but
> that does mean that we might want to test any changes with the various
> clients, at least to let them know if they might break. (Or possibly
> get them some bugzilla accounts, so that they can help verify configs.)

I don't think we should take on the testing burden.

> is...@googlegroups.com seems like a reasonable way for us to
> communicate changes.

agreed.

[Blake Winton]

On 10-05-17 1:25 PM, David Ascher wrote:
> On 5/17/10 10:14 AM, Blake Winton wrote:
>> Well, the Evolution developers had a problem with the speed and
>> connectivity of the website we have it running on. (So they put it on
>> their distributed cluster, with machines all over the world.)
> Interesting. Are they cloning the MX lookups etc as well?

Probably not, but we can ask them once they're on
is...@googlegroups.com. :)

>> I'm as happy as anyone else to have them point at our configs, but
>> that does mean that we might want to test any changes with the various
>> clients, at least to let them know if they might break. (Or possibly
>> get them some bugzilla accounts, so that they can help verify configs.)
> I don't think we should take on the testing burden.

That's fair enough, but then they'll be depending on us, and I don't
know if they want to take the chance that we won't introduce a breaking
change. (I don't think we'll need to, but I also don't believe that
we'll guarantee that we won't by accident.)

Thanks,
Blake.

[Ben Bucksch]

On 17.05.2010 18:12, David Ascher wrote:
> They certainly shouldn't talk to the django app directly

Yes, I pointed that out clearly to him, on the blog and in private.

> I'm fine with them "freeloading" off of what I see as a public benefit
> service.

Great, good to know. Thanks.

> Clearly the above assumes some mechanism for us communicating changes
> to the API, and we don't want to have to support third parties beyond
> our own usage of a particular endpoint, but that seems solvable.

Agreed.

I see it as Internet protocol. I tried to make the format as stable as
possible, and with other clients (i.e. this exact case) in mind, so it
should be OK.

If we need to change the format again, or add calendar, webdav etc
(which Kontact will have good use for), I'll communicate it to Tom
Albers and Michael of Evolution.

> Blake wrote:
>> Well, the Evolution developers had a problem with the speed and
>> connectivity of the website we have it running on. (So they put it
>> on their distributed cluster, with machines all over the world.)

Michael reported: "~20+ seconds in some cases (from our China QA) to
fetch one."

That may be something we should look into, too. Or maybe The Great
Firewall just didn't like SSL? :)

> Are [the Evolution guys] cloning the MX lookups etc as well?

There are no MX lookups on our ISPDB server.
Hopefully, Evolution has less of a problem to use res_query() to make
DNS lookups in the clients :).

Ben

[Philippe M. Chiasson]

On 10-05-17 14:02 , Ben Bucksch wrote:
> On 17.05.2010 18:12, David Ascher wrote:
>> They certainly shouldn't talk to the django app directly

> [...]

>> Blake wrote:
>>> Well, the Evolution developers had a problem with the speed and
>>> connectivity of the website we have it running on. (So they put it
>>> on their distributed cluster, with machines all over the world.)
> Michael reported: "~20+ seconds in some cases (from our China QA) to
> fetch one."

No clue what that could be related to, but we certainly do not serve
static files in 20+ seconds. I've got pretty complete monitoring of our
web cluster, and I can say for certain that's not where such a dramatic
slowdown was.

I'd like to hear more about their findings however.

> That may be something we should look into, too. Or maybe The Great
> Firewall just didn't like SSL? :)

The autoconfig stuff is also available over http://, so they could try
that and at least confirm (or not) if ssl had something to do with it.

--
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/

[Ben Bucksch]

On 18.05.2010 03:29, Philippe M. Chiasson wrote:
> On 10-05-17 14:02 , Ben Bucksch wrote:
>> Michael reported: "~20+ seconds in some cases (from our China QA) to
>> fetch one."
> No clue what that could be related to, but we certainly do not serve
> static files in 20+ seconds. I've got pretty complete monitoring of our
> web cluster, and I can say for certain that's not where such a dramatic
> slowdown was.

Yeah, in my tests, a fetch takes about 1 second or a tad more (from
Europe mainland).
He meant the problem was specifically from Asia or China. Did you test
fetches from there? (I didn't, so I cannot substantiate.)

>> That may be something we should look into, too. Or maybe The Great
>> Firewall just didn't like SSL? :)
> The autoconfig stuff is also available over http://, so they could try
> that and at least confirm (or not) if ssl had something to do with it.

OK