How do you test emails?

281 views
Skip to first unread message

deitch

unread,
Nov 30, 2011, 12:32:44 PM11/30/11
to nodejs
Interested in how people test email systems, e.g. new user
registration or password reset. I see two scenarios, one easy to
figure out, one harder:

1) Automated testing: This is easy, I have the smtp module installed,
before I start automated tests, I start an smtp server, and then use
node_mailer to send mails to that server. I can easily retrieve any
mails using the automated testing harness, no matter who the email
registered was.

2) Manual testing: This is harder. I want to be able to test
registration against the minimal test database with *any* email, but
make sure that all of the emails come to me, no matter which email
address is used. I could have it go against a local server which then
forwards all to my real email, or even have them all go to my email,
but then I lose the ability to have different users (and hence
testers) run against the same codebase. Unless I have some way to
configure run-time what the email target will be, perhaps an
environment variable.

Ideas?

Matt

unread,
Nov 30, 2011, 12:45:16 PM11/30/11
to nod...@googlegroups.com
You could use Haraka as your mail server and write plugins to redirect emails wherever you require.

The main Haraka tutorial actually covers re-writing email addresses: http://haraka.github.com/manual/Tutorial.html


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Nuno Job

unread,
Nov 30, 2011, 12:49:54 PM11/30/11
to nod...@googlegroups.com
If we had something like Nock (http://github.com/pgte/nock) for SMTP that would help mocking your requests. I don't think that exists though.

If you use a cloud provider that allows sendmail via HTTP than nock is fine?

The way we are doing it however is neither. We have a pipeline system integrated with airbreak. One of the steps of the pipeline is sending the email. If that fails we get an email. Does't really cover testing for newly introduced bugs but that's all we have currently so I'm super interested in your findings.

Nuno

deitch

unread,
Nov 30, 2011, 1:39:35 PM11/30/11
to nodejs
I didn't know about Haraka when I did this, so I just used the smtp
node module, and built a lightweight server around it, at least for my
automated testing.

Here is what it does:
1) Create a mail server that can listen on a configurable port.
2) add a bind and unbind call. The bind call allows you to bind a
handler for a particular recipient. So if I bind "f...@bar.com" to a
handler, then every email received by f...@bar.com will call the
handler with the following signature handler(email,msgid,emailText).
Handlers are called exactly once for each message, whether you bind
before or after the message has been received. This makes it real easy
for a testing program to run a lightweight in-process mail handler
that can receive mail sent by a program, and use its own handlers to
process them.

If someone wants me to publish the code, here or as a github, just
tell me.

The issue is that for automated testing, having it retrieve the
messages is in-process, real easy.

For manual testing, even this server doesn't help, because the problem
isn't the server, but rather having the messages end up in a
convenient fashion for the engineer actually doing the manual testing.
If I launch a node app in dev mode, then register a new account with
email f...@bar.com, I want the emails to come to me, either visually
somehow or to some address m...@myco.com. But using the exact same code
and test base, if some other engineer runs it, he wants the emails to
come to him, either visually or h...@myco.com.

In other words, I either need some way to capture those mails and
visualize them to the current user, whoever that user is, or perhaps a
way to tell the currently launched process, "send them here for now".

I could do something like, have a special command-line that launches a
mail server (perhaps the one above), with a config to forward to an
address pulled from an environment variable, and then launches the
app. Seems a bit messy, but?

deitch

unread,
Nov 30, 2011, 1:47:32 PM11/30/11
to nodejs
Nuno,

Nock is cool. I had a problem like that a few years back on the UI
side, so I wrote an embedded Web server for the browser (ewsjs), works
almost identically to Nock for node. It is on my github page
http://github.com/deitch/ewsjs

But your thought here is rather than actually do the email, mock the
email? That is a different approach. Still leaves the open question of
how to tell the engineer, "your mail went (or was mocked to have gone)
through, and here is the content." The natural process seems to be
either console, in which case we solve a lot of the config issues, or
to have an environment var that contains the engineer's email and will
send it through.

Actually, using the smtp server I created (see above), it would be
real easy to just have a handler that dumps it to the console, or
forwards to a particular address, two different handlers, command-line
or env var configurable.

The usage would be:
1) In automated testing, start with a special command line, that
launches the smtp server (and mocks, and db, and whatever else your
auto testing environment requires), sets up the NODE_ENV to "test",
and then runs your app, either in-process or out-of-process, just like
I do now.
2) In manual testing, start with a special command line, just like
automated testing, but reads a config or env var as to what to do -
dump emails to console, send to a particular address, whatever.
3) In UAT, start with a regular command line, so no special launch,
just the NODE_ENV set to "uat"
4) In Prod, start with a normal command line, which runs the app

Might be worth a github project to productize both the lightweight
smtp server harness, and the test launcher environment?

Matt

unread,
Nov 30, 2011, 3:28:08 PM11/30/11
to nod...@googlegroups.com
How about using haraka-couchdb and then just verify they went into couchdb?

deitch

unread,
Nov 30, 2011, 3:59:01 PM11/30/11
to nodejs
So kick off the mail server, have it dump the mails into couch, and
then check in couch? Sort of like the console dump, but cleaner. Still
requires running a couch instance to do lightweight testing, but
avoids the config problem.

Wayne Douglas

unread,
Nov 30, 2011, 4:03:54 PM11/30/11
to nod...@googlegroups.com

deitch

unread,
Nov 30, 2011, 4:11:19 PM11/30/11
to nodejs
Nice idea... but I work on Mac.

Matt

unread,
Nov 30, 2011, 5:55:47 PM11/30/11
to nod...@googlegroups.com
Well it depends where you want to look - some testers aren't happy with the command line.

If your QA team is happy with the command line you can always just run Haraka as an smtp-sink using a null queue plugin. Basically: hook_queue = function (next, conn) { next(OK, "Queued") }

And then run Haraka with config/loglevel = LOGDATA and you see everything in the console.

substack

unread,
Nov 30, 2011, 8:54:10 PM11/30/11
to nodejs
On Nov 30, 9:32 am, deitch <a...@deitcher.net> wrote:
> Interested in how people test email systems, e.g. new user
> registration or password reset. I see two scenarios, one easy to
> figure out, one harder:

It just so happens I just built some modules for doing exactly this
over the past 2 days.

https://github.com/substack/node-smtp-protocol lets you shim out a
test server really easily.
Check out the tests in pony:
https://github.com/substack/node-pony/tree/master/test

deitch

unread,
Dec 1, 2011, 3:48:34 AM12/1/11
to nodejs
Not a bad idea, I will see if it is any easier than just adding
command-line logging to my current module.

deitch

unread,
Dec 1, 2011, 3:52:13 AM12/1/11
to nodejs
Similar idea, and works very similar to require('smtp').

I think, in the end, that my issue is how to have a common config that
allows everyone to run it, but have the resultant emails be sent to
any address, but come only to the person currently running the test.
That can be configured either by forwarding to a given email, which is
set via an env var, or just spitting it out to a command-line.

On Dec 1, 3:54 am, substack <subst...@gmail.com> wrote:
> On Nov 30, 9:32 am, deitch <a...@deitcher.net> wrote:
>
> > Interested in how people test email systems, e.g. new user
> > registration or password reset. I see two scenarios, one easy to
> > figure out, one harder:
>
> It just so happens I just built some modules for doing exactly this
> over the past 2 days.
>

> https://github.com/substack/node-smtp-protocollets you shim out a

deitch

unread,
Dec 1, 2011, 5:55:34 AM12/1/11
to nodejs
I published smtp-tester to github with README and to npm.

It is nowhere near as complete as Haraka, but doesn't mean to be, it
is just a way to receive emails to smtp and call in-process handlers.
Its entire purpose is to easily test email receipt from an app. Done.

npm install smtp-tester
http://github.com/deitch/smtp-tester

On Dec 1, 10:52 am, deitch <a...@deitcher.net> wrote:
> Similar idea, and works very similar to require('smtp').
>
> I think, in the end, that my issue is how to have a common config that
> allows everyone to run it, but have the resultant emails be sent to
> any address, but come only to the person currently running the test.
> That can be configured either by forwarding to a given email, which is
> set via an env var, or just spitting it out to a command-line.
>
> On Dec 1, 3:54 am, substack <subst...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On Nov 30, 9:32 am, deitch <a...@deitcher.net> wrote:
>
> > > Interested in how people test email systems, e.g. new user
> > > registration or password reset. I see two scenarios, one easy to
> > > figure out, one harder:
>
> > It just so happens I just built some modules for doing exactly this
> > over the past 2 days.
>

> >https://github.com/substack/node-smtp-protocolletsyou shim out a

Nuno Job

unread,
Dec 1, 2011, 9:45:17 AM12/1/11
to nod...@googlegroups.com
Wow that's super fast.

Kudos Avi!

Nuno

On Thu, Dec 1, 2011 at 10:55 AM, deitch <a...@deitcher.net> wrote:
mails to smtp a

deitch

unread,
Dec 1, 2011, 1:41:22 PM12/1/11
to nodejs
Thanks!

Now I need to add some standardized handlers...

deitch

unread,
Dec 1, 2011, 3:34:27 PM12/1/11
to nodejs
v0.2.0 is released. Added:

- support for catch-all handlers
- documentation for catch-all handlers
- test cases for catch-all handlers

Already in github and npm

deitch

unread,
Dec 1, 2011, 4:43:15 PM12/1/11
to nodejs
And the next one is released. Added:

- support for pre-shipped modules. Just run
"mailServer.module(name)" (see the README)
- documentation for pre-shipped modules
- test cases for pre-shipped modules
- logAll pre-built module

Already in github and npm

Reply all
Reply to author
Forward
0 new messages