need help - after months of working perfectly email logins stopped working

263 views
Skip to first unread message

John Kullmann Scheinberg

unread,
Aug 15, 2016, 9:57:14 PM8/15/16
to Sandstorm Development
I've had a sandstorm system up on AWS since February and it's been working flawlessly. We've been using Email and google and github sign in's without problems.

Suddenly a few days ago email Sign In's don't work, they hang for awhile and then return "Internal Server Error" in red.

I also see this in the log file which seems to be related.

Exception while invoking method 'createAndEmailTokenForUser' Error: Connection timeout
    at Object.Future.wait (/programs/server/node_modules/fibers/future.js:449:15)
    at smtpSend (imports/server/email.js:66:44)
    at rawSend (imports/server/email.js:75:5)
    at send (imports/server/email.js:164:3)
    at sendTokenEmail (server/accounts/email-token/token-server.js:207:3)
    at _createAndEmailTokenForUser (server/accounts/email-token/token-server.js:259:3)
    at [object Object].createAndEmailTokenForUser (server/accounts/email-token/token-server.js:278:18)
    at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1711:12)
    at packages/ddp-server/livedata_server.js:711:19
    at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
    - - - - -
    at SMTPConnection._formatError (/programs/server/npm/node_modules/smtp-connection/lib/smtp-connection.js:493:15)
    at SMTPConnection._onError (/programs/server/npm/node_modules/smtp-connection/lib/smtp-connection.js:479:16)
    at SMTPConnection.<anonymous> (/programs/server/npm/node_modules/smtp-connection/lib/smtp-connection.js:224:14)
    at Timer.listOnTimeout (timers.js:92:15)

however, I have not changed anything (in fact I was out of the country for the two weeks during the time this started failing). I'm using a free mailgun account and have not modified anything there either

I've restarted sandstorm and then the entire server - neither corrected the problem.

How can I go about figuring out what has happened. 

Jason D.

unread,
Aug 15, 2016, 10:11:06 PM8/15/16
to sandst...@googlegroups.com
I had a similar problem yesterday after the update to Nodemailer. In my case, which may be similar to yours, I was using my custom domain name as the Sandstorm SMTP host instead of the more general host name that my email provider uses. My domain name was not listed on my email provider's certificate, so Sandstorm (via Nodemailer) gave me that error.

In your case, using Mailgun, are you using your own domain name or the subdomain provided by Mailgun? If you're using your own domain, try switching to Mailgun's subdomain as the Sandstorm email host. That may solve your problem for now until the Sandstorm folks get a chance to work on this more on their end. If you're already using that subdomain, then I'm not sure what to do next. I'm not totally sure that changing to that subdomain will work, but that's all I can think of as I don't use Mailgun.

Use this command to sign in and change the host if you think this is the problem:

sudo sandstorm admin-token

Jason D.
--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

PLEASE NOTE: EMAIL IS NOT A PRIVATE FORM OF COMMUNICATION.
You are accepting responsibility for any liability related to personal information given by email.

John Kullmann Scheinberg

unread,
Aug 15, 2016, 10:13:04 PM8/15/16
to Sandstorm Development
Further info - trying the "test" in the email configuration panel just hangs after I put in my email address to send to.

Kenton Varda

unread,
Aug 15, 2016, 10:13:58 PM8/15/16
to Jason D., Sandstorm-dev
Hi all,

Because multiple people have experienced this, we are pushing an emergency release that relaxes the TLS certificate checking back to the previous version's behavior. We will then work on making this configurable in the future. The emergency release should be ready in a few minutes -- I'll email again to confirm.

-Kenton

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--

PLEASE NOTE: EMAIL IS NOT A PRIVATE FORM OF COMMUNICATION.
You are accepting responsibility for any liability related to personal information given by email.

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-dev+unsubscribe@googlegroups.com.

Kenton Varda

unread,
Aug 15, 2016, 10:22:52 PM8/15/16
to Jason D., Sandstorm-dev
Hi all,

I've pushed an update (0.177). You should get it automatically within 24 hours, or you can ssh into your server and run:

    sudo sandstorm update

to get the update immediately.

John, your error message reports a timeout rather than a TLS problem, so I'm not 100% sure if this update fixes it. Can you please try it and let me know ASAP?

-Kenton

Kenton Varda

unread,
Aug 15, 2016, 10:23:40 PM8/15/16
to John Kullmann Scheinberg, Sandstorm-dev
(directly cc'ing John)

JK Scheinberg

unread,
Aug 15, 2016, 10:30:54 PM8/15/16
to Kenton Varda, John Kullmann Scheinberg, Sandstorm-dev
I did ‘’sudo sandstorm update’’ and .177 installed but trying ‘test’ from email configuration still doesn’t ever finish. Actually it did finish with this error displayed

Error while trying to send test email: {"code":"ETIMEDOUT","command":"CONN"} [other-email-sending-error]


i also tested my mailgun setup by using curl to send via the API and that worked, so my mailgun setup is still good.


Anything else I can provide to try to help figure out what’s happening?
>> To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
>
> PLEASE NOTE: EMAIL IS NOT A PRIVATE FORM OF COMMUNICATION.
> You are accepting responsibility for any liability related to personal information given by email.
>
> --
> You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

John Kullmann Scheinberg

unread,
Aug 15, 2016, 10:36:31 PM8/15/16
to Sandstorm Development
Hi
I'm already using the mailgun provided subdomain and it's been working flawlessy since February. Something must have broken during a recent sandstorm update to cause mail to stop working.
Thanks

Drew Fisher

unread,
Aug 15, 2016, 10:38:24 PM8/15/16
to JK Scheinberg, Kenton Varda, John Kullmann Scheinberg, Sandstorm-dev
On Mon, Aug 15, 2016 at 7:30 PM, JK Scheinberg <j...@panix.com> wrote:
I did ‘’sudo sandstorm update’’ and .177 installed but trying ‘test’ from email configuration still doesn’t ever finish. Actually it did finish with this error displayed

Error while trying to send test email: {"code":"ETIMEDOUT","command":"CONN"} [other-email-sending-error]


i also tested my mailgun setup by using curl to send via the API and that worked, so my mailgun setup is still good.


Anything else I can provide to try to help figure out what’s happening?

I'm sorry my change seems to have broken your email setup!  I'll try to get to the bottom of this with you ASAP.

Is the SMTP server you're trying to use "smtp.mailgun.org", or something else?  Have you tried using the same set of SMTP credentials over SMTP (rather than the API)?

I just tested setting up a new mailgun free tier account to attempt to reproduce your issue, and mail seems to be delivering over SMTP fine for me.

Feel free to reply off-list if you'd prefer to debug more privately.

Cheers,
Drew

Drew Fisher

unread,
Aug 15, 2016, 11:24:11 PM8/15/16
to JK Scheinberg, Kenton Varda, John Kullmann Scheinberg, Sandstorm-dev
John and I debugged off-list and discovered that mailgun doesn't appear to listen for SMTP traffic on sandbox subdomains like "sandboxf3b61beca25b4951ad5ea991a5dbe234.mailgun.org" (which seemingly *did* work in the past, but definitely doesn't work now).  To be precise, sandboxf3b61beca25b4951ad5ea991a5dbe234.mailgun.org (example IP 173.203.37.113) is not listening on port 25, whereas smtp.mailgun.org (example IP 104.130.177.23) is listening and speaks SMTP if you connect.

The solution is to set the "SMTP host" field on the email settings page to "smtp.mailgun.org" rather than of one of mailgun's sandbox domains.

Big thanks to John for debugging things with me.  Hopefully if someone else has the same issue, they'll see this thread and know what to do!

Cheers,
Drew
Reply all
Reply to author
Forward
0 new messages