Sendgrid + GC + WHM with Exim

[Matt Stevenson]

Hi

On a GC instance, with WHM and CPANEL installed, our next task using SendGrid send / receive all email.
This is a test domain which is registered with DNS, and development server, I have so far edited the exim advanced configurator with the instruction here.
https://sendgrid.com/docs/Integrate/Mail_Servers/exim.html

When I login to webmail to send test mail it fails.

Also I have a Google network tcp rule for port 587 which is what Send Grid recommends.

I did see a post about SPF records, I have not added these yet,
https://forums.cpanel.net/threads/external-mail-server-and-spf.207622/

I kind of feel I'm attempting that must of been done before, this is really important for us to prove the concept of email delivery through Sendgrid for all accounts.

Please can anyone give me advise on setup and troubleshooting.

Many thanks,

Matt

[Scott Van Woudenberg]

Hi Matt,

TL;DR: switch to port 2525 and you should be golden (you'll find this port documented in our SendGrid instructions, and SendGrid's docs).

GCE blocks outbound traffic on ports 25, 465, and 587; project-level firewalls don't affect this. We do this for a number of reasons, but mainly to avoid becoming the cloud platform of choice for all of the world's email spammers. :)

Regards,

-ScottVW

---

Scott Van Woudenberg

Product Manager

Google Compute Engine

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/4a18bb14-d3ca-4447-bcba-2a91ab4a9446%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Stevenson]

Hi Scott

Thanks I changed the port to 2525 in Exim Advanced conf PRE ROUTERS.
Do I have to do anything else in Google Cloud Network ?

Anyway I sent another test mail which left the inbox, did not bounce but was not delivered either, so slight progress.

The exim_mainlog reads.

R=send_via_sendgrid T=sendgrid_smtp defer (110): Connection timed out

Any more ideas please.

Kind Regards

Matt

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.

[Scott Van Woudenberg]

Hi Matt,

Hrm. Did you also add a firewall rule for port 2525?

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/e267c07f-cbb2-46f5-9fa3-00cc71684370%40googlegroups.com.

[Matt Stevenson]

Hi Scott 

I did add a Firewall rule and tested telnet smtp.sendgrid.net 2525. 

Emails have started to be sent, for some reason I cannot receive email on the domain which can send, any ideas?

Regards

Matt

[Scott Van Woudenberg]

Hi Matt,

Glad to hear you're able to send now. What protocol/port does the receive path use? If it's different, do you also have a firewall rule that allows that traffic?

Hopefully others will chime in with other ideas for troubleshooting...

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/a7d9aa6e-4f69-4c72-8dbf-35c67111c9a7%40googlegroups.com.

[Matt Stevenson]

Hi Scott 

I think its probably going to smtp and will that default to 25 for inbound in which case I'm not sure if storing email on a Google cloud server is going to be possible in that case or not ?

It maybe that Sendgrid said something about the reply to address perhaps that will help. I looked at Mandril which sets up inbound routes but to a URL so not sure thats going to help. 

So basically I can send email but will the server be able receive email or can the in bound port be different from 25 ?

Regards

Matt

[Matt Stevenson]

Hi Scott

Also to say I added the port 2525 to the Exim conf file. 

daemon_smtp_ports = 25 : 587 : 2525

[Matt Stevenson]

Hi Scott 

Am pretty sure Send Grid does not do inbound smtp only to a URL. 

https://sendgrid.com/blog/smtp-service-crash-course/

Not sure what the alternatives are then, externally hosted ?

Regards

Matt

[Kamran (Google Cloud Support)]

Hello Matt,

Thank you for your messages. As I understand you can send out emails from your Exim server but it does not receive emails. In order to troubleshoot this issue, I recommend trying the following steps:

1. If you have created your VM from CentOS image, Postfix is running as the default mail transfer agent (MTA). If you want to use EXIM as the defult MTA instead, run these commands (please note this won't apply to Debian OS):

1a. stop the postfix service and disable it

$ sudo service postfix stop

$ sudo chkconfig postfix off

1b. set Exim as the default MTA

$ sudo alternatives --config mta

There are 2 programs which provide 'mta'.

  Selection    Command

-----------------------------------------------

*+ 1           /usr/sbin/sendmail.postfix

   2           /usr/sbin/sendmail.exim

Enter to keep the current selection[+], or type selection number: 2

1c. Set Exim as auto start service on boot time:

$ sudo chkconfig exim on

$ sudo service exim start

2. Run sudo netstat -plnt command to make sure that Exim service is running and listening on port 25 for 0.0.0.0. Example output:

tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      17098/exim  

tcp6       0      0 :::25                   :::*                    LISTEN      17098/exim

3. Now on GCE firewall, create a rule to allow incoming traffic for tcp:25 protocol/port pair destined to your VM.

After doing the above mentioned steps, you should be able to receive emails on your EXIM server.

Please let me know if you have further questions regarding this.

Sincerely,

[Matt Stevenson]

Hi Karman

Thanks for joining the discussion here I have probably got myself confused.

Anyway for clarification here is where I am at.

Exim wa already MTA and no sign of postfix, although I do see dovecot on ports 110, 993,995. Exim is on 2525, 587, 465, 25.

From reading Scotts reply it appeared that 25 and 587, 465 was blocked outbound by Google so I was not sure about how a firewall rule for 25 inbound would help in this case ? Anyway I have created a rule for now to follow through your thought process to see if it improves anything.

Sending to a email address hosted on the GCE VM Iam looking in the logs.

/var/log/exim_mainlog for any evidence of incoming delivery. I am seeing receipts of outgoing emails only.

I'm arranging a conversation with SendGrid as I am hearing from them that the API for incoming email is only able to parse to a URL. Not sure if thats a relative or absolute URL either so for instance if it were per domain I could perhaps figure out how to hand it off to Exim via the right php URL ?

Any thoughts either of you ?

Kind Regards

Matt

[George (Google Cloud Support)]

Hello Matt,

As mentioned in this Help Center article : "While sending email from blocked ports is not allowed, your instance can still receive emails", which is true if the port on both GCE and the instance firewall is open.

However, you can post your question with the relevant tags on Stackexchange(ServerFault, Stackoverflow...) where community and Google's engineers are active as well and the question looks to be better suited there.

I hope this helps.

Sincerely,

George

[Matt Stevenson]

Hi George 

Many thanks for jumping in here, and for the recommendations. I am happy with the outbound mail, I am now looking for inbound email to function. 

I have an arranged meeting next week with Sendgrid and will update after that. 

Kind Regards

Matt

[Joshua Dharmawan]

Hi George,

I have similar case.

I use sendgrid, and already follow all step from sendgrid, and also using port 2525.

The email never be able to sent out. 

here is the  maillog:

786 P=esmtpa A=dovecot_login:jos...@xxxxxx.net S=776 id=8e67d18c4ec600bbf1e66

c124f354ed...@ns9.icentra.net T="a dsfa sdf adf" for myemail...@yahoo.com

2017-02-27 21:55:31 SMTP connection from (ns9.icentra.net) [::1]:54786 closed by QUIT

2017-02-27 21:55:31 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1ciMi3-000287-MF

2017-02-27 21:55:31 1ciMi3-000287-MF ** myemail...@yahoo.com R=send_via_sendgrid T=s

endgrid_smtp: all hosts for 'yahoo.com' have been failing for a long time (and retry 

time not reached)

Please help..

Disclaimer: This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission.

[Matt Stevenson]

Hi Joshua 

I do feel your pain as the Exim docs don't even refer to ports for GCP. 

After making changes to the exim conf and mx records I did get mail to send. 

The issue we had is the return address and not figuring out how to receive any replies. 

We are currently looking into smart host config and selecting a number of options inc postfix setup. 

Kind Regards

Matt

On Monday, 27 February 2017 15:15:19 UTC, Joshua Dharmawan wrote:

Hi George,

I have similar case.

I use sendgrid, and already follow all step from sendgrid, and also using port 2525.

The email never be able to sent out. 

here is the  maillog:

786 P=esmtpa A=dovecot_lo...@xxxxxx.net S=776 id=8e67d18c4ec600bbf1e66

[Sam Gosper]

Hi Matt,

I also am trying to achieve mail send/receive via WHM on GCE.

Did you solve the receiving end? I am just getting started now. I assume the username/password in AUTH section of Exim is where your apikey goes under username and password?

[Klenio Araujo]

A good tip is to observe the sendgrid password. Delete the special password characters, I spent hours trying the exim settings do not support characters in the password

sendgrid_login:

driver = plaintext

public_name = LOGIN

client_send =: aaaaabbbb: AAAbbCCCCCDDD

do not use: &% $ # "! * +

[Jesse Ebbett]

 

i have found that there are a few steps to completing this task.

first in google cloud console, you must go into VPC Networks -> Add firewall rule

next you will add a new rule and allow tcp:2525 

then in whm we will go to the exim advanced editor and complete the setup described here

So far I have been able to send mail from my server. but i am still working on receiving mail.. will update this post as i go.

REFERENCES: 

GC - Sending with SendGrid
GC - Sending Mail