SMTP Test 8, 14, 18 not working since 7/21/25

[lcm...@emrdirect.com]

Hello,

This test was working as expected on 7/14/25. The associated SUT setup and ETT testing profile values have not been changed since that time.

After clicking the RUN button the test fails with the following displayed in the test logs:

ERROR: No messages found! Send a message and try again. Please make sure that the Vendor Email Address is entered and matches the email address from which the email is being sent. Wait for atleast 30 seconds after sending the email to ensure successful delivery to the ETT.

Waiting for >30 seconds is not the issue. We can confirm from SUT SMTP logs that the messages are being accepted by the james server as expected. This behavior was first noted on 7/21/25.

Thank you for your attention to this issue.

Luis Maas, CTO, EMR Direct

[Sai Valluripalli]

Service has been restarted. Can you please try again.

[lcm...@emrdirect.com]

The error is still occurring. The log message is the same as before.

[Sai Valluripalli]

Can you please try again now and let us know. 

--
You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/edge-test-tool/49554496-eb86-4938-840e-4ac3b3efd622n%40googlegroups.com.

[lcm...@emrdirect.com]

We tried it again this morning and the test worked as expected. 

Thanks.

[lcm...@emrdirect.com]

Well, the ETT was performing as expected briefly for this test today, but it stopped working again after a few hours.

The issue/error is the same as we noted earlier in this thread.

[Sai Valluripalli]

Hi Luis,

Could be an intermittent issue, please try again. 

Thanks

To view this discussion visit https://groups.google.com/d/msgid/edge-test-tool/33d93972-b6b1-46f9-8e4b-e3572ac8d5e3n%40googlegroups.com.

[lcm...@emrdirect.com]

Hello, it was working again as of a few minutes ago.

Since you have noted that this could be an intermittent issue, would you please let us know when the root cause has been determined and corrected so that we may best advise our customers that are currently preparing for certification testing?

Thank you

[lcm...@emrdirect.com]

Hello again,

SMTP Test 8,14,18 has stopped working again since this morning. Same symptoms as discussed above in this thread.

This is impacting some of our customers currently preparing for certification testing.

Has the cause for this been determined?

Thank you,

Luis Maas, CTO EMR Direct

[Nagesh Bashyam]

Luis

 

We will look into it asap, did not realize you were having issues with SMTP 8,14,18

 

To view this discussion visit https://groups.google.com/d/msgid/edge-test-tool/408bd2b7-b728-4677-a737-f1610afc9e73n%40googlegroups.com.

[lcm...@emrdirect.com]

Hi,

FYI, as of a few minutes ago, this test working again.

However, it has come to my attention that someone on the ETT team recently instructed one of our customers to send a regular email message from gmail to wellf...@ett.healthit.gov and that the test would pass. I have confirmed that the wellf...@ett.healthit.gov address will receive messages sent from a "regular" external email address without enforcing the SASL authentication requirements for this edge protocol, and that SMTP 8,14,18 will display a success result after sending a message this way.

This is problematic because, as you know, the tool is supposed to evaluate conformance with the Edge Protocol guide, and b.1 test addresses sending to the ETT via SMTP are required to use SASL for authentication, among other things. The mail server at james.healthit.gov appears to enforce the SASL requirement, but the public MX record for the ett.healthit.gov domain points to a different SMTP server (not james.healthit.gov), so there may be more than one route on your side to deliver mail to this ETT test address.

[Sai Valluripalli]

Hi Luis,

Sending email from gmail was not an instruction but rather a test to make sure the tool is able to receive emails. 

The right way is to always send using SASL authentication.

To view this discussion visit https://groups.google.com/d/msgid/edge-test-tool/962f2005-9c41-4340-b413-08c9a968dd36n%40googlegroups.com.

[lcm...@emrdirect.com]

Of course, I agree that following the requirements and using SASL is the right way to send.

However, the tool still displays 'success' when the message is sent the 'wrong' way.

This means that a SUT that was designed incorrectly and is sending the 'wrong' way will still get a success result for this test, and the test lab may not realize that the developer is not following the requirements.

[lcm...@emrdirect.com]

Hi again, this test has stopped working again for messages submitted the 'right' way through the james.healthit.gov server. It was working two hours ago. We can see from our server logs that the messages have been accepted by the james server, as we received a "250 2.6.0 Message received" SMTP response after submitting the message.

As a side note, we sent a message the 'wrong' way between two failures the 'right' way and this went through with a 'success'.

Also, we have been running this SMTP Test 8,14,18 for many years with many developer customers with only sporadic issues that were addressed when the ETT services were restarted by your team. It appears that something has changed in the ETT more recently (~past two months) that has significantly decreased the reliability of this test.

[Sai Valluripalli]

Hello Luis,

We are looking into finding the root cause of the stability issues. We performed a bunch of clean up/maintenance on the servers which should help.

Thanks

[Faisal Shahzad]

Hello Sai,

We have been encountering the same issue with SMTP Test 8-14-18 its keeps on failing as the message is being sent properly. Please advise when this can be fixed we have a test date scheduled on this Friday.