SMTP Sender MT tests 21, 23, 24, 25, 26, 27, 28 and 29(a)

[bs...@secureexsolutions.com]

In the setup we specify Vendor (Direct) Email Address (in SASL this would be Authorization context), along with Vendor User Name and Vendor Password (SASL Authentication context) and would have expected that messages sent to test various conditions would be at least sent FROM the Vendor Email Address. Instead all messages are sent from the "User Name", which is not necessarily an email address or has a mailbox - and in our case it is not, so there would be no place to receive MDNs, even for the cases where they are expected to come back.

Further, what sort of response is expected and what would we have to demonstrate to the proctor during the test to prove that we properly handle these message tracking conditions? Would report showing the status of the message suffice or do we have to present the logs showing exactly what happened? 

Test description is quite wide open and inconsistent with the specs. For example, this is for test 21 (a):

Due to SMTP-SMTP configuration limitations, the proctor is to visually inspect for the receipt of the positive delivery status notification message.The expectation is that the Vendor will handle the header appropriately and provide a processed MDN.

The MDN would have to come from the destination HISP, which in this case is ETT, not from us. So, are we just supposed to demonstrate that we did receive the MDN and would have sent a DSN if there was a place where to send it? How would the latter be demonstrable?

There are several test cases where Direct message cannot even be sent since there is no MX record, no cert, no valid address - in SMTP-SMTP setting message gets delivered to our mail server, ETT disconnects indicating success of message delivery and we are left holding the proverbial bag - there will be no MDN from anywhere, message was never sent, now what - we send a negative DSN to ourselves? Description again says that:

Due to SMTP-SMTP configuration limitations, the proctor is to visually inspect for the receipt of the negative delivery status notification message. The expectation is that an appropriate delivery failure message will be returned.

How do we show returning DSN if there is no place to return it to?

Last, just a point of curiosity - how does test 28(a), "Delayed Dispatched MDN", differ from test 27(a), "No Dispatched MDN"? In practice it is delayed forever, as it never comes and description follows that:"Hitting ‘Run’ will cause ETT to send a message with a final destination that will not provide a dispatched MDN." Seems like the same as 27(a): "Hitting ‘Run’ will cause ETT to send a message with a final destination that will not provide a dispatched MDN."

[Gavin]

I finally managed to coax the tool into giving a green success message for the negative test cases.

I return MDNs without TLS or authentication to the following.

• Address: fail...@ttpds.sitenv.org
  
• Host: ttpds.sitenv.org:25

The positive cases (21/29) still don't work for me and probably need another return address that I've been unable to find. The above address was conspicuously located in the FAQ: https://github.com/siteadmin/ett/wiki/FAQ

Hope this helps.

Gavin

[bs...@secureexsolutions.com]

I don't know about that... Message is sent TO fail...@ttpds.sitenv.org, so any would you return to it as well? Seems like you are jumping through hoops to make tool show green checkbox, but it is not really the expected test result.

[Gavin]

Yep I'm returning all MDNs to fail...@ttpds.sitenv.org despite the fact that SMTP MT Test 3 also happens to try sending outbound Direct to that address.

The ETT is displaying the contents of my MDNs in the logs so I think it's safe to assume that it is searching that particular inbox for verification. If this is somehow still not the intended workflow then that brings the validity of all test results into question.

[bs...@secureexsolutions.com]

You mean you are doing it for all tests - whether original email was sent to failure15... or not? What about the test where you cannot even send a message - there is no address/domain/MX record, so there is no MDN at all?

[Gavin]

Yep I understand your confusion because I too doubted it would work before I tried. I'm returning MDNs for all failure SMTP MT Sender tests (not 21/29) to the same Edge endpoint. Despite the lack of clarity in the instructions it looks like the validator is checking in failure15 for all of these tests.

Much like the XDR tests that provide non-https endpoints, it appears that TLS/auth is not required here because SMTP Test 1-8 (Send) and SMTP Test 9,16,20 (Receive) each confirm that we are able to perform it correctly.

All of these are testing our ability to provide the appropriate failure/success MDN back to the ETT as Edge. Regarding the test where we cannot even send the outbound Direct message -- we are still obligated to generate and return a failure MDN to the Edge client to tell them as much.

This illustrates the various Edge MDN scenarios: http://wiki.directproject.org/file/view/Implementation+Guide+for+Delivery+Notification+in+Direct+v1.0.pdf

[bs...@secureexsolutions.com]

My confusion just got more confused, did not think it was possible. Why would you (or any HISP for that matter) return MDN to the edge system? Edge *may* expect a DSN, even though it is not required by the IG for delivery, which says that edge may farm it out to the HISP, but edge system should not be receiving MDNs. Are we mixing terminology here, or are you actually sending encrypted/signed MDN to failure15 over Direct or unencrypted DSN over SMTP-SASL (or just SMTP without any authentication as you indicated below)?

[Gavin]

Ah I'm sorry, there's a good chance that I'm misusing the terminology. Your last point is what I've been struggling to describe -- an unencrypted DSN (I thought MDN) sent of SMTP to ttpds.sitenv.org.

[bs...@secureexsolutions.com]

Thank you, now it is clear what you are doing. I think the way test descriptions are formulated does not help, it does talk about MDNs and DSNs and proctor is supposed to visually inspect something - not the test results really...

See, for example, description for MT21(a):

Due to SMTP-SMTP configuration limitations, the proctor is to visually inspect for the receipt of the positive delivery status notification message.The expectation is that the Vendor will handle the header appropriately and provide a processed MDN.

I would not know where to look to inspect the receipt and why vendor (not ETT as receiving HISP) should provide a processed MDN. I still hope to hear from someone in ETT support about these tests and what is supposed to be a proof. Rigging something just to deliver DSNs to an address that has nothing to do with the "from" address seems to defeat the purpose of any testing as the result is just that, rigged.

[Sandeep]

Gavin,

We looked in the failure15 inbox and found that your MDNs have 'success' instead of 'dispatched'.

Reporting-UA: updox.com; Direct Security Agent

Final-Recipient: rfc822; sys...@updox.com

Original-Message-ID: <751650511.7835...@ip-172-31-38-171.us-west-2.compute.internal>

Disposition:automatic-action/MDN-sent-automatically;success

The positive cases look for 'dispatched' field in the MDNs. Try changing the 'success' field to 'dispatched' and you should be able to pass 21 and 29.

Thanks,

Sandeep

On Friday, February 3, 2017 at 4:41:12 PM UTC-5, Gavin wrote:

[bs...@secureexsolutions.com]

OK now, so it looks likeGavin was sending MDNs - unencrypted and over edge protocol - is THIS what you  expect in this test?! Why would HISP send an MDN over EDGE to the edge system instead of the DSN?

Can somebody please explain what is the intent of this test and what are we supposed to accomplish here? Send MDNs to ttpedge over regular SMTP without encryption - so NOT Direct MDN - and empirically figuring out that they should be sent to this one random mailbox failure15?!

On Monday, February 6, 2017 at 9:24:37 AM UTC-5, Sandeep wrote:

Gavin,

We looked in the failure15 inbox and found that your MDNs have 'success' instead of 'dispatched'.

Reporting-UA: updox.com; Direct Security Agent

Final-Recipient: rfc822; sys...@updox.com

Original-Message-ID: <751650511.7835.1486340378935@ip-172-31-38-171.us-west-2.compute.internal>

[srini]

These tests in discussion is SMTP 21(a), 23(a)..etc. (and 1-5) which are testing the HISP choosing SMTP protocol both ways to communicate to Edge. Most of the Edges use IMAP/POP and they correspond to the b/c counterparts - they don't need to run the (a) tests and would run (b) or (c) based on their choice.

ETT is the source Edge; so the mailbox has to live in ETT - and protocol is plain SMTP and not Direct - and that is the endpoint in discussion; work is in progress to make this completely manual - whereby the HISP has to demonstrate that it is delivering the MDNs to the Edge - instead of ETT doing any check. 

the success notification has to be an MDN, but the failure can be an MDN or DSN

- as per 1.1/1.2  of the IG for DN.

Please let us know if this addresses your questions,

thanks

srini

--
You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-tool+unsubscribe@googlegroups.com.
To post to this group, send email to edge-test-tool@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/e85b8773-fbfb-44bc-90d9-92b15b11b913%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[bs...@secureexsolutions.com]

I would respectfully disagree with your interpretation of the IG for delivery notifications. Sections 1.1.and 1.2 are about STA-to-STA, not about STA-to-Edge at all.

The (sending) STA to (sending) edge, the topic of this discussion (again, don't care about SMTP/IMAP/POP, we do only SMTP-SMTP, which is what I am asking about) is described in 2.2.2 and 2.3.

Relevant parts:

2.2.2 Responsibilities of the Sending STA 

When a use case requires notification of delivery for a particular Direct message, the Sending STA: 

 SHALL request delivery notification messages from Receiving STAs. 

 SHALL notify or indicate back to the sender failure to deliver to Receiving STAs. 

 SHALL notify or indicate back to the sender failed or successful delivery to destinations based on any received positive or negative delivery notification messages it receives from Receiving STAs. 

 SHALL notify or indicate back to the sender failed delivery to a destination if no processed MDN is received from the Receiving STA within a reasonable timeframe. 

 SHALL notify or indicate back to the sender failed delivery to a destination if no requested delivery notification messages are received from the Receiving STA within a reasonable timeframe 

And in 2.3:

Post-handoff, regardless of whether the sender and receiver share the same STA or are served by two separate STAs, the sender’s STA SHALL notify the sender of the successful or failed delivery of the original Direct message by delivering a positive or negative delivery notification message as defined in this guide; this delivery notification message MAY not be the actual positive or negative delivery notification that was originally issued by the receiving STA. 

So MAY is not SHALL, and conceivably the STA may just forward the MDNs over to the Edge client, so this clause says you can send any MDN, not just for success - I have no idea how you conclude that positive notification has to be an MDN (dispatched?).

Moreover, there is not a single flow in section 4.0 or Appendix A that would show delivery of the MDN to the edge client.

Separate from whatever is being delivered, how do we deliver it is another issue. The configuration for the (a) tests is SMTP-SASL and messages you are sending are coming FROM *our Vendor Username* which happens to be in the format of the email address, but only provides authentication context. Hereis what message for test 21(a) looks like on our side going through SMTP-SASL interface:

=============================================================================

From: ett-...@mu3-smtp.directaddress.net

To: processedd...@ttpedge.sitenv.org

Message-ID: <1836731889.818...@ip-172-31-38-171.us-west-2.compute.internal>

Subject: Testing sending mail with Disposition Notification Header (Test

 Case MU2-21)!

Content-Type: text/plain; charset=us-ascii

Content-Transfer-Encoding: 7bit

Disposition-Notification-Options: X-DIRECT-FINAL-DESTINATION-DELIVERY=optional,true

Disposition-Notification-To: ett-...@mu3-smtp.directaddress.net

This is a message to a Address 6!

==============================================================================

How am I supposed to figure out where to send the delivery notification in whatever form?

To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To post to this group, send email to edge-te...@googlegroups.com.

[OS - ONC SI&T Team]

The tests were broken apart to meet the requirements for a variety of implementations; SMTP-SMTP, SMTP-IMAP, or SMTP-POP.  The configurations were designed with consultation from the Direct Community Working Groups. 

 

The SMTP – SMTP tests are supposed to execute from the ETT to the SUT, using the:  Vendor Hostname/IP, Vendor UserName, and Password.  Currently, the test is configured to send from the ETT to the users mail inbox from the users Vendor Username (email address) with a return an MDN address of fail...@ttpds.sitenv.org.  The limitation we encountered, most systems can’t return the MDN via SMTP.  We made the allowance for system for manual verification, due to this issue.  We will changing the test to manual verification with no automatic checking.  The MDN should be in your inbox, depending upon your configuration.

 

Thank you

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/1b9e3ce8-aad2-42da-bb8b-6c7a2e9673a3%40googlegroups.com.

[bs...@secureexsolutions.com]

Thank you for what looks like an official response. Two points to note: the Disposition-Notification-To header in the message sent by ETT has the same address as a from address, so I do not see how it is that you specify return MDN address to be something else. Which according to the RFC3798it should NOT be:

 MDNs SHOULD NOT be sent automatically if the address in the
   Disposition-Notification-To header differs from the address in the
   Return-Path header (see [RFC-MSGFMT]).  In this case, confirmation
   from the user SHOULD be obtained, if possible.  If obtaining consent
   is not possible (e.g., because the user is not online at the time),
   then an MDN SHOULD NOT be sent.

MDNs in Direct are *automatic* MDNs and as such they cannot be sent to addresses that do not match the Return-Path, which is:

Received: from WIN-8S263A00D58 (ec2-23-21-54-166.compute-1.amazonaws.com [23.21.54.166])

by WIN-8S263A00D58 with ESMTP

; Mon, 6 Feb 2017 15:49:20 +0000

Return-Path: <ett-...@mu3-smtp.directaddress.net>

MIME-Version: 1.0

X-UserIsAuth: true

Received: from ec2-52-26-237-9.us-west-2.compute.amazonaws.com (EHLO ip-172-31-38-171.us-west-2.compute.internal) ([52.26.237.9])

          by WIN-8S263A00D58 (JAMES SMTP Server ) with ESMTPA ID -2130764088

          for <processedd...@ttpedge.sitenv.org>;

          Mon, 06 Feb 2017 15:49:29 +0000 (UTC)

From: ett-...@mu3-smtp.directaddress.net

To: processedd...@ttpedge.sitenv.org

Message-ID: <1836731889.818...@ip-172-31-38-171.us-west-2.compute.internal>

Subject: Testing sending mail with Disposition Notification Header (Test

 Case MU2-21)!

Content-Type: text/plain; charset=us-ascii

Content-Transfer-Encoding: 7bit

Disposition-Notification-Options: X-DIRECT-FINAL-DESTINATION-DELIVERY=optional,true

Disposition-Notification-To: ett-...@mu3-smtp.directaddress.net

And one more point of confusion. I can definitely show MDNs received from *valid* destination addresses, but for addresses like "badad...@ttpds2.siten.org" and a few others that cannot be sent, what kind of MDNs am I supposed to produce? MDNs that I am basically sending to myself, where I have no way to sign them with the private key for the "bad address" that does not exist? These are immediate notifications that - if not for SMTP interface - would be returned to the sending client synchronously. Up to this point, all messages in the inbox were Direct messages, signed by the sender and encrypted by recipients key, but now there would have to be an MDN that is not signed, but still indicates that original recipient is "badad...@ttpds2.siten.org" - is that the approach?

From: ett-...@mu3-smtp.directaddress.net

To: processed...@ttpedge.sitenv.org

Message-ID: <1836731889.8189.1486396168178@ip-172-31-38-171.us-west-2.compute.internal>

To post to this group, send email to edge-t...@googlegroups.com.

[Gavin]

Hi Sandeep,

I retried with "dispatched" and it worked as you said.

Thanks for the help.

Gavin

On Monday, February 6, 2017 at 9:24:37 AM UTC-5, Sandeep wrote:

Gavin,

We looked in the failure15 inbox and found that your MDNs have 'success' instead of 'dispatched'.

Reporting-UA: updox.com; Direct Security Agent

Final-Recipient: rfc822; sys...@updox.com

Original-Message-ID: <751650511.7835.1486340378935@ip-172-31-38-171.us-west-2.compute.internal>

[OS - ONC SI&T Team]

Hello, 

The return path in your example is correct, the email from your SUT will send to processedd...@ttpedge.sitenv.org and returns back the MDNs; however, the ETT is looking for the MDN’s in the failure15 mailbox.  We are looking for the Message ID with the correct content.  The test was designed this way to ensure SMTP was used to send and we needed a different receiving account.  The test will be changing in the next release to a manual verification process where the proctor will need to review the SUT’s inbound and outbound email and ensure the proper MDN’s are managed.

 As far as MDN’s, all the negative test cases will produce a failure notification. All failures would be unencrypted and unsigned in your outbox for proctor verification.  The intention was for all MDN’s to route failure15@... with the Original MessageID and Disposition in that MDN.                                                                                                                                                                                                                                              

From: ett-...@mu3-smtp.directaddress.net

To: processed...@ttpedge.sitenv.org

Message-ID: <1836731889.818...@ip-172-31-38-171.us-west-2.compute.internal>

Subject: Testing sending mail with Disposition Notification Header (Test

 Case MU2-21)!

Content-Type: text/plain; charset=us-ascii

Content-Transfer-Encoding: 7bit

Disposition-Notification-Options: X-DIRECT-FINAL-DESTINATION-DELIVERY=optional,true

Disposition-Notification-To: ett-...@mu3-smtp.directaddress.net

 

This is a message to a Address 6!

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/e0c9daa2-4431-4a1c-aca7-bee9dfc1829c%40googlegroups.com.

[Gary Isaac]

Thank you sitteam. You mentioned changing to manual verification. Will this change be applied to all of the cases in this post subject heading? And, when will this change be done? I wondering if this manual verification can be implemented by ATLs now for those vendors having current issues?

"The limitation we encountered, most systems can’t return the MDN via SMTP.  We made the allowance for system for manual verification, due to this issue.  We will changing the test to manual verification with no automatic checking.  The MDN should be in your inbox, depending upon your configuration."

On Friday, February 3, 2017 at 3:03:08 PM UTC-5, bs...@secureexsolutions.com wrote:

[bs...@secureexsolutions.com]

Can someone comment on this - this is holding our certification testing since the requirements for verifying test results are less than clear. Can the proctor examine MDNs in the SUT mailbox to verify that they are being received/created appropriately given the condition of the test?

[OS - ONC SI&T Team]

That change is scheduled for the next ETT update, scheduled for 16 Feb 2017.  Please see the “Announcements” page for the release schedules, the link can be found at the bottom of each page of the ETT along with links to the “Release Notes”, and the “FAQ” page.

 

Any questions for pertaining to certification should be sent to the certification team:  ONC.Cert...@hhs.gov

 

Thank you

 

From: edge-te...@googlegroups.com [mailto:edge-te...@googlegroups.com] On Behalf Of Gary Isaac
Sent: Tuesday, February 07, 2017 5:56 PM
To: Edge Test Tool (ETT)
Subject: Re: SMTP Sender MT tests 21, 23, 24, 25, 26, 27, 28 and 29(a)

 

--

You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To post to this group, send email to edge-te...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/48c8f906-d2dc-4fee-a3dc-2b3746364cbd%40googlegroups.com.

[OS - ONC SI&T Team]

That’s the intent of the upcoming change for the HISP SMTP MT tests you listed, the proctor will need to examine the SUT’s mail box to ensure the system has managed the MDN’s properly.  We already added the narrative to the test, “Due to SMTP-SMTP configuration limitations, the proctor is to visually inspect for the receipt of the negative delivery status notification message.”   The upcoming update with change the functionality of button for those tests to allow for manual validation only.

 

Thank you

 

From: edge-te...@googlegroups.com [mailto:edge-te...@googlegroups.com] On Behalf Of bs...@secureexsolutions.com
Sent: Wednesday, February 08, 2017 2:37 PM
To: Edge Test Tool (ETT)
Subject: Re: SMTP Sender MT tests 21, 23, 24, 25, 26, 27, 28 and 29(a)

 

Can someone comment on this - this is holding our certification testing since the requirements for verifying test results are less than clear. Can the proctor examine MDNs in the SUT mailbox to verify that they are being received/created appropriately given the condition of the test?

--

You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To post to this group, send email to edge-te...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/d8cdf164-df9b-44a1-852e-fbc7104452b3%40googlegroups.com.

[Gary Isaac]

Thank you. What is the expexlctation of the MDN in the SUT mailbox? This is, does tthe MDN for the positive test cases indicate that the message was received? And, the MDNs for the negative test cases indicate the message was not received? Or, does the MDN just have to exist in the SUT's mailbox?

[OS - ONC SI&T Team]

The expectations are in the test narratives and the "More Info" pages for each test. E.g. SMTP MT Test 21, "... the proctor is to visually inspect for the receipt of the positive delivery status notification message. The expectation is that the SUT will handle the header appropriately and provide a processed MDN." The messages should be available for inspection in the incoming and outgoing mail boxes; receipt of the message, actioned or processed by the SUT, then sent. The expectation would be the same for any negative test cases.

Thank you

-----Original Message-----
From: edge-te...@googlegroups.com [mailto:edge-te...@googlegroups.com] On Behalf Of Gary Isaac
Sent: Friday, February 10, 2017 2:52 AM
To: Edge Test Tool (ETT)

--
You received this message because you are subscribed to the Google Groups "Edge Test Tool (ETT)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edge-test-too...@googlegroups.com.
To post to this group, send email to edge-te...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/edge-test-tool/7a331dea-460f-4e22-bd22-ce9896cc4fd4%40googlegroups.com.

[Gary Isaac]

Thank you. I apprexiate the quick response for my previous question. The test cases' descriptions mentioned that the messages are to be inspected for the appropriate response, or something to that affect. I would expect that a positive test case retuns a MDN with some content or indication that the message was successfully received. And, for the negative test cases that the MDN would be returned with some kind of content inducating that the message was not received or could not be delivered. Am I correct? Or, does the SUT just have to return a MDN stating that the message was processed for both negative and positive test cases?