sandcats.io and DNS records for SPF, DKIM, DMARC, etc.

[Troy Farrell]

Hi Sandstormers,

I have a Sandstorm server that uses a sandcats.io domain.  I send e-mail from
this server (through a Postfix SMTP server to enable TLS) and have intermittent
delivery problems.  This is not surprising, as sandcats.io subdomains lack SPF,
DKIM and DMARC records.  Having read some of the related issues on GitHub, I
would like to get your sense of the most reasonable path to implementing SPF,
DKIM and DMARC records for sandcats.io subdomains.  Here are some options that
I have considered:

- sandcats.io could delegate to an authoritative DNS server hosted in a grain.
  This would possibly make DNS less reliable, but would give the server
  administrators maximum flexibility in creating the desired records.

- sandcats.io could be extended to receive and host the desired SPF, DKIM and
  DMARC records.

- sandcats.io could decline to support the necessary records as a matter of
  policy.  Instead, administrators should use domains which they control.  This
  hands the problem to the administrators, but seems to go against the spirit
  of Sandstorm and sandcats.io, in that it increases the difficulty of setting
  up a fully-functional Sandstorm server.

Does any of these options look like a good way to solve the specific problem of
mail deliverability?  Are there any potential vulnerabilities in the options

that I've suggested?

Is there any interest in a broader discussion about other DNS records, like MX, SRV, SSHFP, etc?

Thanks,
Troy

[Jacob Weisz]

My understanding is that at present, Sandcats is not intended to support mail. For instance, I believe it is presently impossible to receive mail using a Sandcats address.

I imagine the best case if it were to support mail, would be for the Sandcats service to provide and manage all of the necessary records, and provide an API for the Sandstorm server to gather any keys needed to configure itself to use it. Though this would be a nontrivial expansion of Sandcats that I imagine would have to be a community contribution.

Additionally, I do have concerns about reputation: While Sandcats is technically wired to work for Sandstorm servers, an implementer looking at the API could abuse it to register Sandcats domains for other uses... including potentially spam. A big risk to supporting mail in Sandcats is that the reputational risk might affect all Sandcats servers if a single Sandcats domain was used for spam. I am unsure of the extent of the risks here.

That's a big reason Sendgrid/Mailgun/etc. seems to be a preferable choice for Sandstorm mail.

--

  Jacob Weisz

  goo...@jacobweisz.com

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/75dff461-77ff-4d39-9dd0-590a7b050a1fn%40googlegroups.com.

[Troy Farrell]

Thanks for taking the time to reply.

Jacob Weisz wrote:

My understanding is that at present, Sandcats is not intended to support mail. For instance, I believe it is presently impossible to receive mail using a Sandcats address.

I currently receive mail using a Roundcube grain on a sandcats.io subdomain.

 

I imagine the best case if it were to support mail, would be for the Sandcats service to provide and manage all of the necessary records, and provide an API for the Sandstorm server to gather any keys needed to configure itself to use it. Though this would be a nontrivial expansion of Sandcats that I imagine would have to be a community contribution.

Are you suggesting that this would be built into Sandstorm's admin interface and connected to sandcats.io via an HTTP API?

Yes, I don't expect this to magically appear.  I would have walked away from Sandstorm long ago if that were the case. ;-)

 

Additionally, I do have concerns about reputation: While Sandcats is technically wired to work for Sandstorm servers, an implementer looking at the API could abuse it to register Sandcats domains for other uses... including potentially spam. A big risk to supporting mail in Sandcats is that the reputational risk might affect all Sandcats servers if a single Sandcats domain was used for spam. I am unsure of the extent of the risks here.

You're absolutely right to be concerned about the reputation of the domain.  Currently, most mail reputation models are based on IPv4 addresses, but domains are expected to become more important as the use of IPv6 for mail delivery increases.  However, this risk is a present reality.  I'm not convinced that supporting SPF, DKIM and DMARC increases the risk.  If anything, ultimate control rests with the person in control of the sandcats.io name server.  (When a spammer is identified, sandcats.io can simply stop resolving that subdomain.)

 

That's a big reason Sendgrid/Mailgun/etc. seems to be a preferable choice for Sandstorm mail.

Using Sendgrid or Mailgun does not resolve the question of SPF, DKIM and DMARC records.  That would be handled by using a non-sandcats.io domain.

Troy

[Ian Denhardt]

Quoting Troy Farrell (2022-01-26 11:45:35)

> I currently receive mail using a Roundcube grain on a sandcats.io
> subdomain.

To be clear, are you somehow using a f...@bar.sandcats.io address to
receive mail? How, without MX records? My understanding is that the
Roundcube grain is just acting as an MUA that happens to live in a
sandstorm grain, rather than enabling sandstorm to act as an email
server. But perhaps I have missed something.

> I imagine the best case if it were to support mail, would be for the
> Sandcats service to provide and manage all of the necessary records,
> and provide an API for the Sandstorm server to gather any keys needed
> to configure itself to use it. Though this would be a nontrivial
> expansion of Sandcats that I imagine would have to be a community
> contribution.

I think if we we're going to do this, it would be better to have
sandstorm support DNS configuration more generally, so it could also
configure mail for domains with any of the DNS providers that it can
currently manage ACME for. This also could be a step towards making
static publishing easier, avoiding the user needing to manually set
up a TXT record for the grain.

This might be a good feature to add even if we decide we don't want
to support it for sandcats itself.

> Additionally, I do have concerns about reputation: While Sandcats is
> technically wired to work for Sandstorm servers, an implementer looking
> at the API could abuse it to register Sandcats domains for other
> uses... including potentially spam. A big risk to supporting mail in
> Sandcats is that the reputational risk might affect all Sandcats
> servers if a single Sandcats domain was used for spam. I am unsure of
> the extent of the risks here.

I guess this would ultimately be dependent on whether Kenton wants to
deal with the administrative overhead. I think the worst case scenario is
that we are unable to maintain an acceptable reputation for the domain and
end up dropping support after having built out the feature.

[Troy Farrell]

Ian wrote:

Quoting Troy Farrell

> I currently receive mail using a Roundcube grain on a sandcats.io
> subdomain.

To be clear, are you somehow using a f...@bar.sandcats.io address to
receive mail? How, without MX records? My understanding is that the
Roundcube grain is just acting as an MUA that happens to live in a
sandstorm grain, rather than enabling sandstorm to act as an email
server. But perhaps I have missed something.

RFC 5321 Simple Mail Transfer Protocol

Section 5.1 Locating the Target Host

https://datatracker.ietf.org/doc/html/rfc5321#section-5.1

"If an empty list of MXs is returned, the address is treated as if it was associated with an implicit MX RR, with a preference of 0, pointing to that host."

Sandstorm will receive mail send to example.sandcats.io provided it is listening on port 25 of the IP addresses that are resolved by example.sandcats.io.

 

I think if we we're going to do this, it would be better to have
sandstorm support DNS configuration more generally, so it could also
configure mail for domains with any of the DNS providers that it can
currently manage ACME for. This also could be a step towards making
static publishing easier, avoiding the user needing to manually set
up a TXT record for the grain.

This might be a good feature to add even if we decide we don't want
to support it for sandcats itself.

I agree that we should make static publishing easier and I'm glad if this supports that goal.

Would it be possible to run authoritative DNS server in a grain, provided that grain implements IpInterface/listen{Tcp,Udp}?  I don't really understand that part of Sandstorm, but I want to.

 

I guess this would ultimately be dependent on whether Kenton wants to
deal with the administrative overhead. I think the worst case scenario is
that we are unable to maintain an acceptable reputation for the domain and
end up dropping support after having built out the feature.

I would likely use a builtin authoritative DNS server, whether with sandcats.io or not.  I will need one eventually and making it part of Sandstorm is generally desirable in my opinion.

[Ian Denhardt]

Quoting Troy Farrell (2022-01-28 01:11:07)

> Would it be possible to run authoritative DNS server in a grain,
> provided that grain implements IpInterface/listen{Tcp,Udp}? I don't
> really understand that part of Sandstorm, but I want to.

Yes, this should work fine.

> I would likely use a builtin authoritative DNS server, whether with
> sandcats.io or not. I will need one eventually and making it part of
> Sandstorm is generally desirable in my opinion.

For sandcats, we would still need to point an NS record at it.

[Troy Farrell]

Ian wrote:

Quoting Troy Farrell (2022-01-28 01:11:07)

> Would it be possible to run authoritative DNS server in a grain,
> provided that grain implements IpInterface/listen{Tcp,Udp}? I don't
> really understand that part of Sandstorm, but I want to.

Yes, this should work fine.

Great.  I'll give it a shot with Rust.  We probably want something that starts up quickly and I feel better about using Rust than C or C++.  I'll probably get stuck and need help at some point.

 

> I would likely use a builtin authoritative DNS server, whether with
> sandcats.io or not. I will need one eventually and making it part of
> Sandstorm is generally desirable in my opinion.

For sandcats, we would still need to point an NS record at it.

I'm sure that Kenton would be amenable to experimenting with it. ;-)

[Ian Denhardt]

One possible snag occurs to me now that didn't the other day: you may
not be able to listen on privileged ports (<1024).

Quoting Troy Farrell (2022-01-30 09:02:53)

> Ian wrote:
>
> Quoting Troy Farrell (2022-01-28 01:11:07)
> > Would it be possible to run authoritative DNS server in a grain,
> > provided that grain implements IpInterface/listen{Tcp,Udp}? I
> don't
> > really understand that part of Sandstorm, but I want to.
> Yes, this should work fine.
>
> Great. I'll give it a shot with Rust. We probably want something that
> starts up quickly and I feel better about using Rust than C or C++.
> I'll probably get stuck and need help at some point.
>
> > I would likely use a builtin authoritative DNS server, whether
> with

> > [1]sandcats.io or not. I will need one eventually and making it

> part of
> > Sandstorm is generally desirable in my opinion.
> For sandcats, we would still need to point an NS record at it.
>
> I'm sure that Kenton would be amenable to experimenting with it. ;-)
>

> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [2]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [3]https://groups.google.com/d/msgid/sandstorm-dev/f57af7a6-60aa-4cba-9
> e83-191a976f3c5en%40googlegroups.com.
>
> Verweise
>
> 1. http://sandcats.io/
> 2. mailto:sandstorm-de...@googlegroups.com
> 3. https://groups.google.com/d/msgid/sandstorm-dev/f57af7a6-60aa-4cba-9e83-191a976f3c5en%40googlegroups.com?utm_medium=email&utm_source=footer

[Troy Farrell]

Ian wrote:

One possible snag occurs to me now that didn't the other day: you may
not be able to listen on privileged ports (<1024).

True.  That may require extra effort.  I haven't gotten that far yet.

I've been trying to get Deno to compile my TypeScript files and output a source map.  I finally sorted that out to discover that Firefox's dev tools cannot get the TypeScript source files because of a problem with the network request.  (I can't see the dev tools' requests in the Network pane, so I don't know whether it's unhappy because http://local.sandstorm.io:6090 is not HTTPS or whether it's not using the proper Firefox account container or what.)  So I thought that I might publish the TypeScript source files to the static space (/var/www) until I discovered that .ts files have a MIME type of video/vnd.dlna.mpeg-tts because of MPEG Transport Streams.  I wondered whether we have a good mechanism for specifying the desired MIME type of content in /var/www.  (I didn't find one.)

That's how it's going.

[Ian Denhardt]

Quoting Troy Farrell (2022-02-02 12:25:53)

> I don't know whether it's unhappy because http://local.sandstorm.io:6090

> is not HTTPS [...]

One thing you could try is setting up a local dev install with config:

BASE_URL=http://localhost:6080
WILDCARD_HOST=*.localhost:6080

These days most (all?) browsers support the localhost wildcard, and I
*think* treat it as a secure context. IIRC we mainly continue to
default to local.sandstorm.io for the sake of non-browser clients
for api usage.

> I wondered whether we have a good mechanism for specifying the
> desired MIME type of content in /var/www. (I didn't find one.)

We don't, unfortunately :(

[Troy Farrell]

I used Wireshark to verify that my guesses were correct.  When I disabled Firefox's HTTPS-only mode, the dev tools' requests for the TypeScript source code made it to Sandstorm.  Sandstorm rejected them because the dev tools were not using the correct container for Sandstorm development, thus did not provide Sandstorm with the correct cookie.  When I open http://local.sandstorm.io:6090 in incognito mode and disable HTTPS-only mode, then the inline source map works and I can view the TypeScript source code in Firefox's dev tools.

Onward to Powerbox requests!

[Troy Farrell]

I'm finally getting to making the powerbox request and realizing that I'm not sure that I understand what to do.  Here's what I think is the correct approach:

1. Implement {TcpPort,UdpPort}::Server in my grain.
2. Put {TcpPort,UdpPort} into ViewInfo::MatchOffers.

3. Make a Powerbox request for IpInterface @0xe32c506ee93ed6fa.  Get a claim token.  Send the claim token to the server.

4. Call SandstormApi.save() on the IpInterface capability.

5. Call listenTcp(53, DnsOverTcpPortImpl::Server) using the IpInterface capability and do some unknown thing with the Util.Handle that I get back.
6. Call listenUdp(53, DnsOverUdpPortImpl::Server) using the IpInterface capability and do some unknown thing with the Util.Handle that I get back.

Questions:

1. Does the above make sense?

2. It seems like I would need to request a PersistentIpSession, but that doesn't have an ID, so I don't think that I can request it.  Is this correct?
3. Can I save()/restore() my IpInterface capability?

4. Is save()/restore() what I should be doing for a server grain like this?

5. What do I do with the Util.Handle from {listenTcp,listenUdp}?

Thanks for your help!

Troy

[Ian Denhardt]

Quoting Troy Farrell (2022-02-06 11:02:44)

> Here's what I think is the correct approach:
>
> 1. Implement {TcpPort,UdpPort}::Server in my grain.
> 2. Put {TcpPort,UdpPort} into ViewInfo::MatchOffers.
>
> 3. Make a Powerbox request for IpInterface @0xe32c506ee93ed6fa. Get a
> claim token. Send the claim token to the server.
>
> 4. Call SandstormApi.save() on the IpInterface capability.
>
> 5. Call listenTcp(53, DnsOverTcpPortImpl::Server) using the IpInterface
> capability and do some unknown thing with the Util.Handle that I get
> back.
> 6. Call listenUdp(53, DnsOverUdpPortImpl::Server) using the IpInterface
> capability and do some unknown thing with the Util.Handle that I get
> back.

Exactly.

> 2. It seems like I would need to request a PersistentIpSession, but
> that doesn't have an ID, so I don't think that I can request it. Is
> this correct?

No, you don't need to request that. It exists because interanlly
sandstorm needs a way to export a capability that implements both
Ip{Network,Interface} and SystemPersistent, and so it needs an interface
that inherits from both. But PersistentIpInterface doesn't have any
meaning beyond that, and the SystemPersistent interface is an
implementation detail that you don't need to worry about (and isn't
exposed directly to the grain anyway).

...though fwiw, all interfaces have an id; if there isn't one specified
explicitly in the schema, the schema compiler derives one from the
parent scope and the name.

> 3. Can I save()/restore() my IpInterface capability?
>
> 4. Is save()/restore() what I should be doing for a server grain like
> this?

Yes.

> 5. What do I do with the Util.Handle from {listenTcp,listenUdp}?

Just hang on to it; dropping it is how you stop listening. Usually when
you see a Handle it's something like that. This should probably be
stated explicitly.

> Thanks for your help!

No problem!

-Ian

[Troy Farrell]

I'm about halfway through the above steps.

Thinking ahead, how are we going to manipulate DNS records?  We need to manage temporary records for ACME.  We need to manage records for static web publishing.  We need to manage records for SPF, DKIM and DMARC.  It would be nice if we could set records for virtual hosting; for example, this server used to be known as example.sandcats.io and is now known as example.com and should respond to both so old links are not broken.  It would be nice if there was a way to drop records when the grain that created them is destroyed.  Supporting zone transfers would be good, so we can have backup servers.

It seems that our use cases point to having a Cap'n Proto interface, which could be given to other grains so they can add records.  RFCs 2136 and 2137 describe a means of updating records using DNS requests, which is mostly used by DHCP servers, so I don't know how useful that would be.  An HTTP API would be useful for integration with system management tools, like Puppet and Ansible, but that seems like a secondary function.  However, implementing a common HTTP API (like PowerDNS or AWS Route 53) would make it easier to implement management clients.

Thoughts?

[Ian Denhardt]

+1 for a capnp api.

Quoting Troy Farrell (2022-02-08 22:53:55)

> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [1]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [2]https://groups.google.com/d/msgid/sandstorm-dev/95660406-82cc-4a6e-9
> 78c-d795f65a8be2n%40googlegroups.com.
>
> Verweise
>
> 1. mailto:sandstorm-de...@googlegroups.com
> 2. https://groups.google.com/d/msgid/sandstorm-dev/95660406-82cc-4a6e-978c-d795f65a8be2n%40googlegroups.com?utm_medium=email&utm_source=footer