EMAILCORE (Revision of core email standards) WG

• Date: Wednesday, March 10, 2021
• Time: 12:00-14:00 UTC
• Meetecho: https://meetings.conf.meetecho.com/ietf110/?group=emailcore&short=&item=1

• Jabber: emailcore@jabber.ietf.org
• Notes: https://codimd.ietf.org/notes-ietf-110-emailcore
• Scribe: Bron Gondwana

Chairs

• Alexey Melnikov alexey.melnikov@isode.com
• Todd Herr todd.herr@valimail.com

Documents:

https://datatracker.ietf.org/doc/draft-ietf-emailcore-rfc5322bis/?include_text=1
https://datatracker.ietf.org/doc/draft-ietf-emailcore-rfc5321bis/?include_text=1
https://datatracker.ietf.org/doc/draft-ietf-emailcore-as/?include_text=1

MINUTES

• Intro by Alexey
• No agenda bashing

Ticket 7: better definition for trace header fields

• Alexey - discussion on mailing list is towards making definition extensible
• Pete was going to talk about it, but doesn’t appear to be here.
• Hopefully non-controversial, but let’s see if there are comments.
• Pete Resnick shows up
• Pete: does anyone have questions? This seems not controversial
• John Klensin: seems to put all the burden of defining this on 5321… (laughter: yep).
• John: That works for me.
• It’s clearly better to have it in one place, and this is a compelling argument that it’s the right place.

Issue with “lone return path” (slide 8)

• Pete: Wording here also seems uncontroversial
• Alexey: if we agree that lone return path is OK, then this will require changing to “zero or more return-path”, but otherwise this seems good.
• Pete: yep, just wanted to check if the general structure is good.

Syntax (slide 9)

• Pete: described it - don’t know how people feel about this.
• John Levine: poked around and noticed that anything which adds a return path will also add at least one Received header.
• Alexey: trying to channel Ned Freed (who isn’t here) - if we’re talking about message created post-delivery. Fortunately or unfortunately, other types of software will strip Received:, so we need to accomodate that. This is more generic than just SMTP.
• Pete: we’ve got obsolete / read only syntax. Does that satisfy this problem?
• Alexey: it’s not a question of being obsolete.
• Pete: yes, there’s lots of things in Obsolete which mean “we shouldn’t send this on the wire”. IMAP is wire, so not sure we should reference that.
• Alexey: with “when you create draft messages they may have missing fields” exception that’s OK.
• Dave Crocker: think you’re proposing requiring fields which weren’t required in the past. There’s no compelling operational requirement. Don’t understand utility of obs field in terms of real world effect. Putting it there doesn’t match reality.
• Pete: obs - if we had to do it over again. Apologise for old self. They were meant to be “you have to deal with messages that have these, but don’t create them”
• Dave -> audio issues. Have described a theory of its use, but doesn’t match real world use. Would advise against using it this way.
• Dave: the more effort it takes to deal with a trace field, the more it raises questions about whether it’s worth this energy. The nature of this group is to fix significant problems. In abstract I get what’s driving this, inconsistencies etc - but there’s no operational problems that this is solving.
• Alexey: the issue is that existing practices didn’t match the document, so the document needs to be fixed.
• Pete: practice -> they get treated like other trace fields by sometimes stripping them out or using them to trace, but they weren’t allowed into that block before - which is why you introduce extension into the trace block to indicate in the document that “these are permissable in a block of trace fields”.
• Dave: as a construct that sounds great, and if the goal was to produce a clean version of these documents, that would be great - but don’t think that’s the scope.
• Bron: would it create problems?
• Dave: I can predict that it would be outside the scope of this working group?
• Barry: I think Dave is correct, we made the scope very minimal, and we should try to stay in that scope, so let’s try to keep this simple.
• Alexey: I don’t agree with not affecting operations, because what people are using the fields for don’t match what the spec says - so this is aligning the documentation with actual practice.
• Barry: the issues is “IMAP creates an email and it doesn’t have trace fields” then it’s invalid.
• Pete: if it doesn’t have ANY trace fields, then it’s perfectly valid. The entire trace block is optional. Two things we’re discussing here:
• 1. is the Return path WITHOUT any other fields legitimate?
• 2. there’s a separate question about whether optional fields need to be expressed as part of the trace block because of the other use of trace in other documents, or is it OK for them to just float freely in the top of the message?
• Barry: and up to now they have just floated freely correct? Alexey: that was the next slide! Are there other things free in the top of the block that have just floated?
• Barry: suggesting “let’s not fix that now”. Not as AD, just personal opinion.
• John Klensin: the path we’re going down is making me a little nervous. Because it seems to say “if something was not in 5322 but people were doing it, then it’s OK and we don’t need to say anything about it” which is inconsistent with a definition of an internet standard because a standard is supposed to represent reality.
• Because things have been done that haven’t caused 5322 to collapse entirely into disrepute. Think I’m OK with this either way, but the argument that it wasn’t in 5322 and people have been doing it, so we don’t need to reflect it at all now puts us on a slipery slope.
• Seems we’re creating some nasty spaghetti threading and ignoring what the goal of an internet standard is supposed to be. If the goal of this group is to do things that aren’t consistent with that, that’s a problem.
• Alexey: share John’s concern - if it doesn’t reflect reality, we should fix it.
• Alexey: we were concentrating on IMAP, but not only IMAP does this. In IMAP we can potentially fix it in other ways. Consider mail filters or other systems that strip received fields for security reasons (to hide internal network information)
• Pete: are they stripping just Received headers?
• Alexey: good question, I don’t know whether or not they also stip Return-Path. (If they do, this change is not needed)
• Bron: as a software dev - they’ll be stripping based on a regular expression: 90% they’re just looking at messages and 10% they are refering to our documents.
• Alexey: think we’ll have to take this to the mailing list, unless something in the next slide makes it clear.

Slide 10 - better definition of trace header fields

There’s an optional block of stuff at the top: trace, optional-field, additional fields…

• Pete: proposal from the list. If “optional field” is going to be in trace anyway, do we need it separately as an optional field in the whole top of the optional fields?

• OR - is every additional field either trace or the resent-* fields? Was added when people realised you couldn’t put Authentication-Results at the top of a message, which everyone agreed was wrong.

• Alexey: for me question is: can non-trace header fields be inserted at the top? Side effect of removing this is disallowing them if they already existed.

• John: if your intent per the earlier conversation is to put the definition into 5321 then there has to be optionality there, regardless of what additional optionality 5322 provides. Gut says “while taking that out feels like a simplification” then it might not be across both

• Bron: systems don’t always do ordering.

• Pete: system in 5322 (all trace, etc have to be above from / sender /etc) but there’s an explicit warning in 5322 that things could be re-ordered but you can’t depend on it.

• Alexey: as a participant, would rather leave optional field here for the reasons Bron and John stated.

• Pete in that case for issue number 7 -> we’re really just talking about whether Trace itself should allow fields other than Return-Path and Received, and whether it should allow Return-Path by itself.

• Alexey: think we do have consensus on extensibility, but ABNF is still in question.

• Pete: the way syntax in 3.6 appears, if you have resent or other optional fields, you need to have a Trace, which may not be what you want.

• Alexey: maybe the list will have people with specific experience who can help.

Slide 11 - issues for 5321 (should registry be mentioned - see next slide)

• Alexey: John, do you want to say things about trace header fields in 5321bis?
• John: will need to think, but snap reaction is that if the trace header fields are going to be shifted in 5322, then we need to define the registry in 5321 rather than 5322. But that’s just instinct now.
• Alexey: so do you prefer for the registry to be definied in 5321 or applicability statement?
• John: instinct is define in 5321 and mention in 5322.

Slide 12 - issue 8, registry of header fields to add during and after submission.

• Alexey: new subregistry would show where the header fields could be added, and when.
• Original proposal said that header fields in this registry MUST also be in the header fields registry. Current proposal is to have a SHOULD. Various implementations have their own header fields.
• Procedure: expert review? Can the experts say “no” to proprietary field? What about First Come First Serve?
• What if someone registers something already defined in an RFC and registers it incorrectly?
• John Levine: think that expert review makes sense, so long as we know that the expert’s job is to weed out stuff that seems obviously wrong (e.g. Content-Type!).
  • Wonder if this makes sense as columns on the main table instead?
• Alexey will propose latest text on mailing list and hopefully we can close this ticket.
• Barry: strongly suggest when you propose this text that you include instructions to expert reviewer along the lines of what John said. Expert “sanity check” only.

EHLO (ticket 19)

• Proposal - server MAY verify the ehlo name matches the IP address.
• Alexey: question, do we want to clarify what “address record” means?
• Barry: as a participant, not AD - like change that’s proposed. Would also add the word “alone” to the second sentance of that paragraph. Don’t need to clarify address record, it’s OK the way it is.
• John Klensin: don’t see necessity for doing this, don’t think it makes it worse, makes it better. Address record was deliberately left vauge in 5321, concerned ipv6 might implode and wind up with something else. Value in keeping that flexibility.
• Alexey: gut reaction was “avoid people doing something silly”, e.g. MX records?
• Bron: this is really saying to clients “this is what SMTP servers are likely to do”.

Slide 14 - the “must not refuse to accept the message on that basis”

• Some servers have an option whether to check this or not.
• Suggestion: change MUST NOT to SHOULD NOT or even MAY.
• Barry: agree with “SHOULD NOT”, and would add either the word “solely” or “alone” to say they can use it as part of a decision, but it shouldn’t be the only basis.
• Todd Herr: disagree, it should a sole basis for the decision, based on anti-abuse practice. If they HELO as mail.google.com and are coming from other space, that’s enough to know. You clearly have a client that’s either misconfigured or abusive. That’s enough.
• Barry: so you’re advocating for SHOULD NOT?
• Bron: propose removing this entirely - we don’t need any text around this at all. The server will do what it will do.
• John Klensin: if we’re just saying “the server may look at this, and may do what they’re going to do” then agree with just removing the paragraph.
• Alexey: are you happy with some text in the applicability statement about this?
• John K: absolutely.
• Pete: just wanted to point out that Todd’s example isn’t a problem with the original intent - (simply if the IP address and name in the DNS don’t match, that alone isn’t sufficient). Todd’s example was “they didn’t match and I have additional information about the name or IP set”. Think it’s perfectly reasonable to pull and put into applicability statement.
• Pete: but putting into AS and doing a fuller description is fine.
• Todd: must have mis-stated something. It could have been anything.
• Harald Alvestrand: please don’t use mail.google.com as an example. If you look at this, there’s nothing like a simple reverse lookup. The name they use in EHLO is something that maps back, because it spends significant time fighting with spam systems.
  • Tossing solely because of mismatch here is inviting issues. So no, don’t remove - keep it at least as SHOULD NOT.
• Dave Crocker: Peter Koch said “this confused protocol and policy”. The reality is that email operations has become complicated and varies over time. This is the realm of things that change over time, and aren’t central to the protocol.
  • Considering taking actions like removing something like this that changes over time is good.
• Alexey: so remove the text? Dave: yes, this is a philosophy reason, and it’s good to look at for all our considerations.
• Bron: likewise - we should separate policy from the protocol, and not have a “SHOULD NOT” that we know most people won’t obey.
• Alexey: I will do a strawman for what to say in the A/S draft.

Slide 15 - IP address literals in EHLO

Sounds like we need at minimum some text in applicability statement about this.

• Pete: will try to channel Keith since he’s not here. Syntactically it’s reasonable to leave it, even if by policy you might reject.
  • IoT cases, etc - you may want to allow by policy in spaces where you can’t use domain names in a reasonable way.
  • Protocol wise, OK to leave in - policy wise, OK to reject.
• John Klensin: taking it further, seeing many SMTP clients still operating today using IP addresses because they are using SMTP to send messages to local servers and don’t have access to DNS or don’t use DNS.
  • Showing up in LANs and isolated networks. Using IP addresses because they don’t have any choice
  • Retained in 2821 because of this same discussion.
  • AS saying “IP addresses on public internet are bad” but don’t change this.

Slide 16: quoted strings (ticket 21)

• John K: have seen implementations in which all the various crazy quotings are considered legit or not. Are the same, or not.

• There’s an argument for figuring out what we think we meant and narrowing it down, even if there are legacy things out there.

• Argument for clarity - may want to deprecate some of these combinations.

• Have history of operating systems having their own quoting rules too, then it gets very difficult.

• Pete via Jabber - hope it doesn’t change syntax in 5322.

• John: we may need to define some things as being equal or not equal - define the behaviour more closely.

• Alexey: am sure there are lots of bugs in this area!

• John: first issue (what’s valid) is important. Maybe move second issue (canonicalize and compare) into a second issue.

ACTION: separate issue for canonicalize and compare from ticket 21

Slide 18 - empty quoted strings (issue 35)

Wanted feedback on applicability statement -> where strings can be empty vs non-empty.

• John K: need to make sure 5322 doesn’t prohibit anything that 5321 allows.

• Alexey: agrees.

• Pete: given how much 5322 allows! Unlikely to be an issue.

• Alexey: think the obvious place that’s OK is Display-Name. But name of group?

  • should we create some test messages and see?
  • Pete: sure
  • John K: seems to me that a lot of this is about a warning that some of these constructions are not likely to work in the general case. If you created mailboxes that look like this, then it’s not going to work.
  • “Security through obscurity” for anti-spam, not much we can say about that.
  • Pete is correct that 5322 permits just about anything, and maybe that’s OK.

• Bron: as an end user, what I want is a clear “here’s what I can expect others to implement if they’re being reasonable” so I can complain about them if a service doesnt’ support them - e.g. “+ in local part is valid and if your web form doesn’t accept it you are wrong”.

• Alexey: this would be something for applicability statement;

• Bron: yep - what I want is something actionable.

Slide 20 - behaviour on server closing connection.

• John: not sure we need this, but it’s OK.
• Alexey: have a slide -> server says “421 shutting down, client does non-blocking write, polls”.
  • Client will have closing message from the server in its TCP buffer after sending command.
  • Idea is to say “it’s a good idea do this”.
  • Text suggested “client will recieve this after sending a command”.
• Barry: with John, don’t think text needs to be fixed here. Applicability statement should say how clients should handle this, but what’s in 5321 now is OK.
• Alexey: thinks current text can be read as prohibiting.
• Barry: I don’t read it as that - if people are reading it as that, maybe we DO need to tweak the text.
• John K: I also don’t read the current text as prohibiting this. It’s an operational detail on how client interacts with the TCP interface. If others are reading it as prohibition then it should be patched.
• Barry: but don’t think that proposed text patches it.

Slide 21 - more on clients - issue 46

• Similar to previous slide.
• Barry: strikes me as telling people how to deal with transport layer issues. Not sure that 5321 is the right place to do that.
• Bron: agree with Barry, but we should be very clear when retrying is right, when backing off is right, when assuming delivery if right.
• John: agree, we’re messing around in implementation details and don’t want to go there if we can avoid it.
• Alexey will take to the mailing list.
• John: would be hesitant about getting far into this in applicability statement either - they’re issues in local system model of transport interactions.
• Alexey: trying to figure out whether documenting this makes a difference. Hypothetical case, where server returns completely different 5xx code here.
• John K: in the past we weren’t on TCP, in future maybe not again - we shouldn’t make assumptions!

Slide 24 - ticket 5 - deprecate or remove 552 as 452.

• Todd Herr: Asked many providers, and most are using 552 as a permanent reject.
• Alexey: clarify - as far as we know, no client is following this recommendation.
• Todd: haven’t heard from anybody who says they are treating it as a temporary failure. Haven’t surveyed everyone of course.
• John K: posted to the list sayiung that not only is this not useful, it may be wrong. So another reason to take it out.
• Alexey: suggest mini WGLC on this issue, proposal to drop this sentence.

Slide 25 - review timeout specifications.

• Todd: haven’t heard back from providers. These limits are far longer than they need to be!
• John K: flag rather than opinion -> reach out to not only things that we consider ordinary clients, but also DTN people.
  • If making decision on what implementations might do, we want to make sure we don’t mess up others.
  • Alexey: do you know who to contact?
  • John: not any more, but know who to ask! There’s still a DTN working group, that’s a logical place to start.
• Bron: what’s the reason to reduce them? If high disk load, it’s good to have the time - and high volume servers don’t pay much cost for an ongoing TCP connection.
• John K: I think from combination of Bron/Todd comments - probably shouldn’t be changing anything in 5321 about this, just something in the AS about the various tradeoffs.
• Todd: they’re only SHOULD anyway. Perhaps AS is a better place to discuss the entire topic. No discussion on list or in the ticket to guide us.
• Bron: since there are no complaints about this, maybe we shouldn’t be changing it at all.

No more issues!

• Alexey - didn’t expect us to use so much time, but we’re nearly at 2 hours! Any other questions?

See you on the mailing list.