Bluesheets: 7 August 2024 Virtual MOQ Interim

1. Martin Duke, Google
2. Alan Frindell, Meta
3. Daniel Fay, Meta
4. Mathis Engelbart, TUM
5. Will Law, Akamai
6. Luke Curley, Discord
7. Ian Swett, Google
8. Mo Zanaty, Cisco
9. Erik Herz, Vivoh
10. Dinesh Adhithya, Zoho
11. Victor Vasiliev, Google
12. Manish, Dyte
13. Mike English, id3as

Minutes

Hybrid interim

• 1-3 Oct
• Move from NYC to MA?
• Akamai offered to host
• Martin will send last note to list before final decision

MoQT Issues

(PR#491)

• Ian: Start with PR about MAX_SUBSCRIBE_ID (PR#491)
• Is this the right direction?
• Note that if we had MAX_SUBSCRIBE_ID and MAX_ANNOUNCE_ID we
  might get rid of role
• Alan: Not sure if it is required to limit announces
• Will: Could get rid of role anyway. Just because you need protection
  from bad endpoints anyway.
• Martin: I could drop announces
• Ian: No default limit, initial one communicated in handshake. If you
  exceed it, connection gets closed. New MAX_SUBSCRIBE_ID parameter
  in setup handshake. and new frame to udpate. Similar to QUIC.
• Luke: We don't have subscribe state machine. Do you have to
  unsubsribe to stop using flow control. How to get more credit.
• Ian: Implementation detail, similar to QUIC. If you never increase
  id, you never get more credit.
• Will: ABsolute count right?
• Ian: Yes, closing doesn't change anything because it's the max ID.
• Martin: That simplifies state machine.
• Will: Mechanism for negotiating more?
• Ian: Expect that if you close the peer wuold provide more when you
  close a subscription.
• Will: Can you explicitly ask for it?
• Ian: No. Allows to open more based on resource situation.
• Mo: Not sure mirroring QUIC is good. In our context, relays don't
  have knowledge about content. Default to 0. How would subscriber
  know how the relay is going to do flow control since it is dependent
  on the implementation of the specific relay.
• Martin: Subscriber doesn't infer anything, sender decides based on
  his resources.
• Mo: Sender is relay here, relay has no idea of dynamics of traffic
  because it doesn't know the actual application. That's different
  from QUIC where the endpoints are the application and know what's
  going to happen. Imagine relay to relay and may need to allow many
  more subscribes compared to endpoints.
• Will: Based on the resources of the relay you can request more.
• Mo: No, there is no request message for more credits.
• Will: Subscribe would ask for more
• Mo: No seems to be protocol error if subscribe for too much.
• Martin: If max is 5 and client subscribes for 4, than relay should
  provide more credit.
• Mo: But that's dependent on relay implementation
• Martin: In QUIC there's a blocked message which we could also mirror
  if we think we need it. But I think relays could do the right thing
  here. Would be fine with having blocked messages.
• Mo: Biggest issue is that default depends on who the relay is
  talking to. small limit is fine for end subscriber but not for
  another relay.
• Martin: What about basing it on auth in setup? To identify relays
  and end subscribers.
• Alan: Do you have an alternative proposal?
• Mo: Having a request or something like that would make sense so you
  don't have to wait for implementation decision to provide more
  credit.
• Victor: This is improvement over current situation which is like if
  i have a limit, I will just send subcribe error if client subscribes
  to too many. Alternative is like not have any limit which is a bad
  idea. This PR just makes perfect sense. Practical differences as to
  why flow control in MoQT is different from flow control in HTTP 3.
  HTTP 3 try to send requests and reply comes back based on default
  100 requests limit. Expectation is that those will be immediately
  processed. In MoQ there's a good reason to have 1000 active
  subscribes at a single time. I believe we should document
  considerations about limits but merge this.
• Will: Why line 1702 says you should close the whole session instead
  of just deny the one subscribe that exceeds the limit.
• Ian: Simplifies debugging.
• Will: You get a subscribe error.
• Victor: But it is a protocol violation
• Ian: Closing with error code makes it much easier to find
  implementation error. Experienced recently in QUIC implemenation
• Will: Will make it difficult to build fault tolerant systems. There
  might be race conditions leading to accidentally over subscribing.
• Ian: Max increases when it sends it and receiver must receive it
  before using credit.
• Martin: As chair, acknowledge Mo's concerns and propose merging it
  and [...]
• Mo: Why do we need any machinery at all. Maybe signal when limit
  reached and client can try again later. don't need to mimic QUIC
  here.
• Will: WebTransport invented same kind of message to signal a hint.
  MoQ could do something similar.
• Luke: QUIC flow control works well once implemented even though it
  sounds complicated on paper. Waiting and trying again is bad. Please
  write down alternative flow control mechanisms. I think we will end
  up with a similar mechanism. I like it to be fatal error as well.
• Martin: Alternative proposal is to not have an explicit message and
  only error if limit is exceeded. Do people besides Mo prefer this?
  No one raising their hand. Mo was that correct?
• Mo: Yes. Add subscribe error: "flow control"
• Alan: Relay can still do that but tell the peer when it is going to
  happen in advance. Every relay has some limit and you set it in
  advance vs. blowing up after it reached the limit.
• Will: It doesn't have to blow up
• Luke: But the limit is still there
• Will: What if [...]
• Victor: I hear suggestions for rejecting subscribes with error and I
  believe there's we [...]. I don't see reason not to ship this PR,
  works well in QUIC. It's over engineered because control messages
  are oredered which is not even required for this.
• Alan: Bad actor just says subscribe for ever and you have rate
  limiter to close vs. having defined behavior and remove bad
  subsribers. Also solves subscribe/unsubscribe issue. You can reuse
  ID 1.
• Will: Ack
• Mo: We should have a talk about protection mechanisms overall. Not
  convinced this adds protection since we have a single control
  channel. Bad actor could send lots of control messages and one would
  have to react to all.
• Ian: Going back to rapid resets. Sending small messages could make
  the peer do a lot of work.
• Victor: Sending duplicate subscribe ID should be fatal error. We can
  make them sequential.
• Alan: Mos point is we should take a holistic approach and we only
  solve one with this PR.
• Ian: Yes, I think there's more work to be done. This one is most
  obvious only.
• Martin: Alan, Show of hands?
• Martin: Separate issue to have client hints.

• Martin: Do you prefer MAX_SUBSCRIBE_ID type solution. Raise your
  hand

  • 7 people

• Martin: Rais hand if you prefer subscribe error based mechanism

  • 2 people

• Martin: Who would like more discussion?

  • 0

• Martin: Alan and I talk about it offline and go to list?

• Alan: Yes, few opinionated people are missing today. 7 to 2 seems
  like rough consensus
• Mo: Not completely opposed but think holistic approach seems better.
  If this solves it hollistically, I could be convinced, but not yet.
• Martin: I can summarize questions on list after meeting.
• Alan: Anyone opposed that we also need a broader analysis of DoS
  threads on control messages? We have a lot of issues and should do
  small steps.
• Mo: On list please make it clear what the semantic change of
  subscribe ID is due to this PR, since it now needs to be sequential.

• Martin: Will try but please chime in when I fail to point it out on
  the list.

• Martin: Please sig bluesheets.

##

• Ian: We don't have much text about how streams are used. Are there
  normative statements that we can already agree on? First one: If
  subscribption is cancelled or expired, SHOULD publisher reset open
  streams?
• Victor: Was trying to write fetch proposal. Should split messages
  into control and data messages. Wrote PR for that and we should
  renumber stream types. Implemetnation can currently handle any
  message anywhere but that doesn't make sense.
• Ian: That's PR 503
• Victor: Yes.
• Victor: Might merge text for streams and datagrams.
• Alan: Editorial?
• Victor: Almost. Things that were allowed are not anymore now but
  does not matter.
• Will: Control stream is bidirectional right and data on
  unidirectinoal streams. What about a client opening a second
  bidirectional stream.
• Ian: Currently illegal in spec.
• Mo: I support split and would go further and split the spaces
  themselves. So there's no list of all messages.
• Victor: Splits namesapces but doesn't renumber them
• Ian: agree
• Luke: Support it and renumber such that stream header group is only
  one byte varint.
• Martin: I like having separate spaces. Will be huge pain to
  implement properly but would have been better in the first place.
• Ian: Ok seems like all think this is going in right direction.
• Alan: Just merge today.

• Ian: Will do

• Alan: 10 minutes left

• Ian: Let's look at ALPN PR

• Ian: WebTransport has ALPN feature. Discussed long time ago so we
  couldn't rely on it. Intent here is to use ALPN for both instead of
  existing version in SETUP. ALlows to remove some text.
• Luke: Is this deployed in chrome?
• Victor: Not yet
• Luke: Can we wait until it does?
• Ian: Practical concern
• Victor: Shipping is blocked on bikeshed of sub-protocol vs. protocol
• Alan: Can IETF figure it out?
• Mo: Protocol has consensus
• Martin: Group has consensus on using ALPN and will after Victor gets
  it done.
• Luke: As long as we can offer multiple versions. Could then remove
  setup round trip
• Ian: Setup has other things like auth.

• Luke: But version is the requirement you need before sending other
  messages.

• Alan: Other open PRs? Yes, 497

• Ian: Mostly donon

[Missed some part about 497 due to hedgedoc issue]

• Will: Want to do some special things because it is talking to end
  subscriber.
• Mo: Intended to capture that content may be e2e encrypted There
  should be something informative saying that content may be
  encrypted.
• Alan: Is there but is lowercase may.
• Mo: About things that are not just relays but not full endpoints. We
  have example of a conferencing server that is not a relay and not an
  endpoint which we expect to do forwarding but not have knowledge of
  content of messages. We should decide if we need a name for that.
• Ian: Add text once we need to add features to support those things
• Martin: +1 to Mo, Should merge this.

• Ian: Will merge

• Martin: Anything in two minutes?

• Alan: Can talk about subscribe namespace. It changes the definition
  of tracknamespace. Currently single string of bytes, change to
  sequence of 1 to 32 fields of bytes. So becomes a tuple. That
  enables subscribe_namespace message where you use a tuple subset to
  subscribe to announce and unannounce messages matching that prefix
  of tuples.
• Martin: Would those be required to get announces.
• Alan: No, announce is not required. If you have a case with a
  million tracks publisher and only some people should receive
  announces about some of them.
• Mo: I think this goes right direction. Namespace and name was
  initially monolithic. If we actually have tuples, we could just make
  the whole thing a tuple and don’t need trackname anymore.
• Alan: Will also raised the point. We do have distinction to match
  for announces vs match full names. This solves one problem, could go
  further later.