Date: 2024-11-08 09:30 (GMT)
Chairs: Alvaro Retana, Ignacio Castro
Room: Wicklow Hall 1
Notetakers: Ryo Yanagida, Laurent Ciavaglia
Speaker: Chairs
Speaker: Justus Baron
Stephane Bortzmeyer: Question about methodology; meeting at Buenos Aires
were less well atte3nded due to availability of flights etc. e.g.
Singapore was better attended for many due to better availability of the
flights. Maybe this is hard, but maybe this can be taken into the
account.
Justus: agrees. Checked in 3GPP though he did not observe any effect and
believes that opportunity cost is likely to matter more than price.
Stephane: Price was a big factor here
Justus: while distance is quite a strong factor but I agree price and
other factors can impact
Andrew Campling: Do you have any plans to look at more recent ones, as
the older meetings had a quite primitive remote capabilities
Justus: I agree that remote meeting capabilities have impact. We have
not quite done that yet as too many events have happened simultaneously
in the recent years that impacts the attendance e.g. war/conflict/covid
etc.
Jaime Jimenez: Distance is a good way to approximate cost. It might be
good to see if there is a way to pre-normalise the data.
Justus: agrees
Carolina Caeiro: We are discussing at IETF re: venue selection and
gender diversity. Your research would be useful to help decide. I had a
quesion re: nomcom, female participants were not impacting the
leadership gener bias. Can you comment more on that? I was surprised
that it didn't have much of an impact. Maybe this is to do with the 2019
cutoff. Also comment around gender participation — perhaps the domestic
responsibility have something to do with the women's participation to
these meetings.
One of the previous questions talked about flights re: Buenos Aires, but
even for the locals, the registration costs vs average earnings has a
huge impact.
Justus: re NOMCOM, the cutoff wasn't the same with others as this is
from different data. The discussion around 'why women aren't in the
leadership' have been interpreted as the 'norm change'.
It's not about the selector (people selecting leaders) but the
participation of female overall seemed to have had more of an effect.
Ignacio: Due to time please be brief
Jonathan Hoyland: 2 suggestions re; cost function for travel. The real
cost seems to be timezone difference and visas. Accounting for visa
difficulty would probably be important. Also could you comment on
whether the onsite childcare at IETF helped. Have you also looked at
changes of gender, where they started attending as one and shifted. Has
this affected the population?
Justus: Timezone is probably good to look at. We did manage to look at
visa req. for 3GPP, did not seem to have a huge impact? wrt gender, we
don't have enough data and it is quite difficult and quite delicate.
Colin Perkins: I suspect the childcare situation will complicate the
analysis. Wrt women's attendance, did you see the shift to on-site to
remote or on-site to not attending at all. The IETF do have more
granular data from registration. Some are public in the datatracker. It
might be interesting to talk to LLC or IESG to see if you can obtein
this.
Speaker: Ryo Yanagida
Olaf Kolkman: intuition on why decreasing number of authors at constant
level of email engagement and communication engagement, larger mesh of
communicaiton? Any idea why these things are tied together?
Ryo:
Olaf: I think more people talking together, takes more time to reach
consensus
Igancio: more emails, drafts, more time to converge. overhead of
producing at IETF has increased. Cosnider in new areas has an effect
Olaf: also quality of the work coming in
Will Earp: building new system is easier than changing existing stuff.
This may explain why it takes longer to publish.
authorship on the mailing list...
Ryo:
Laurent Ciavaglia: on authors' interactions: are you considering
individuals only or also per affiliation, any correlation?
Have you observed patterns related to the topic or domain, e.g.
security which is more transverse in the number of people individuals
communicate with, and finally do you have data to obsreve differences
e.g. between IETF and IRTF?
Ryo: only individuals for now.
Ignacio: e.g. security: yes, observe it is connecting to more people
Speaker: Jaime Jiménez
Olaf: One of the things I do is to generate oncology. Do you think we
can dig into enough that we can see the correlations to policy imapcts?
Jaime: Not 100% sure but maybe
Ignacio: maybe with additional input to clarify this
Laurent: have you tried Google's LLM in private context? If your work is
applicable to IETF in general, that'll be good but perhaps there is a
way to take it to anoter context too.
Colin: Have you compared it to the human generated minutes? How much the
human take the creative liberties?
Jaime:
Colin: Is there any obvious way of comparing the accuracy of the
human/LLM generated minutes?
Ignacio: There is but it's quite problematic
Caroline: Have you expanded this to another relevant SDOs? How the
division of the labours working with other SDOs by looking at the
overlap topics
Jaime: I tried with 3GPP but it didn't quite work well due to format
etc. I encourage SDOs to avoid binary formats
Dhrov Dhody: I saw the paper and my WG's data there. It seemed to miss
final conclusions or action items, is there any way of improving this?
Jaime: For me, it worked quite well. But can look into it further.
Dhrov: there are some random BoFs
Jaime: there are hallucinations, they are further entrenched deeper as
more data is fed in, and I don't think there is a good way of removing
them.
Speaker: Kaliya Young
Question in the room about definition of "Pattern Language"
Kaliya: look at slides shared in previous meeting
Andrew Campling: Maybe you can come to a conclusion that it's amazing;
but did you come across anything that we can improve upon?
Kaliya: I see things that IETF is trying to do better, is getting
better. e.g. newbiew introduction. Maybe ISG could improve on...
The fact that you are asking this question is a positive aspect of the
IETF.
Rich Salz: Use of patterns is quite interesting; previous people looking
at IETF weren't particularly well-received. e.g. Corrine had quite a
negative view on it so there were a push back from the community. It is
useful to hear things we can improve upon. Now, 'external' criticism is
tricky and 'internal' criticism often works better.
Kaliya: When looking at IETF, I was intrigued by how it is organised.
Patterns are a way to look at the processes and gain understanding.
Things work well because a good pattern is kept and amplified.
Day: In many cases, positive pattern has a particular effects. Almost
every time, there are something that's not working well, another
positive pattern can be used to combat them. Key is to maybe
communicate/discuss with a common language.
Speakers: Chairs