RASP interim meeting (IETF116-117)
agenda-interim-2023-rasprg-01-rasprg-01-03

Preliminary Agenda (interim-2023-rasprg-012023-05-16)

- List of Research Questions
- RG Github Repository (https://github.com/IRTF-RASPRG):
        - Readme listing existing projects/code/tools
        - Tools: eg., reviewer recommender [5] bigbang [15]
        - Code
- Ethics and Privacy
- Exploration for possible joint work
- Next meeting

Preliminary list of Research Questions

# Dynamics
        - What are current 'trending' topics per SDO / area / WG
        - What are hotly debated drafts, email threads, words
        - Who are the main people / organizations involved per drafts, email
        threads, words - How long do people stay engaged in SDO / area / WG
        [1][8] - Time from first to last engagement with mailinglist - Trends
        in their participation - Do people join gradually or in cohorts? - How
        does participation in the SDO / area / WG evolve [12][8][10] - Per
        individual, affiliation, nationality, gender - How active is an SDO /
        area/ WG [2][3][8][10] - How many emails are sent on the list(s) per
        day? - Who are the authors that are sending most emails? - How is the
        distribution of emails / senders? - Which individuals are most dominant
        in the discussion(s)? - Which organizations are most dominant in the
        discussion(s)? - What is the sentiment of discussions per SDO / area /
        WG [7] - Meeting attendance [4] - What are the dynamics between IETF
        hierarchical structure and communications? [11][14] - How does the
        languaje of different IETF participants differ? [11] - How are are the
        dynamics of communication evolving in the IETF? [8] - How has the
        process of publication of drafts/RFCs evolved in the IETF? [10] - How
        does influence in the IETF correspond/relate to effectivity per person
        / organization [13] - How is the distribution in nationality in
        mailinglist participation? - How do collaborative relationships form
        between mailing list participants over time? [9]

# Discovery
        - How can we map particular drafts and discussions to a particular
        technology? - How can it be made easier for non-SDO experts to find
        discussions and documents on particular technologies - How can a
        non-SDO expert / new author, find who the main people discussing (on
        mailinglists) and writing (drafts and RFCs) about a particular
        technology? How do they find out who is the relevant AD or WG chairs? -
        How to do entity recognition to create knowledge graphs of complex SDO
        mailinglists and documents data

# Data: which data we would like to have but we don't?
        - ground-truth data on protocol deployment [6]
        - ground-truth data on reviewer recommendations [5]
        - agreement/disagreement in mail-lists
        - annotated mailinglists for ML training

[1]
https://github.com/datactive/bigbang/blob/main/examples/activity/Cohort%20Visualization.ipynb
[2]
https://github.com/datactive/bigbang/blob/main/examples/activity/Cohort%20Visualization.ipynb
[3]
https://github.com/datactive/bigbang/blob/main/examples/activity/Analyze%20Senders.ipynb
[4]
https://github.com/datactive/bigbang/blob/main/examples/activity/Analyze%20Senders.ipynb
[5] https://sodestream.github.io/recommender.html [6]
https://sodestream.github.io/deployment-survey.html [7]
https://convokit.cornell.edu/ [8]
https://sodestream.github.io/paper-the-web-we-weave-untangling-the-social-graph-of-the-ietf.html
[9] 
https://github.com/datactive/bigbang/blob/main/examples/experimental_notebooks/Collaboration%20Robustness.ipynb
[10]
https://sodestream.github.io/paper-characterising-the-ietf-through-the-lens-of-rfc-deployment.html
[11] Accepted at ACL, publication pending [12]
https://github.com/datactive/bigbang/blob/main/examples/experimental_notebooks/IETF%20Participants.ipynb
[13]
https://github.com/datactive/bigbang/blob/main/examples/experimental_notebooks/IETF%20Participants.ipynb
[14] Undersubmission (ICWSM) [15] https://github.com/datactive/bigbang