Presentation by Stefanos
Stefanos introduces himself.
There is a trend of networks getting bigger, entering hyperscale data,
and devices are challenged
Multiple neural networks reside on device, per app or across apps, and
need privacy preserving techniques for this
Transformers: parallelize very well for training, support multiple
modalities (text,speech etc). They can scale to very large sizes. The
decoder is aggressive, generates next token as soon as it can. More
intro on the slides about the basics on transformers. Usage -
pre-training on very large corpora such as the web and then fine tuning
for specific uses.
Modes - application level, integration with the browser. Web level, the
system is the website
Challenges include performance - download, memory to secondary memory
etc. Paper and code are both freely available.
Why should we deploy in device? Users can control how they use these
models. This is also important for privacy and personalization (to their
own tasks). In device also more sustainable, smaller, suited to task
Trend of LLMs going to smaller ones - many billion parameters versus
maybe a billion.
Energy measurement is specific to when the model is on the device.
Gathering all the process of measuring energy etc. is important. Mention
of Internet Thermometer (see slide).
Brevity: download model, parameterizing, research with specific tools to
understand the impact on the accuracy of the model tackling the task.
Automation allows the device code to be measured.
Experiments across a range of parameter numbers (1B to 10sB)
Platform is capable of testing multiple devices, models, and operations
(Conversation or fixed input tokens). Some results showing smaller
models behave better. Numbers for GPU versus CPU in paper - GPU more
efficient, of course
KPIs: Discharge and performnance vs model characteristics
Modern phones have large power draws now
Measured performance when running sustained workloads. Could verify
these measurements via programming API. Could get iPhones to overheat
with 3B parameters.
Measurement of impact of quantization - different levels and methods and
parameter models have different impact.
A device oculd be collocated with a TV or such, and they looked at
multi-device modes for the models. They have a lot of tools for others
to use for further studies of mobile with these models.
In the future, the rise of smaller LMs for specific behaviors (and
routing across different models) and multi-modal models. Will need to
explore support for dynamic models. Look into model-to-model
communications...boost efficiency and energy with this. Backbone of the
model could be middleware with a layer of small more specialized models
over it as a future deployment approach.
Question from Shivan - evaluating open source models. Any idea what the
performance will look like for models shipped by the manufacturers? Not
so known. If it will be a middleware, it will need to have ways for the
applications to interact. Don't expect more than 10B parameters at
Google. Apple models will only be supported on most-memory latest
phones. First party applications first, and third party applications
will follow.
Question from Jeff ? - model eval of quantization was the only thing
that addressed Jeff's question [chairs, please fill in more on this]
Allison and Nick asked some online questions about practical privacy of
these on-device models, and Shivan will relay them to the speaker
"Ariadne: a Privacy-Preserving Network Layer Protocol" Antoine
Fressancourt (25 mins)
This will present the protocol and why its design - anonymity goal.
Source of packet cannot be associated at destination. Also flow
unlinkability.
Basis of the technology is onion routing (specifically as described
in 2020 paper by Kuhn et al). Ensure the formal properties of the
onion routing are in the protocol. Layer unlinkability, can't tell
two onions are related. Tail indistinguishability. And Onion
correctness - attacker hasn't modified payload. This is very
important because many attacks have been based on manipulation of
the payload.
The researchers characterized recent schemes versus these
properties. None achieve the properties.
Ariadne uses a source routing approach. Use sequential encryption -
at every node, the packet is re-encrypted. Avoid Public Key as much
as possible for performance reasons (and PK was also presented as an
issue for operating at network layer in the review of past work)
Primitives of the protocol:
Anonymous key reference (important for avoiding many attacks that
Tor is subject to). Source and intermediate node agree on a key
pattern, store in a dictionary. Encryption of this pattern changes
with every packet. It's a moving reference. Loops addressed by this.
Routing element shuffling - Sphinx did this, but in a
resource-intensive way. Instead, a routing element vector. When a
node processes a packet, it can decrypt slots in the routing element
vector.
Protocol operations given the primitives - the paper gives a
detailed specification of this.
Link to paper on Ariadne: https://arxiv.org/abs/2406.03187
Key idea and results: the routing vector in the common header is
created with the source route. Each node in the source route finds
its randomized routing slot. This routing vector turned out to leak
some privacy information, which is why the comparison table shows
only partial achievement of the privacy properties if the routing
vector is used.
They wanted to bring it to IRTF because they are trying out
implementing Ariadne as an IPv6 extension header (Rust
implementation in progress). The proofs of the protocol are there,
so this is something that could be built on and developed.
Question from Jonathan Hoyland - proofs surprise him because the
anonymity trilemma says it's impossible to prevent correlation
attacks without adding some features missing here. Answer: it's a
formal description rather than a proof
Jonathan: what if the attacker is the first router and the last
router, and does a simple counting attack? Answer: This protocol has
to be used with a traffic padding or other algorithms to prvent.
Getting rid of metadata gets rid of correlation attacks based on the
structure of the packet. We think that Ariadne is compatible with
use of such approaches.
[End]