P2P Research Group V. Gurbani, Ed.
Internet-Draft Bell Labs, Alcatel-Lucent
Intended status: Informational A. Basu
Expires: December 30, 2011 Tokai University and University of
Sussex
T. Schmidt
HAW Hamburg
S. Fleming
University of Sussex
M. Kolberg
University of Stirling
M. Waehlisch
link-lab & FU Berlin
June 28, 2011
Peer-to-peer simulation frameworks: a survey
draft-irtf-p2prg-simulation-survey-00
Abstract
Peer-to-peer (p2p) protocols, like all distributed protocols, are
complex, and therefore harder to debug and study in the wild. This
is more true of existing p2p protocols, where changing the behaviour
of the protocol --- however minor the change may be --- may result in
unknown manifestations on the dynamics of the swarm using that
protocol. In lieu of the unintended consequences of perturbing a
live swarm, researchers have resorted to simulation frameworks.
However, simulation results obtained from one simulator are often
hard to reproduce when using another simulation framework. This
document surveys existing simulator frameworks prevalent in
simulating p2p protocols today in order to quantify any assumptions
and characteristics inherent in the simulator. This, we hope, will
aid future researchers in choosing the right simulation framework for
their abstraction.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
Gurbani, et al. Expires December 30, 2011 [Page 1]
Internet-Draft P2P simulation survey June 2011
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 30, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Gurbani, et al. Expires December 30, 2011 [Page 2]
Internet-Draft P2P simulation survey June 2011
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Criteria for evaluating simulation frameworks . . . . . . . . 5
4. List of simulation frameworks . . . . . . . . . . . . . . . . . 5
5. Security Considerations . . . . . . . . . . . . . . . . . . . . 6
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6
7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 6
7.1. Normative References . . . . . . . . . . . . . . . . . . . 6
7.2. Informative References . . . . . . . . . . . . . . . . . . 6
Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 7
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 7
Gurbani, et al. Expires December 30, 2011 [Page 3]
Internet-Draft P2P simulation survey June 2011
1. Introduction
Peer-to-peer (p2p) protocols, like all distributed protocols, are
complex, and therefore harder to debug and study in the wild. This
is more true of existing p2p protocols, where changing the behaviour
of the protocol --- however minor the change may be --- may result in
unknown manifestations on the dynamics of the swarm using that
protocol.
Researchers contemplating on changing the behavior of an existing p2p
protocol have to be careful still, least they inadvertently do more
harm than good by introducing their changes. Furthermore, any
changes to an existing p2p protocol or a newly developed p2p protocol
must be tested and evaluated for validity and reproducibility by the
research community. While analytical and mathematical modeling
(fluid models, optimization and linear programming) is easily
validated, it is harder to validate empirical experiments due to the
dynamic nature of the networks, hosts, and interconnections between
them. Simulation frameworks are attractive since they provide a
controlled environment under which new behavior of p2p protocols can
be studied and quantified.
The good news is that there is a plethora of simulation frameworks
for p2p protocols available today, some of them are surveyed in
Naicken et al. [naicken]. However, that survey is dated and does not
include simulation frameworks like ns-3 [ns-3] and ProtoPeer
[protopeer] that have become available since the survey was
published.
The aim of this document is to update the state-of-art with respect
to p2p simulation frameworks available today. We will survey
simulator frameworks prevalent --- and actively used --- in
simulating p2p protocols today in order to quantify any assumptions
and characteristics inherent in the simulator. This, we hope, will
aid future researchers in choosing the right simulation framework for
their abstraction. This document can also serve as a guidance for
those researchers who are interested in developing P2P simulators
from scratch.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Gurbani, et al. Expires December 30, 2011 [Page 4]
Internet-Draft P2P simulation survey June 2011
3. Criteria for evaluating simulation frameworks
This is a non-exhaustive list of all criteria that we should evaluate
when surveying a simulation framework.
o Type of simulator: flow-level, message-level, or packet- level.
Advantages and disadvantages of each.
o Does the simulator specifically target p2p networks? Some like
ns-3 are general purpose simulators, but p2p models can be
constructed and evaluated over a general-purpose simulator.
o Level of documentation (APIs, wiki, etc.)
o Support for building models: script level, compiled language,
through a visualization editor, etc.
o System limitations imposed by the simulator framework, if any.
o Learning curves associated with the simulator framework.
o Support for trace-driven simulation (i.e., using live traces to
inject events in the simulator queue.
o Scalability of the simulator.
o Whether or not the simulator framework supports distributed
simulations synchronized on a common time source or event queue.
o Support for transitioning from a simulation environment to actual
system implementation (or, can the code developed for a simulator
be used with minimal or no modifications in a real host)? See
Galuba et al. [protopeer].
o Support for modeling link-level (delay, latency, loss, data rate)
and host-level characteristics (i.e., simulate both low-level
events and application PDUs).
o Support for interfacing real hosts that inject events into the
simulator.
o Support for collecting statistics and measurements from the
models.
o Visualization tools for creating topologies, viewing the
simulation in action, etc.
o Support for importing existing topologies (GT-ITM) and others.
o Support for exporting topologies in a standard graph markup
language.
o Should we focus on only academic and research simulators or
commercial simulators as well?
o ...
4. List of simulation frameworks
A list of simulation frameworks that we can survey appears below
(original list is in Naicken et al. [naicken], I have added a couple
more simulators). This is a rather exhaustive list, however, going
forward, we should focus on those frameworks that are: newer,
actively in use today, and those frameworks that are actively used
Gurbani, et al. Expires December 30, 2011 [Page 5]
Internet-Draft P2P simulation survey June 2011
today and have been surveyed before, but could stand to be looked at
again in light of hardware and software advances in the last few
years (multi-cores, parallel programming, etc.):
o ns-3 [ns-3].
o ProtoPeer [protopeer].
o GPS.
o PeerSim.
o P2PSim.
o OverSim.
o DHTSim.
o PlanetSim.
o VPDNS.
o Narses.
o Neurogrid.
o GnutellaSim.
o myNS --- we could probably drop this in favor of ns-3.
o Overlay Weaver.
o Query-cycle Sim.
o GTNetS [gtnets] --- seems to be abandoned.
o PeerfactSim [peerfactsim].
o ...
5. Security Considerations
This document does not introduce any new security considerations in
p2p protocols.
6. IANA Considerations
This document does not require any IANA considerations.
7. References
7.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
7.2. Informative References
[gtnets] "The Georgia Tech Network Simulator (GTNetS)",
http://www.ece.gatech.edu/research/labs/MANIACS/GTNetS/.
[naicken] Naicken, S., Basu, A., Livingston, B., and S. Rodhetbhai,
Gurbani, et al. Expires December 30, 2011 [Page 6]
Internet-Draft P2P simulation survey June 2011
"A Survey of Peer-to-Peer Network Simulators",
Proceedings of the Seventh Annual Postgraduate Symposium,
Liverpool, UK, 2006.
[ns-3] "The ns-3 network simulator", http://www.nsnam.org.
[peerfactsim]
"PeerfactSim.KOM - A Large Scale Simulation Framework for
Peer-to-Peer System",
http://peerfact.kom.e-technik.tu-darmstadt.de/.
[protopeer]
Galuba, W., Aberer, K., Despotovic, Z., and W. Kellerer,
"ProtoPeer: A p2p toolkit bridging the gap between
simulation and live deployment", Proceedings of
SIMUTools, Rome, Italy, 2009.
Appendix A. Acknowledgments
Authors' Addresses
Vijay K. Gurbani (editor)
Bell Labs, Alcatel-Lucent
1960 Lucent Lane, Rm 9C-533
Naperville, IL 60563
USA
Email: vkg@bell-labs.com
Anirban Basu
Tokai University and University of Sussex
Email: a.basu@sussex.ac.uk
Thomas Schmidt
HAW Hamburg
Email: schmidt@informatik.haw-hamburg.de
Gurbani, et al. Expires December 30, 2011 [Page 7]
Internet-Draft P2P simulation survey June 2011
Simon Fleming
University of Sussex
Email: S.Fleming@sussex.ac.uk
Mario Kolberg
University of Stirling
Email: mko@cs.stir.ac.uk
Matthias Waehlisch
link-lab & FU Berlin
Email: mw@link-lab.net
Gurbani, et al. Expires December 30, 2011 [Page 8]
