Performance Evaluation of the Routing Protocol for Low-Power and Lossy Networks (RPL)
draft-tripathi-roll-rpl-simulation-08

In general, it is extremely good that we have documents like this. Thank you for doing this work! I have not dug in deep enough to understand the details and whether all the measurements make sense, but I think we should encourage both this document and other similar ones to be published. Thomas Clausen has been showing some measurement results (incidentally with somewhat different conclusions) and I think we should ask him to publish his results also as a similar RFC. As well as others who have data.

I also believe that the RFC Editor stream is a good place to put this type of publications. 

The one high-level comment that I have is that we should be very careful about conclusions from these types of measurements in the resulting RFCs. Its a natural tendency for people to make generic conclusions when most of the results apply to a specific scenario that they measured, not everything. You also have to be very careful in making system-level applicability statements based on single-packet level performance (e.g., 99% packet reliability => system reliability is good -- I don't know if that's the case or not. We'd have to do the math to really understand if 99% is good or bad). I think the authors, the RFC Editor, and the IESG should all check to make sure there are no too wide ranging conclusions in documents like this.

Finally, I did have some reactions to this specific document. Please consider them as improvement suggestions. It is of particular importance that we pay attention to various conclusions about the good or bad operation of a given IETF technology, as noted above.

----

Section 3 seems to say that nodes send 80% of their traffic to root, but I can't find a statement that says something about the root sending traffic back to the nodes. I think in most realistic cases there will also be acknowledgments. Am I just missing the text that explains the return traffic, or did you not simulate the return traffic at all?

Section 4.3 states that as 90% of nodes are capable of operating under 10 routing table entries, then the protocol is capable of working in constrained nodes. I do not think this can be inferred. If 10% of my nodes have to store significantly larger routing tables (and those nodes are not the root or some other conveniently a priori known device), its not going to help. I still have to deploy more memory to every node.

Section 4.4 should make a similar conclusion about the satisfaction of the latency requirements. Again, if 99% of communications are timely, it may not help satisfy requirements of industrial installations, for instance. The requirements are usually stated as, say, no communication failures within a period of time (say, a month or even a year) that could not be handled with the used reliability mechanisms. These mechanisms could include, for instance, repeated transmissions, acknowledgments, or transmissions to duplicate destinations. If a single packet reliability is 99%, what's the likelihood of the a communications problem in an N device system with communication event frequency F and, say, triple transmissions to ensure reliability? You need to do the math to provide an evaluation of whether 99% is good or bad.

The PDF version makes for interesting reading. Unfortunately, the ASCII version is considerably less valuable because so much of the text makes reference to graphical aspects that are available only in the PDF version.

- Luckily, I read the PDF. I agree with the comments that
noting its existence in the ascii would be good.

- Documents such as this are always much more valuable
(IMO) if the underlying simulation code and data can be
made available for others. I don't know if that's 
possible here or has been done, but if so, a link 
from which those could be downloaded would be a great
addition.

- Figure 1 has two colours for links but doesn't say
why.

- I didn't get the spike to the right in figure 9 - it'd
be no harm to say why that's there for (I guess) node
38. The scaling on that figure could also be better, the
control traffic is hardly visible - a logarithmic scale
would be clearer.

- 1st para of 5.2 is repeated. 
- 1st para of 5.2 is repeated. 

Picking up from Peter's point I also started to review text version and realized that there must be a pdf version. Many readers will not know this and will not know how to get the pdf version. They would be better served if a note directing them to read the pdf version was provided at the beginning of the document. Indeed the RFC editor may be spared considerable work by making the text version of the document simply the abstract and a pointer to the pdf.

I note that there is no security section to the document. Is this not also mandatory in the independent stream?