Last Call Review of draft-koster-rep-06
review-koster-rep-06-artart-lc-herr-2022-03-03-00

Comments from Todd Herr (todd.herr@valimail.com)

* I found section 2.2.3, Special Characters, confusing. Specifically, the
  first table in that section includes an entry for the "$" character,
  and the middle column in that row contains text that says "A URI MUST
  end with a $."

  However, there is at least one URI in the table in section 2.2.2 that
  doesn't end with a $, as well as one in the second table in section 2.2.3
  that doesn't end with a $, as well as other URIs scattered throughout the
  document that don't end with a $.

  Can this be clarified, please?

* In section 2.3.1.3, Unavailable status, the crawler is given a choice
  of accessing any resources on the server or using a cached version of
  robots.txt. 

  I'm having trouble envisioning a crawler that wouldn't take advantage 
  of this situation to crawl the entire site, so I'm wondering if the 
  second MAY clause is even worth stating?

* Section 2.6 has a subject/verb disagreement:

  "Listing URIs in the robots.txt file exposes the URI publicly and
   thus making the URIs discoverable."

  In that sentence, "making" should be replaced by "makes".