[Iccrg] Re: Procedure for obtaining ICCRG reviews

[Mark Allman]

> Before I use it for the CTCP draft (which is the only one
> on our table so far), 

Note, as far as I can tell CTCP has not been brought to the IETF.
Perhaps I have that wrong, but it seems that someone should be
announcing such things to the TCPM list at least in parallel with some
ICCRG effort that is supposedly for the IETF's benefit.

> As you probably know, ICCRG has agreed to obtain reviews
> on experimental congestion control proposals before they
> are brought to the IETF (specifically the TCPM group). While
> the competence to actually decide about acceptance or not is
> with TCPM, it is expected that they will take our reviews
> into account. This process is outlined here:
> http://www.ietf.org/IESG/content/ions/drafts/ion-tsv-alt-cc.txt

As far as templates go I think "TCPM" should be an "XXX" because it is
possible for things to be tracked elsewhere within the IETF.  E.g., it
seems likely that TSVWG---instead of TCPM---would handle something like
XCP since it is not TCP.

> If you're interested in doing a review, please send a note
> to Wes and me. Reviews can be sent to the list, or directly
> to us for anonymization before reflecting them out to the
> list, if desired.

Since this is a community effort and not peer review, I'd like to see
your note encourage folks to make their reviews open and signed.  (I.e.,
to use the anonymization scheme as a 'last resort'.)

> * consider not only the draft alone, but also papers referenced
>   therein, where the authors should have carried out a performance 
>   evaluation of their mechanism, including studies which show the

Nit: I'd change "performance evaluation" to "evaluation". 

>   impact of the new mechanism on standard TCP. When looking at
>   such studies, this document is recommended to be used for guidance:
>   http://www.ietf.org/internet-drafts/draft-irtf-tmrg-metrics-09.txt
> 
> Reviews should include one of the following final recommendations:
> 
> * Experimental 1:
>   The proposed mechanism is judged to be safe to deploy for
>   best-effort traffic in the global Internet and further
>   investigated in that environment.
> 
> * Experimental 2 A:
>   While promising, the proposed mechanism is not deemed safe
>   enough for widespread deployment as best-effort traffic on
>   the Internet, but can be specified to facilitate investigations
>   in simulation, testbeds, or controlled environments.
> 
> * Experimental 2 B:
>   The IETF does not yet have sufficient understanding to decide
>   if the algorithm is or is not safe for deployment on the Internet.
> 
> * Informational:
>   The proposed mechanism does not meet the criteria for recommending
>   publication of the draft as an Experimental RFC. It should be
>   considered for publication as an Informational RFC for the benefit
>   of the IETF and IRTF communit, provided that it carries an
>   explicit warning against using the scheme in the global Internet.
>   
> * Not ready:
>   The draft is not ready for publication as an Experimental or
>   Informational RFC.
> 
> A review can be as short as a selection from the list of options,
> although preferably it would include at least some brief
> discussion or justification. No matter how long the feedback
> is, we expect reviewers to have thoroughly read all the
> necessary material.

First, I would note in here that if the TCPM group (or at least this
*participant*) gets a review that is simply a selection from the above
list then that review may not carry much weight.  The more careful,
complete and descriptive a review is the more credence it will be given,
I strongly suspect.  It might be good to note this in your summation
above.

Second, I am not sure if I like the list.  Every time I try to make a
'which option do you like best?' list people always start adding
options.  Maybe I am lousy with lists.  ;)  The CC-ALT document (in the
rfc-ed queue) that Sally and I wrote says that draft authors have to
make statements in their documents about the "safeness" of their scheme
for use in the global Internet.  The text from our document:

    Each alternate congestion control algorithm published is required to
    include a statement in the abstract indicating whether or not the
    proposal is considered safe for use on the Internet.  Each alternate
    congestion control algorithm published is also required to include a
    statement in the abstract describing environments where the protocol
    is not recommended for deployment.  There may be environments where
    the protocol is deemed *safe* for use, but still is not
    *recommended* for use because it does not perform well for the user.

So, perhaps instead of choosing which option the reviewers think is
appropriate from a canned list that might not be quite right for a
particular proposal, the question for reviewers should be whether or not
they buy the statement-on-safeness included in the draft itself.  Of
course, with the reviewers showing their work at how they arrived at
their decision.

allman



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20070806/53f203b0/attachment.bin