[Iccrg] Proposal for ICCRG operations

[Michael Welzl]

[cc'ing TMRG]


Hi all,

I'm very happy to see some movement here. Clearly, many of us
would want something concrete to come out of ICCRG, but how to
reach agreement is the tricky bit.


> > I think a TCP benchmarking suite will be a very helpful tool to
> > guide us in this process.  It will be useful especially in your
> > Phase 3 to evaluate various proposals, but just thinking through
> > the design and implementation of a good benchmark lends clarity
> > to issues of Internet congestion control and metrics to optimize
> > against.
> >
> > Aaron Falk is chairing a panel at a conference to discuss this
> > very issue.  Hopefully, we wll hear from the participants soon
> > on their thoughts.
> 
> So, the panel isn't for several hours (at PFLDnet 2006) but one thing  
> that has become clear to me so far is that this community has not  
> converged on metrics and scenarios for evaluation.  The TMRG work is  
> really needed to make progress.

I have an overly simplistic proposal ... this is something I had
in my mind for a long time, but I kept dismissing it again and
again because it probably isn't feasible. Nevertheless, I'm now
bringing it up as food for thought rather than a concrete
suggestion:

Wouldn't it be great if we had a single "congestion control
performance index" ? (or wait, before anybody agrees with me,
let me call it "Welzl index" instead   :)   )

After all, it's not that much that we demand from a
congestion control mechanism: it must avoid congestion
while efficiently and fairly utilizing the available
bandwidth, right? Congestion means that queues grow, leading
to increased delay and eventually loss beyond a certain
length (or depending on some AQM function); should the
index then contain loss _AND_ delay? For simplicity, I'd
suggest to ignore delay. Let's say we only have loss
in the index, and someone comes up with a result that
shows how good a mechanism is doing in a certain scenario,
while in fact queues are growing - well, then it's a
matter of changing the scenario just a little bit (more
flows and/or shorter queues), and we have a case of
"inadequate performance evaluation" (which we can always
have).

So, the index should contain:

* normalized throughput
* loss
* fairness (let's use the Jain Fairness Index)

It could then look something like this:

WI = (T*t + L*l + F*f) / 3

where WI is "Welzl Index" (tm)   ;-)
T is normalized throughput
L is normalized loss
F is the Jain fairness index (which is already a value
in the 0..1 range)

and t, l, f are weighting factors which decide how
important the parameters are to us.

Now the question would be how to appropriately set these
factors... we'd have to decide on fixed values, I
guess, or otherwise the simplification obtained with the
index isn't really a simplification after all. Should
t, l and f all be 1?

While I don't seriously believe that we can all just
agree on the Welzl index, I believe that thinking in
this direction of simplifying performance evaluations
is worthwhile.

Just imagine what congestion control papers could look
like if we had such an index... any comparisons could be
more focused. Sure, all the usual scenarios would still
have to be checked (dumbbell and beyond  :-)  ), but
we'd reduce the problem space by one dimension - instead
of showing loss, throughput, fairness etc., the Welzl index
would be enough.

Cheers,
Michael