[Tmrg-interest] Re: [Iccrg] Proposal for ICCRG operations

[Filipe Abrantes]

Hi Michael,

before I agree with you, let me disagree for a bit :) .

Michael Welzl wrote:
> [cc'ing TMRG]
> 
> 
> Hi all,
> 
> I'm very happy to see some movement here. Clearly, many of us
> would want something concrete to come out of ICCRG, but how to
> reach agreement is the tricky bit.
> 
> 
> 
>>>I think a TCP benchmarking suite will be a very helpful tool to
>>>guide us in this process.  It will be useful especially in your
>>>Phase 3 to evaluate various proposals, but just thinking through
>>>the design and implementation of a good benchmark lends clarity
>>>to issues of Internet congestion control and metrics to optimize
>>>against.
>>>
>>>Aaron Falk is chairing a panel at a conference to discuss this
>>>very issue.  Hopefully, we wll hear from the participants soon
>>>on their thoughts.
>>
>>So, the panel isn't for several hours (at PFLDnet 2006) but one thing  
>>that has become clear to me so far is that this community has not  
>>converged on metrics and scenarios for evaluation.  The TMRG work is  
>>really needed to make progress.
> 
> 
> I have an overly simplistic proposal ... this is something I had
> in my mind for a long time, but I kept dismissing it again and
> again because it probably isn't feasible. Nevertheless, I'm now
> bringing it up as food for thought rather than a concrete
> suggestion:
> 
> Wouldn't it be great if we had a single "congestion control
> performance index" ? (or wait, before anybody agrees with me,
> let me call it "Welzl index" instead   :)   )
> 
> After all, it's not that much that we demand from a
> congestion control mechanism: it must avoid congestion
> while efficiently and fairly utilizing the available
> bandwidth, right? Congestion means that queues grow, leading
> to increased delay and eventually loss beyond a certain
> length (or depending on some AQM function); should the
> index then contain loss _AND_ delay? For simplicity, I'd
> suggest to ignore delay. Let's say we only have loss
> in the index, and someone comes up with a result that
> shows how good a mechanism is doing in a certain scenario,
> while in fact queues are growing - well, then it's a
> matter of changing the scenario just a little bit (more
> flows and/or shorter queues), and we have a case of
> "inadequate performance evaluation" (which we can always
> have).
> 

I have thought for a while why do we talk about "congestion control" 
when what all these mechanisms are responsible for, is a bit more than 
solely avoid congestion. Rate control would be a bit more accurate from 
my point of view. That said, I just wanted to disagree about leaving 
delay out of the metric. I didn't quite understood your explanation, but 
certainly a mechanisms that maintains a lower queue is better than one 
that maintains an higher queue (and note that there are mechanisms that 
maintain stable queues avoiding most of the packet drops).

Other than that I agree with you about the need of nice little index for 
uniform performance measurement.

br

-filipe

> So, the index should contain:
> 
> * normalized throughput
> * loss
> * fairness (let's use the Jain Fairness Index)
> 
> It could then look something like this:
> 
> WI = (T*t + L*l + F*f) / 3
> 
> where WI is "Welzl Index" (tm)   ;-)
> T is normalized throughput
> L is normalized loss
> F is the Jain fairness index (which is already a value
> in the 0..1 range)
> 
> and t, l, f are weighting factors which decide how
> important the parameters are to us.
> 
> Now the question would be how to appropriately set these
> factors... we'd have to decide on fixed values, I
> guess, or otherwise the simplification obtained with the
> index isn't really a simplification after all. Should
> t, l and f all be 1?
> 
> While I don't seriously believe that we can all just
> agree on the Welzl index, I believe that thinking in
> this direction of simplifying performance evaluations
> is worthwhile.
> 
> Just imagine what congestion control papers could look
> like if we had such an index... any comparisons could be
> more focused. Sure, all the usual scenarios would still
> have to be checked (dumbbell and beyond  :-)  ), but
> we'd reduce the problem space by one dimension - instead
> of showing loss, throughput, fairness etc., the Welzl index
> would be enough.
> 
> Cheers,
> Michael
> 
> _______________________________________________
> Tmrg-interest mailing list
> Tmrg-interest at ICSI.Berkeley.EDU
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest
> 

-- 
Filipe Lameiro Abrantes
INESC Porto
Campus da FEUP
Rua Dr. Roberto Frias, 378
4200-465 Porto
Portugal

Phone: +351 22 209 4266
E-mail: fla at inescporto.pt