[Iccrg] Re: [Tmrg] A reminder about An NS2 TCP Evaluation Tool Suite

[Lachlan Andrew]

Greetings Gang Wang,

Your tool looks nice.  Here are some suggestions:

1. Each topology seems to specify a single bottleneck capacity and
RTT.  Is there a way to make it test for a a range of capacities, like
10, 100, 155 and 622 Mbps?

2. It specifies a diff_RTT variable so that flows can have different
RTTs, but it seems that RTTs are equally spaced within the allowed
range.  This equal spacing may cause artefacts.  More importantly,
real RTTs aren't uniformly distributed.  It would be good to have a
more realistic distribution of RTTs.  (If they're generated randomly,
it will be important to make it repeatable still.)

3. Jain's measure of fairness does not reflect the user's experience.
  - A fairness measure should give more weight to flows receiving very
low throughput.  If 10 flows get equal throughput, and one flow gets
nothing, that is very unfair, but scores highly in Jain's index.
  - This can partly be overcome by applying Jain's index to the
*download times* instead of the rates.  As an approximation of the
download time, you could use the reciprocal of the rate.
  - Jain's measure also doesn't consider the impact of multiple
bottlenecks.  In a parking-lot topology with links of unequal
capacity, the "fairest" solution IMO is for the flow which only uses
the high-capacity link not to be restricted by the fact that there is
another low-capacity link which it doesn't use.  Jain's index only
gives a high score if the high-capacity link is under-utilized.

4. The parking lot topology is very symmetric.  It would be
interesting to look at parking-lot topologies with different
bandwidths on the different bottlenecks.

Cheers,
Lachlan

On 07/08/07, Wang gang <wanggang at research.nec.com.cn> wrote:
>
> Dear colleagues,
>
> We have released the tool 'An NS2 TCP Evaluation Tool Suite' for some time.
> Since
> then, we have received some feed back from users. We expect to receive wider
> comments,
> and seek collaborations or contributions to make the tool towards a useful
> one.
>
> The download page is,
> http://labs.nec.com.cn/tcpeval.htm
>
>
> Here is a brief introduction,
> This tool is motivated by the observation that there is significant overlap
> among (but lack
> of an agreed set of) the topologies, traffic, and metrics used by many
> researchers in the
> evaluation of TCP alternatives: effort could be saved by starting research
> from an existing
> framework.  As such, our tool includes several typical topologies and
> traffic models; it measures
> some of the most important metrics commonly used in TCP evaluation; and it
> can automatically
> generate simulation statistics and graphs ready for inclusion in latex and
> html documents.  The
> tool also contains an extendable open-source framework.  With community
> effort, we hope the
> tool evolves into a widely accepted, well-defined set of TCP performance
> evaluation benchmarks.
>
> Best Regards.
>
> Gang Wang.
>
> ----------------------------------------
> Gang Wang
> NEC Labs, China
> 010-62705962/63  (ext.511)
>
> wanggang at research.nec.com.cn
>
>
>
> _______________________________________________
> Tmrg-interest mailing list
> Tmrg-interest at ICSI.Berkeley.EDU
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest
>
>


-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Phone: +1 (626) 395-8820    Fax: +1 (626) 568-3603