[Concept] Disconnect Policies
 

Hi,

Currently Phex has individual disconnect policies, which are off by default.

This means, that it routes all packets, and if a "greedy" client connects, that client can severly decrease our own search performance.

To gather ideas how to change this (and talk about if it's needed), I want to add the following idea as starting point for discussion:

So I think it would be useful to instead try to "get into a part of the network we can manage".

A possible way to archieve that is the following set of disconnect policies:

Drop single congested outgoing connections
If only one outgoing connection is severly congested, disconnect it.
Metric: outgoing dropped % / mean outgoing dropped percentage.

used parameters:
dropped_threshold: below this threshold, no connection gets dropped.
dropped_ratio_threshold: How much the ratio of a single connection must be below the mean ratio to drop a connection.

close the single connection if
- outgoing drop % > dropped_threshold, AND
- outgoing drop % / mean outgoing drop % > dropped_ratio_threshold

example values are:
dropped_threshold: 30%
dropped_ratio_threshold: 3

The result of this is, that a single too weak link gets closed, so we don't batter too weak nodes.

If only one node drops packets, this node gets removed, if it reaches the dropepd_threshold.

On general congestion, drop the most greedy incoming connection

If the mean dropped ratio rises above a threshold (and only then), check all incoming connections and close the one with the highest dropped %.

(is it possible to only count incoming packets we have to forward?)

Target overall effect

The goal is to have Phex move itself automatically towards other nodes who have similar bandwidth requirements.

Slow nodes group themselves around other slow nodes, and greedy nodes get pushed out automatically, but only when they begin to harm the network (no need to do unnecessary policing - as long as it doesn't harm the network the node with the highest incoming packet rate may well be the best source for Query Replies).

Since all used values are accessible from the network tab, I assume that Phex already has all necessary information.

What do you think about the idea?

They are not off, they are active just not configurable through the UI anymore.

Check NetworkHostsContainer.periodicallyCheckHosts()

Good to know :)
Thanks!

I think I can code a part of the policy myself - do you see any direct weaknesses in it (means: is it ready to go into code)?

What I don't yet know is how to efficiently get the mean drop ratio and the mean send queue (I don't want to iterate over all hosts everytime I need to check it).

A possible way would be a simple attribute of the NetworkHostContainer. It could be updated by adding ((host.dropRatio - networkHostsContainer.overallDropRatio) / number_of_hosts) to it for every host (doing it for all hosts once before doing any checks and then inside the checks to keep it up to date).

Which concepts do other vendors follow? Is our current implementation so inefficient that it needs to be replaced? Will the new concept be so much better?

For LimeWire I don't know.

gtk-gnutella limits
* duplicate packets to 5 or 1.5% (the higher one)
* flow controlled time (max. 70% of the time, max 180s) - I'm not sure what that means, but I assume it's the time when packets get dropped.

I don't think that our implementation is inefficient, and the code is very nice to read, but I think that the proposed one will be better for the network and the user, since the client will automatically move towards clients with similar bandwidth capacity and query behaviour.

I like the use of time when calculating the drops.. When you are very good connected to a host for a long time and it's drop count rises it will take very very long until you reach the limit when you don't calculate in time frames.

You mean, for example, taking the mean drop rate over time as measure?

That has the disadvantage of not avoiding overload if the other host suddenly gets flooded and forwards the flood to us.

But maybe the measured drop rate could be adjusted via the connect time (for example reducing the drop rate used for selecting the host to disconnect by up to 1/3rd of the maximum allowed drop rate for long standing connections) to avoid disconnecting good hosts to early.

As far as I know, there's already a measure for "connection already lasts long" (at least for Phex hosts "is good phex").

I think I just misunderstood your post.

Did you mean: "Have a time window in which to calculate the drop rate"?

If so, I think your perfectly right.

With that added, what do you think about the concept of a more "directed" disconnect policy?

directed?