GitLab

This enhances the handling of swarms in the client, especially to avoid a
long-running one to accumulate too many swarm entries, which results in the
client very seldom pinging injectors and thus stop announcing itself as a
bridge.

Enhancements:

  - Use a limited-size LRU cache to keep swarm entries at the client (to
    eventually drop spurious entries).
  - Do not ping injectors if a connection to one of them was successfully
    established while waiting for the next ping round.

This includes some enhancements to cope with transitory addresses appearing in
injector and bridge swarms, be they fake (attackers or BitTorrent spy nodes)
or genuine (like clients no longer available), as everything seen in the
swarms was never forgotten by the client, which caused the local view of the
swarm to grow indefinitely, causing issues like legitimate injectors never
getting picked for pinging in long-running clients.

  - Use an LRU cache for swarm entries in the client (to eventually drop
    spurious entries).
  - Also, do not ping injectors if a connection to one of them was
    successfully established while waiting for the next ping round.

Further refinement in swarm and ping handling paramenters may be needed, as
well as more sophisticated handling of the ping process.

Since we do not want iteration to allow altering the cache contents.  Also, it
is how it was being used until now.

Although we should probably do some serious statistics on them.

To help choose meaningful values.

Still need to figure out proper values.

To avoid entries from accumulating indefinitely, especially since they come
and go all the time.

Still failing to build as immutable is not yet properly supported by the LRU
cache.

Proper, different sizes need to be chosen for injector and bridge swarms.

The pause is not cancelled though, injectors are simply considered to be
reachable if the exchange happened during the pause.

This fixes some issues with the client announcing itself in the bridge swarm,
particularly in the pinging of injectors, which may eventually leave the
client out of the swarm and thus unreachable to others, especially after being
running for an extended period.

Enhancements:

  - Ignore DHT martians in the `bt-bep5` utility.  As a result, its output and
    that of `ping-swarm` are less cluttered and more useful.

Bugfixes:

  - Fix the computation of pauses between pinging injectors at the client, so
    that pings are not too far apart, which may cause the client's entry in
    the bridge swarm to become questionable.
  - Fix the selection of injectors to be pinged in each round, avoiding never
    actually pinging interesting injectors if they are located after a certain
    point in the internal list of nodes seen in the injector swarm.

If we choose a subset then shuffle it, we risk never pinging that appear after
the subset in the complete set of known injectors.  So first shuffle the whole
list and then choose a subset from the beginning.

This also avoids accidentally waiting more than expected when the time passed
since last ping is larger than the time remaining until the expected next ping.

This updates protocol specs to reflect Ouinet protocol v6, as well as
signalling with TODOs where the implementation needs work to completely
support it.

The description of the multi-peer download protocol is still missing, though.

This release improves logging of injector and bridge client announcements to
BitTorrent swarms, while fixing some issues which may result in both types of
nodes randomly disappearing from those swarms for a while.  It also makes
nodes more able to recover from local connectivity loss.

The release also fixes an issue affecting new Android builds (and breaking
CI/CD).

Enhancements:

  - Better logging of injector pings and BEP5 announcements.
  - Shorter interval of announcements to the injector swarm, to avoid the DHT
    entry to become "questionable" in BEP5 terms.

Bugfixes:

  - Avoid pausing after a successful announcement to the helper/bridge swarm,
    so that announcements follow injector pings.

Otherwise a build which starts with no Android SDK/NDK downloads may fail with
"NDK is missing a "platforms" directory.".

The export is done in `maybe_install_sdk()` (bootstrap mode) much in the same
vein as `ANDROID_HOME` is set in `setup_deps()`, although it may be better to
set them between bootstrap and build.

This enhances logging of the BEP5 announcer and injector pinger classes.

It also fixes a couple of issues which may result in

  - A client not announcing itself to the helpers/bridges swarm for too
    long (as a pause was done after successful announcement was done which
    should only be made with periodic announcements as done in the injector).

  - An injector not announcing itself to the injectors swarm for too long (as
    the pause between announcements may make the DHT entry "questionable"
    according to BEP5).

Otherwise we risk the entry being considered "questionable" in the DHT.  For
injectors, this may mean getting temporarily removed from the injector swarm.

To avoid having to wait for at most 30 minutes until the next injector
announce.

This may cause the announcer to miss manual updates because of a pause that
should only be done with periodic announcement behaviour.  For the client,
this may mean not announcing in time in the helpers/bridges swarm, possibly
rendering it unfindable to other peers.

As done further down the function.  At any rate, DHT announcement would
probably fail with network unreachable.

The wait finishes successfully if `cv` is notified.

Otherwise a wait error may be passed into the DHT announce call.

It uses normal debugging, as a client or injector will only have one such
instance which triggers not so frequently.

So their purpose and usage becomes clearer.

Including a not-intrusive (verbose-only) message when injectors are not
reachable.

To avoid having to wait for 10 minutes until the next ping.