README.md 29 KB
Newer Older
wldhx's avatar
wldhx committed
1
2
[![CircleCI](https://circleci.com/gh/equalitie/ouinet/tree/master.svg?style=shield)](https://circleci.com/gh/equalitie/ouinet/tree/master) [![pipeline status](https://gitlab.com/equalitie/ouinet/badges/master/pipeline.svg)](https://gitlab.com/equalitie/ouinet/commits/master)

Peter Jankuliak's avatar
Peter Jankuliak committed
3

Peter Jankuliak's avatar
Peter Jankuliak committed
4
5
# Ouinet

6
7
See [lightning talk at the Decentralized Web Summit 2018](http://archive.org/details/dweb-8_2_18_Lightning_Talks_New_Discoveries_5?start=547).

8
9
**Ouinet** is a Free/Open Source technology which allows web content to be
served with the help of an entire network of cooperating nodes using
10
peer-to-peer routing and distributed caching of responses.  This helps
11
12
mitigate the Web's characteristic single point of failure due to a client
application not being able to connect to a particular server.
13

14
15
16
17
The typical Ouinet *client* node setup consists of a web browser or other
application using a special HTTP proxy or API provided by a dedicated program
or library on the local machine.  When the client gets a request for content,
it attempts to retrieve the resource using several mechanisms.  It tries to
18
19
20
21
22
fetch the page from a *distributed cache* by looking up the content in a
*distributed cache index* (like the [BitTorrent][] DHT), and if not available,
it contacts a trusted *injector* server over a peer-to-peer routing system
(like [I2P][]) and asks it to fetch the page and store it in the distributed
cache.
23

24
[BitTorrent]: https://www.bittorrent.org/
25
[I2P]: https://geti2p.net/ "Invisible Internet Project"
26

27
28
![Ouinet request/response flow](./doc/request-response-flow.svg)

Jenny Ryan's avatar
Jenny Ryan committed
29
30
31
32
33
34
35
Future access by client nodes to popular content inserted in distributed
storage shall benefit from increased redundancy and locality, which
translates to: increased availability in the face of connectivity problems;
increased transfer speeds in case of poor upstream links; and reduced
bandwidth costs when internet access providers charge more for external or
international traffic.  Content injection is also designed to
allow for content re-introduction and seeding on extreme cases of total
36
37
38
39
40
41
42
connectivity loss (e.g. natural disasters).

The Ouinet library is a core technology that can be used by any application to
benefit from these advantages.  Ouinet integration provides any content
creator the opportunity to use cooperative networking and storage for the
delivery of their content to users around the world.

43
**Warning:** Ouinet is still **highly experimental**.  Some features (like
44
peer-to-peer routing) may or may not not work smoothly depending on the
45
46
47
48
different back-end technologies, and random unexpected crashes may occur.
Also, Ouinet is **not an anonymity tool**: information about your browsing
might be leaked to other participants in the network, as well as the fact that
your application is seeding particular content.  Running some components (like
49
50
injector code) may turn your computer into an open web proxy, and other
security or privacy-affecting issues might exist.  Please keep this in mind
51
when using this software and only assume reasonable risks.
52

53
**Note:** The steps described below have only been tested to work on GNU/Linux
54
55
56
on AMD64 platforms.  Building and testing Ouinet on your computer requires
familiarity with the command line.  At the moment there are no user-friendly
packages for Ouinet on the desktop.
57

58
## Cloning the source tree
Veggie Monster's avatar
Veggie Monster committed
59

60
Ouinet uses Git submodules, thus to properly clone it, use:
Veggie Monster's avatar
Veggie Monster committed
61

62
    $ git clone --recursive https://github.com/equalitie/ouinet.git
63

64
You can also clone and update the modules separately:
65

66
67
68
    $ git clone https://github.com/equalitie/ouinet.git
    $ cd ouinet
    $ git submodule update --init --recursive
69

70
## Build requirements (desktop)
71

72
73
To build Ouinet natively on your system, you will need the following software
to be already available:
74

75
76
77
* CMake 3.5+
* `g++` capable of C++14
* The [Boost library](http://www.boost.org/) 1.67+
78

79
80
Assuming that `<SOURCE DIR>` points to the directory where the
`CMakeLists.txt` file is, and `<BUILD DIR>` is a directory of your choice
81
where all (even temporary) build files will go, you can build Ouinet with:
82

83
84
85
    $ mkdir -p <BUILD DIR>
    $ cd <BUILD DIR>
    $ cmake <SOURCE DIR>
86
    $ make
87

88
89
90
91
However, we encourage you to use a Vagrant environment for development, or
Docker containers for deploying a Ouinet client or an injector.  These have a
different set of requirements.  See the corresponding sections below for
further instructions on Vagrant and Docker.
92

93
## Running integration tests
94

95
96
The Ouinet source comes with a set of integration tests.  To run them you will
need the [Twisted](https://twistedmatrix.com/) Python framework.
97

98
99
If you already built Ouinet from `<SOURCE DIR>` into `<BUILD DIR>` (see
above), you can run the tests as follows:
100

101
102
103
    $ export OUINET_REPO_DIR=<SOURCE DIR>
    $ export OUINET_BUILD_DIR=<BUILD DIR>
    $ ./scripts/run_integration_tests.sh
Peter Jankuliak's avatar
Peter Jankuliak committed
104

105
## Using a Vagrant environment
106

107
108
109
One of the easiest ways to build Ouinet from source code (e.g. for development
or testing changes and fixes to code) is using a [Vagrant][] development
environment.
Peter Jankuliak's avatar
Peter Jankuliak committed
110

111
[Vagrant]: https://www.vagrantup.com/
112

113
To install Vagrant on a Debian system, run:
114

115
    $ sudo apt-get install vagrant
116

117
Ouinet's source tree contains a `Vagrantfile` which allows you to start a
118
119
Vagrant environment ready to build and run Ouinet by entering the source
directory and executing:
120

121
    $ vagrant up
122

123
124
If your Vagrant installation uses VirtualBox by default and you find problems,
you may need to force it to use libvirt instead:
Peter Jankuliak's avatar
Peter Jankuliak committed
125

126
127
128
    $ sudo apt-get install libvirt-bin libvirt-dev
    $ vagrant plugin install vagrant-libvirt
    $ vagrant up --provider=libvirt
Peter Jankuliak's avatar
Peter Jankuliak committed
129

130
### Building Ouinet in Vagrant
Peter Jankuliak's avatar
Peter Jankuliak committed
131

132
Enter the Vagrant environment with `vagrant ssh`.  There you will find:
Peter Jankuliak's avatar
Peter Jankuliak committed
133

134
135
  - Your local Ouinet source tree mounted read-only under `/vagrant`
    (`<SOURCE DIR>` above).
Peter Jankuliak's avatar
Peter Jankuliak committed
136

137
138
  - Your local Ouinet source tree mounted read-write under `/vagrant-rw`.  You
    can use it as a bridge to your host.
Peter Jankuliak's avatar
Peter Jankuliak committed
139

140
  - `~vagrant/build-ouinet-git.sh`: Running this script will clone the Ouinet
141
142
143
    Git repository and all submodules into `$PWD/ouinet-git-source` and build
    Ouinet into `$PWD/ouinet-git-build` (`<BUILD DIR>` above).  Changes to
    source outside of the Vagrant environment will not affect this build.
Peter Jankuliak's avatar
Peter Jankuliak committed
144

145
146
  - `~vagrant/build-ouinet-local.sh`: Running this script will use your local
    Ouinet source tree (mounted under `/vagrant`) to build Ouinet into
147
148
    `$PWD/ouinet-local-build` (`<BUILD DIR>` above).  Thus you can edit source
    files on your computer and have them built in a consistent environment.
Peter Jankuliak's avatar
Peter Jankuliak committed
149

150
    Please note that this requires that you keep submodules in your checkout
151
    up to date as indicated above.
Peter Jankuliak's avatar
Peter Jankuliak committed
152

153
### Accessing Ouinet services from your computer
Peter Jankuliak's avatar
Peter Jankuliak committed
154

155
156
The Vagrant environment is by default isolated, but you can configure it to
redirect ports from the host to the environment.
Peter Jankuliak's avatar
Peter Jankuliak committed
157

158
159
For instance, if you want to run a Ouinet client (with its default
configuration) in Vagrant and use it as a proxy in a browser on your computer,
160
you may uncomment the following line in `Vagrantfile`:
Peter Jankuliak's avatar
Peter Jankuliak committed
161

162
    #vm.vm.network "forwarded_port", guest: 8077, host: 8077, guest_ip: "127.0.0.1"
Peter Jankuliak's avatar
Peter Jankuliak committed
163

164
And restart the environment:
Peter Jankuliak's avatar
Peter Jankuliak committed
165

166
167
    $ vagrant halt
    $ vagrant up
Peter Jankuliak's avatar
Peter Jankuliak committed
168

169
Then you can configure your browser to use `localhost` port 8077 to contact
170
the HTTP proxy.
Peter Jankuliak's avatar
Peter Jankuliak committed
171

172
173
## Docker development environment

174
175
176
We provide a *bootstrap* Docker image which is automatically updated with each
commit and provides all prerequisites for building the latest Oiunet desktop
binaries and Android libraries.
wldhx's avatar
mew    
wldhx committed
177

178
179
180
To exchange with the container data like Ouinet's source code and cached
downloads and build files, we will bind mount the following directories to
`/usr/local/src/` in the container (some we'll create first):
wldhx's avatar
mew    
wldhx committed
181

182
183
184
185
186
187
188
189
  - source (assumed to be at the current directory),
  - build (in `../ouinet.build/`),
  - and the container's `$HOME` (in `../ouinet.home/`), where `.gradle`,
    `.cargo`, etc. will reside.

Note that with the following incantations you will not be able to use `sudo`
in the container (`--user`), and that all the changes besides those in bind
mounts will be lost after you exit (`--rm`).
190
191

```sh
wldhx's avatar
mew    
wldhx committed
192
mkdir -p ../ouinet.build/ ../ouinet.home/
193
194
195
196
197
198
199
200
sudo docker run \
  --rm -it \
  --user $(id -u):$(id -g) \
  --mount type=bind,source="$(pwd)",target=/usr/local/src/ouinet \
  --mount type=bind,source="$(pwd)/../ouinet.build",target=/usr/local/src/ouinet.build \
  --mount type=bind,source="$(pwd)/../ouinet.home",target=/mnt/home \
  -e HOME=/mnt/home \
  registry.gitlab.com/equalitie/ouinet:android
201
202
```

203
204
205
206
207
208
If you only need to build Ouinet desktop binaries, you may replace the image
name at the end of the command with `registry.gitlab.com/equalitie/ouinet`,
which is much lighter.

After running the command, you should find yourself in a new terminal, ready
to accept the build instructions described elsewhere in the document.
wldhx's avatar
mew    
wldhx committed
209

210
211
Please consult the GitLab CI scripts to see how to build your own bootstrap
images locally.
212
213

## Docker deployment
214

215
Ouinet injectors and clients can be run as Docker containers.  An application
216
217
configuration file for Docker Compose is included for easily deploying all
needed volumes and containers.
218

219
220
221
222
To run a Ouinet node container only a couple hundred MiB are needed, plus the
space devoted to the data volume (which may grow considerably larger in the
case of the injector).

223
224
A `Dockerfile` is also included that can be used to create a Docker image
which contains the Ouinet injector, client and necessary software dependencies
225
226
running on top of a Debian base system.

227
### Building the image
228

229
Ouinet Docker images should be available from the Docker Hub.  Follow the
230
instructions in this section if you still want to build the image yourself.
231
You will need around 3 GiB of disk space.
232
233
234
235

You may use the `Dockerfile` as included in Ouinet's source code, or you
can just [download it][Dockerfile].  Then build the image by running:

236
    $ sudo docker build -t equalitie/ouinet:latest - < Dockerfile
237

238
239
That command will build a default recommended version, which you can override
with `--build-arg OUINET_VERSION=<VERSION>`.
240

241
242
243
After a while you will get the `equalitie/ouinet:latest` image.  Then you may
want to run `sudo docker prune` to free up the space taken by temporary
builder images (which may amount to a couple of GiB).
244

245
246
[Dockerfile]: https://raw.githubusercontent.com/equalitie/ouinet/master/Dockerfile

247
248
249
250
251
252
253
254
255
256
257
258
#### Debugging-enabled image

You can also build an alternative version of the image where programs contain
debugging symbols and they are run under `gdb`, which shows a backtrace in
case of a crash.  Just add `--build-arg OUINET_DEBUG=yes` to the build
command.  We recommend that you use a different tag for these images
(e.g. `equalitie/ouinet:<VERSION>-debug`).

Depending on your Docker setup, you may need to change the container's
security profile and give it tracing capabilities.  For more information, see
[this thread](https://stackoverflow.com/q/35860527).

259
### Deploying a client
260

261
262
You may use [Docker Compose](https://docs.docker.com/compose/) with the
`docker-compose.yml` file included in Ouinet's source code (or you can just
263
264
265
266
267
268
[download it][docker-compose.yml]).  Whenever you run `docker-compose`
commands using that configuration file, you must be at the directory where the
file resides.

If you just plan to **run a single client** with the latest code on you
computer, you should be fine with running the following command:
269

270
    $ sudo docker-compose up
271

272
That command will create a *data volume*, a main *node container* for running
273
the Ouinet client or injector (using the host's network directly), and a
274
275
276
convenience *shell container* (see below) to allow you to modify files in the
data volume.  It will then run the containers (the shell container will exit
immediately; this is normal).
277

278
279
280
281
To **stop the node**, hit Ctrl+C or run `sudo docker-compose stop`.  Please
note that with the default configuration in `docker-compose.yml`, the node
will be automatically restarted whenever it crashes or the host is rebooted,
until explicitly stopped.
282

283
284
A new client node which starts with no configuration will get a default one
from templates included in Ouinet's source code and it will be missing some
285
important parameters, so you may want to stop it (see above) and use the
286
**shell container** (see below) to edit `client/ouinet-client.conf`:
287

288
289
290
  - If using a local test injector, set its endpoint in option `injector-ep`.
  - Set the injector's credentials in option `injector-credentials`.
  - Unless using a local test injector, set option `injector-tls-cert-file` to
291
292
    `/var/opt/ouinet/client/ssl-inj-cert.pem` and copy the injector's TLS
    certificate to that file.
293
294
295
296
  - Set the public key used by the injector for HTTP signatures in option
    `cache-http-public-key`.
  - To enable the distributed cache, set option `cache-type`.  The only value
    currently supported is `bep5-http`.
297

298
After you have set up your client's configuration, you can **restart it**.
299
The client's HTTP proxy endpoint should be available to the host at
300
`localhost` port 8077.
301
302
303

If you get a "connection refused" error when using the client's proxy, your
Docker setup may not support host networking.  To enable port forwarding,
304
follow the instructions in `docker-compose.yml`.
305
306

Finally, restart the client container.
307

308
[docker-compose.yml]: https://raw.githubusercontent.com/equalitie/ouinet/master/docker-compose.yml
309

310
### Using the shell container
311

312
You may use the convenience *shell container* to access Ouinet node files
313
directly:
314

315
    $ sudo docker-compose run --rm shell
316

317
This will create a throwaway container with a shell at the `/var/opt/ouinet`
318
directory in the data volume.
319

320
321
If you want to *transfer an existing repository* to `/var/opt/ouinet`, you
first need to move away or remove the existing one using the shell container:
322

323
    # mv REPO REPO.old  # REPO is either 'injector' or 'client'
324

325
Then you may copy it in from the host using:
326

327
    $ sudo docker cp /path/to/REPO SHELL_CONTAINER:/var/opt/ouinet/REPO
328

329
330
331
332
333
334
### Other deployments

If you plan on running several nodes on the same host you will need to use
different explicit Docker Compose project names for them.  To make the node an
injector instead of a client you need to set `OUINET_ROLE=injector`.  To make
the container use a particular image version instead of `latest`, set
335
336
337
`OUINET_VERSION`.  To limit the amount of memory that the container may use,
set `OUINET_MEM_LIMIT`, but you will need to pass the `--compatibility` option
to `docker-compose`.
338
339
340
341
342
343
344
345
346

An easy way to set all these parameters is to copy or link the
`docker-compose.yml` file to a directory with the desired project name and
populate its default environment file:

    $ mkdir -p /path/to/ouinet-injector  # ouinet-injector is the project name
    $ cd /path/to/ouinet-injector
    $ cp /path/to/docker-compose.yml .
    $ echo OUINET_ROLE=injector >> .env
347
348
    $ echo OUINET_VERSION=v0.1.0 >> .env
    $ echo OUINET_MEM_LIMIT=6g >> .env
349
    $ sudo docker-compose --compatibility up
350

351
### Injector container
352

353
354
After an injector has finished starting, you may want to use the shell
container to inspect and note down the contents of `injector/endpoint-*`
355
356
357
358
359
(injector endpoints) and `injector/ed25519-public-key` (public key for HTTP
signatures) to be used by clients.  The injector will also generate a
`tls-cert.pem`  file which you should distribute to clients for TLS access.
Other configuration information like credentials can be found in
`injector/ouinet-injector.conf`.
360

361
362
363
364
365
366
367
368
369
370
To start the injector in headless mode, you can run:

    $ sudo docker-compose up -d

You will need to use `sudo docker-compose stop` to stop the container.

To be able to follow its logs, you can run:

    $ sudo docker-compose logs --tail=100 -ft

371
## Testing (desktop)
372

373
### Running a test injector
374

375
376
377
If you want to run your own injector for testing and you have a local build,
create a copy of the `repos/injector` repository template directory included
in Ouinet's source tree:
378

379
    $ cp -r <SOURCE DIR>/repos/injector /path/to/injector-repo
380

381
382
383
384
385
When using a Docker-based injector as described above, just run and stop it so
that it creates a default configuration for you.

You should now edit `ouinet-injector.conf` in the injector repository (for
Docker, use the shell container to edit `injector/ouinet-injector.conf`):
386

387
 1. Enable listening on loopback addresses:
388

389
        listen-tcp = ::1:7070
390

391
392
    For clients you may then use `127.0.0.1:7070` as the *injector endpoint*
    (IPv6 is not yet supported).
393

394
 2. Change the credentials to use the injector (use your own ones):
395

396
        credentials = injector_user:injector_password
397

398
    For clients you may use these as *injector credentials*.
399

400
All the steps above only need to be done once.
401

402
403
Finally start the injector.  For the local build you will need to explicitly
point it to the repository created above:
Peter Jankuliak's avatar
Peter Jankuliak committed
404

405
    $ <BUILD DIR>/injector --repo /path/to/injector-repo
406
407
    ...
    [INFO] HTTP signing public key (Ed25519): <CACHE_PUB_KEY>
Peter Jankuliak's avatar
Peter Jankuliak committed
408
409
    ...

410
Note down the `<CACHE_PUB_KEY>` string in the above output since clients will
411
412
need it as the *public key for HTTP signatures*.  You may also find that value
in the `ed25519-public-key` file in the injector repository.
Peter Jankuliak's avatar
Peter Jankuliak committed
413

414
415
When you are done testing the Ouinet injector, you may shut it down by hitting
Ctrl+C.
Peter Jankuliak's avatar
Peter Jankuliak committed
416

417
### Running a test client
418

419
420
421
422
423
To perform some tests using a Ouinet client and an existing test injector, you
first need to know the *injector endpoint* and *credentials*, its *TLS
certificate*, and its *public key for HTTP signatures*.  These use to be
respectively a `tcp:<IP>:<PORT>` string, a `<USER>:<PASSWORD>` string, a path
to a PEM file, and an Ed25519 public key (hexadecimal or Base32).
424

425
You need to configure the Ouinet client to use the aforementioned parameters.
426
427
428
429
If you have a local build, create a copy of the `repos/client` repository
template directory included in Ouinet's source tree:

    $ cp -r <SOURCE DIR>/repos/client /path/to/client-repo
430

431
432
When using a Docker-based client as described above, just run and stop it so
that it creates a default configuration for you.
433

434
435
Now edit `ouinet-client.conf` in the client repository (for Docker, use the
shell container to edit `client/ouinet-client.conf`) and add options for the
436
injector endpoint (if testing), credentials and public key.  Remember to
437
438
replace the values with your own:

439
    injector-ep = tcp:127.0.0.1:7070
440
    injector-credentials = injector_user:injector_password
441
442
    cache-http-public-key = 00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff
    cache-type = bep5-http
Peter Jankuliak's avatar
Peter Jankuliak committed
443

444
All the steps above only need to be done once.
Peter Jankuliak's avatar
Peter Jankuliak committed
445

Jenny Ryan's avatar
Jenny Ryan committed
446
Finally, start the client.  For the local build you will need to explicitly
447
point it to the repository created above:
Peter Jankuliak's avatar
Peter Jankuliak committed
448

449
    $ <BUIDL DIR>/client --repo /path/to/client-repo
Peter Jankuliak's avatar
Peter Jankuliak committed
450

451
The client opens a web proxy on local port 8077 by default (see option
452
453
454
455
`listen-on-tcp` in its configuration file).  When you access the web using
this proxy (see the following section), your requests will go through your
local Ouinet client, which will attempt several mechanisms supported by Ouinet
to retrieve the resource.
Peter Jankuliak's avatar
Peter Jankuliak committed
456

457
458
When you are done testing the Ouinet client, you may shut it down by hitting
Ctrl+C.
Peter Jankuliak's avatar
Peter Jankuliak committed
459

460
### Testing the client with a browser
Peter Jankuliak's avatar
Peter Jankuliak committed
461

462
463
464
Once your local Ouinet client is running (see above), if you have Firefox
installed, you can create a new profile (stored under the `ff-profile`
directory in the example below) which uses the Ouinet client as an HTTP proxy
465
(listening on `localhost:8077` here) by executing the following commands on
466
another shell:
Peter Jankuliak's avatar
Peter Jankuliak committed
467

468
    mkdir -p ff-profile
469
    env http_proxy='http://localhost:8077/' firefox --no-remote --profile ff-profile
Peter Jankuliak's avatar
Peter Jankuliak committed
470

471
472
Otherwise you may manually [modify your browser's settings][Firefox proxy] to:

473
  - Make the client (listening on port `localhost:8077` here) its HTTP proxy
474
475
476
477
478
479
  - Make sure that `localhost` is not listed in the *No Proxy for* field
  - Check *Use this proxy for all protocols* (mostly for HTTPS)

[Firefox proxy]: http://www.wikihow.com/Enter-Proxy-Settings-in-Firefox
    "How to Enter Proxy Settings in Firefox"

480
481
482
483
484
Additionally, you may want to disable Firefox's automatic captive portal
detection so that you get less noise in client logs.  Enter `about:config` in
the location bar and change `network.captive-portal-service.enabled` to
`false`.

485
486
487
488
489
490
Once done, you can visit `localhost` in your browser and it should show you
the *client front-end* with assorted information from the client and
configuration tools:

  - To be able to browse HTTPS sites, you must first install the
    *client-specific CA certificate* linked from the top of the front-end page
491
492
493
494
495
496
497
498
499
    and authorize it to identify web sites.  Depending on your browser
    version, you may need to save it to disk first, then import it from
    *Preferences / Privacy & Security / Certificates / View Certificates…*
    into the *Authorities* list.

    The Ouinet client acts as a *man in the middle* to enable it to process
    HTTPS requests, but it (or a trusted injector when appropriate) still
    performs all standard certificate validations.  This CA certificate is
    unique to your device.
500
501
502

  - Several buttons near the top of the page look something like this:

503
        Injector access: enabled [ disable ]
504
505
506
507
508
509
510

    They allow you to enable or disable different *request mechanisms* to
    retrieve content:

      - *Origin*: The client contacts the origin server directly via HTTP(S).
      - *Proxy*: The client contacts the origin server through an HTTP proxy
        (currently the configured injector).
511
512
513
514
515
      - *Injector*: The client asks the injector to fetch and sign the content
        from the origin server, then it starts seeding the signed content to
        the distributed cache.
      - *Distributed Cache*: The client attempts to retrieve the content from
        the distributed cache.
516
517
518
519
520
521
522
523
524
525

    Content retrieved via the Origin and Proxy mechanisms is considered
    *private and not seeded* to the distributed cache.  Content retrieved via
    the Injector and Cache mechanisms is considered *public and seeded* to the
    distributed cache.

    These mechanisms are attempted in order according to a (currently
    hard-wired, customizable in the future) *request router configuration*.
    For instance, if one points the browser to a web page which it is not yet
    in the distributed cache, then the client shall forward the request to the
526
527
528
    injector.  On success, (A) the injector will fetch, sign and send the
    content back to the client and (B) the client will seed the content to the
    cache.
529

530
  - Other information about the cache index is shown next.
531

532
533
534
535
536
537
538
**Note:** For a response to be injected, its request currently needs to carry
an `X-Ouinet-Group` header.  The [CENO Extension][] takes care of that
whenever browsing in normal mode, and it does not when browsing in private
mode.  Unfortunately, the Extension is not yet packaged independently and the
only way to use it is to clone its repository locally and load it every time
you start the browser; to do that, open Firefox's *Add-ons* window, then click
on the gears icon, then *Debug Add-ons*, then *Load Temporary Add-on…* and
539
540
541
choose the `manifest.json` file in the Extension's source tree.  Back to the
*Add-ons* page, remember clicking on *CENO Extension* and allowing *Run in
Private Windows* under *Details*.
542
543
544

[CENO Extension]: https://github.com/censorship-no/ceno-ext-settings/

545
546
547
548
549
550
After visiting a page with the Origin mechanism disabled and Injector
mechanism enabled, and waiting for a short while, you should be able to
disable all request mechanisms except for the Cache, clear the browser's
cached data, point the browser back to the same page and still get its
contents from the distributed cache even when the origin server is completely
unreachable.
Peter Jankuliak's avatar
Peter Jankuliak committed
551

552
## Android library and demo client
553

554
Ouinet can also be built as an Android Archive library (AAR) to use in your
555
Android apps.
556

557
558
559
### Build requirements

A lot of free space (something less than 15 GiB).  Everything else shall be
560
561
562
downloaded by the `build-android.sh` script.

The instructions below use Vagrant for bulding, but the `build-android.sh`
563
script should work on any reasonably up-to-date Debian based system.
564

565
566
567
In the following instructions, we will use `<ANDROID>` to represent the
absolute path to your build directory.  That is, the directory from which you
will run the `build-android.sh` script (e.g. `~/ouinet.android.build`).
568
569
570

### Building

571
572
573
The following instructions will by build a Ouinet AAR library and demo client
APK package for the `armeabi-v7a` [Android ABI][]:

Peter Jankuliak's avatar
Peter Jankuliak committed
574
575
576
577
578
579
    host    $ vagrant up --provider=libvirt
    host    $ vagrant ssh
    vagrant $ mkdir <ANDROID>
    vagrant $ cd <ANDROID>
    vagrant $ git clone --recursive /vagrant
    vagrant $ ./vagrant/scripts/build-android.sh
580

581
582
583
584
585
586
587
588
589
590
591
Note that we cloned a fresh copy of the Ouinet repository at `/vagrant`.  This
is not strictly necessary since the build environment supports out-of-source
builds, however it spares you from having to keep your source directory clean
and submodules up to date at the host.  If you fullfill these requirements,
you can just skip the cloning and run `/vagrant/scripts/build-android.sh`
instead.

If you want a build for a different ABI, do set the `ABI` environment
variable:

    vagrant $ env ABI=x86_64 /path/to/build-android.sh
592

593
In any case, when the build script finishes successfully, it will leave the
Sarah Fortune's avatar
Sarah Fortune committed
594
Ouinet AAR library at `build.ouinet/build-android-$ABI/builddir/ouinet/build-android/outputs/aar/ouinet-debug.aar`.
595
596
597

[Android ABI]: https://developer.android.com/ndk/guides/abis.html

598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
#### Using existing Android SDK/NDK and Boost

By default the `build-android.sh` script downloads all dependencies required
to build the Ouinet Android library, including the Android SDK, Android NDK
and Boost for Android.  If you already have these installed on your system you
can tune the script to use them:

    $ export SDK_DIR=/opt/android-sdk
    $ export NDK_DIR=/opt/android-sdk/ndk-bundle
    $ export ABI=armeabi-v7a
    $ export PLATFORM=android-26
    $ export BOOST_V=1_67_0
    $ export BOOST_SOURCE=/path/to/Boost-for-Android
    $ export BOOST_INCLUDEDIR=$BOOST_SOURCE/build/out/${ABI}/include/boost-${BOOST_V}
    $ /path/to/build-android.sh
613

Peter Jankuliak's avatar
Peter Jankuliak committed
614
### Testing with Android emulator
615
616
617
618

You may also use the `build-android.sh` script to fire up an Android emulator
session with a compatible system image; just run:

619
    host $ /path/to/build-android.sh emu
620
621
622
623
624
625
626
627

It will download the necessary files to the current directory (or reuse files
downloaded by the build process, if available) and start the emulator.  Please
note that downloading the system image may take a few minutes, and booting the
emulator for the first time may take more than 10 minutes.  In subsequent
runs, the emulator will just recover the snapshot saved on last quit, which is
way faster.

628
629
630
The `ABI` environment variable described above also works for selecting the
emulator architecture:

631
    host $ env ABI=x86_64 /path/to/build-android.sh emu
632

633
634
635
You may pass options to the emulator at the script's command line, after a
`--` (double dash) argument.  For instance:

636
    host $ /path/to/build-android.sh emu -- -no-snapshot-save
637
638
639
640
641
642
643

Some useful options include `-no-snapshot`, `-no-snapshot-load` and
`-no-snapshot-save`.  See [emulator startup options][] for more information.

[emulator startup options]: https://developer.android.com/studio/run/emulator-commandline.html#startup-options

While the emulator is running, you may interact with it using ADB, e.g. to
644
645
install the APK built previously.  See the script's output for particular
instructions and paths.
646

647
### Integrating the Ouinet library into your app
648

649
650
651
In order for your Android app to access the resources it needs using the HTTP
protocol over Ouinet, thus taking advantage of its caching and distributed
request handling, you need to take few simple steps.
652

653
654
Here we assume that the app is developed in the Android Studio environment,
and that `<PROJECT DIR>` is your app's project directory.
655

656
657
658
659
First you need to compile the Ouinet library for the ABI environment your are
aiming at (e.g. `armeabi-v7a` or `x86_64`) as described above.  After the
`build_android.sh` script finishes successfully, you can copy the
`ouinet-debug.aar` file to your app libs folder:
660

661
    $ cp /path/to/ouinet-debug.aar <PROJECT DIR>/app/libs/
662

663
Then look for the following section of your `<PROJECT DIR>/build.gradle`:
664
665
666

    allprojects {
      repositories {
Sarah Fortune's avatar
Sarah Fortune committed
667
        ...
668
669
      }
    }
670
671

And add this:
672
673
674
675
676

    flatDir {
      dirs 'libs'
    }

677
Then look for the following section of your `<PROJECT DIR>/app/build.gradle`:
678
679

    dependencies {
680
      ...
681
682
    }

683
And add this:
684

685
    implementation(name:'ouinet-debug', ext:'aar')
686

687
688
689
At this stage your project should compile with no errors.  Now to tell Ouinet
to take over the app's HTTP communications, in the `MainActivity.java` of your
app import Ouinet:
690

691
    import ie.equalit.ouinet.Ouinet;
692

693
Then add a private member to your `MainActivity` class:
694

Sarah Fortune's avatar
Sarah Fortune committed
695
    private Ouinet ouinet;
696

697
698
And in its `OnCreate` method initiate the Ouinet object (using the BEP5/HTTP
cache):
699

Sarah Fortune's avatar
Sarah Fortune committed
700
    Config config = new Config.ConfigBuilder(this)
701
702
                .setCacheType("bep5-http")
                .setCacheHttpPubKey(<CACHE_PUB_KEY>)
Sarah Fortune's avatar
Sarah Fortune committed
703
704
705
706
                .setInjectorCredentials(<INJECTOR_USERNAME>:<INJECTOR_PASSWORD>)
                .setInjectorTlsCert(<INJECTOR_TLS_CERT>)
                .setTlsCaCertStorePath(<TLS_CA_CERT_STORE_PATH>)
                .build()
707

Sarah Fortune's avatar
Sarah Fortune committed
708
709
    ouinet = new Ouinet(this, config);
    ouinet.start();
710

711
From now on all of the app's HTTP communication will be handled by Ouinet.