Commit 518e6e83 authored by Ivan Vilata-i-Balaguer's avatar Ivan Vilata-i-Balaguer
Browse files

Merge branch 'http-range-vuln'.

This fixes the Ouinet protocol (at HTTP response signing) to avoid a
vulnerability where verifying data block signatures in a partial
response (i.e. with a range) from another client would work even in the client
sent a correct series of blocks, but from a different range start.

Protocol bumped to version 4.
parents 4b3dc091 2259237d
......@@ -35,7 +35,7 @@ When the injector gets an injection request from the client, it gets the respons
This signature is provided so that the partial response (head an body) can still be useful if the connection is interrupted later on.
The injector then sends data blocks of the size specified above (the last one may be smaller), each of them followed by a **data block signature** bound to its own data, previous blocks' data, and injection. The client need not check the signatures but it can save them to provide them to other clients in case the connection to the injector is interrupted.
The injector then sends data blocks of the size specified above (the last one may be smaller), each of them followed by a **data block signature** bound to its injection, offset, previous blocks' data, and its own data. The client need not check the signatures but it can save them to provide them to other clients in case the connection to the injector is interrupted.
When all data blocks have been sent to the client, the injector sends additional headers to build the **final response head** including:
......@@ -54,7 +54,7 @@ Please note that neither the partial signature nor framing headers (`Transfer-En
```
HTTP/1.1 200 OK
X-Ouinet-Version: 3
X-Ouinet-Version: 4
X-Ouinet-URI: https://example.com/foo
X-Ouinet-Injection: id=d6076384-2295-462b-a047-fe2c9274e58d,ts=1516048310
Date: Mon, 15 Jan 2018 20:31:50 GMT
......@@ -70,11 +70,11 @@ Trailer: Digest, X-Ouinet-Data-Size, X-Ouinet-Sig1
80000
0123456789...
80000;ouisig=BASE64(BSIG(d607…e58d NUL HASH[0]=SHA2-512(BLOCK[0])))
80000;ouisig=BASE64(BSIG(d607…e58d NUL 0 NUL HASH[0]=SHA2-512(BLOCK[0])))
0123456789...
4;ouisig=BASE64(BSIG(d607…e58d NUL HASH[1]=SHA2-512(HASH[0] BLOCK[1])))
4;ouisig=BASE64(BSIG(d607…e58d NUL 1048576 NUL HASH[1]=SHA2-512(HASH[0] BLOCK[1])))
abcd
0;ouisig=BASE64(BSIG(d607…e58d NUL HASH[2]=SHA2-512(HASH[1] BLOCK[2])))
0;ouisig=BASE64(BSIG(d607…e58d NUL 2097152 NUL HASH[2]=SHA2-512(HASH[1] BLOCK[2])))
Digest: SHA-256=BASE64(SHA2-256(FULL_BODY))
X-Ouinet-Data-Size: 1048580
X-Ouinet-Sig1: keyId="ed25519=????",algorithm="hs2019",created=1516048311,
......@@ -84,12 +84,16 @@ X-Ouinet-Sig1: keyId="ed25519=????",algorithm="hs2019",created=1516048311,
The signature for a given block comes in a chunk extension in the chunk right after the block's end (for the last block, in the final chunk); if the signature was placed at the beginning of the block, the injector would need to buffer the whole block in memory before sending the corresponding chunks.
The signature string for each block covers the following values (separated by a null character):
The signature string for each block covers the following values (separated by null characters):
- The injection identifier (string).
This helps avoid replay attacks where the attacker sends correctly signed but different blocks from a different injection (for the same or a different URI).
- The offset (decimal, no padding).
This helps detecting an attacker which replies to a range request with a range of the expected length, with correctly signed and ordered blocks, that however starts at the wrong offset.
- A **chain hash** (binary) computed from the chain hash of the previous block and the data of the block itself: for the i-th block, `HASH[i]=SHA2-512(HASH[i-1] BLOCK[i])`, with `HASH[0]=SHA2-512(BLOCK[0])`.
Signing the hash instead of block data itself spares the signer from keeping the whole block in memory for producing the signature (the hash algorithm can be fed as data comes in from the origin).
......@@ -106,11 +110,13 @@ If a client got to get and save a full response from the injector, it may send t
## Range requests
If a client sends an HTTP range request to another client, the later aligns it to block boundaries (this is acceptable according to [RFC7233#4.1][] — "a client cannot rely on receiving the same ranges that it requested"). The `Range:` header in the response is not part of the partial nor final signatures. If the range does not start at the beginning of the data, the first block `i` is accompanied with a `ouihash=BASE64(HASH[i-1])` chunk extension to enable checking its `ouisig`. Please note that to ease serving range requests, a client storing a response may cache all chain hashes along their blocks, so as to avoid having to compute the `ouihash` of the first block in the range.
If a client sends an HTTP range request to another client, the later aligns it to block boundaries (this is acceptable according to [RFC7233#4.1][] — "a client cannot rely on receiving the same ranges that it requested"). The `Content-Range:` header in the response is not part of the partial nor final signatures. If the range does not start at the beginning of the data, the first block `i` is accompanied with a `ouihash=BASE64(HASH[i-1])` chunk extension to enable checking its `ouisig`. Please note that to ease serving range requests, a client storing a response may cache all chain hashes along their blocks, so as to avoid having to compute the `ouihash` of the first block in the range.
[RFC7233#4.1]: https://tools.ietf.org/html/rfc7233#section-4.1
HTTP range requests from client to injector may not be supported since the injector would need to download all data from the beginning to compute an initial `ouihash`. This could be abused to make the injector use resources by asking for the last block of a big file. At any rate, in such an injection, `Digest:` and `X-Ouinet-Data-Size:` may be missing in the final response head, if the injector did not have access to the whole body data. Also, the (aligned) `Range` header would never be signed to allow later sharing of different subranges, which can be validated independently anyway.
Also note that partial responses mush have a `206 Partial Content` status, which would break signature verification. To avoid this issue, the original response status code (usually `200`) is saved to the `X-Ouinet-HTTP-Status` header, and head verification should automatically replace the `206` status (if so allowed) with the saved one (only for verification purposes).
HTTP range requests from client to injector may not be supported since the injector would need to download all data from the beginning to compute an initial `ouihash`. This could be abused to make the injector use resources by asking for the last block of a big file. At any rate, in such an injection, `Digest:` and `X-Ouinet-Data-Size:` may be missing in the final response head, if the injector did not have access to the whole body data. Also, the (aligned) `Content-Range:` header would never be signed to allow later sharing of different subranges, which can be validated independently anyway.
## Issues
......
......@@ -61,9 +61,9 @@ http_injection_head( const http::request_header<>& rqh
{
using namespace ouinet::http_;
// TODO: This should be a `static_assert`.
assert(protocol_version_hdr_current == protocol_version_hdr_v3);
assert(protocol_version_hdr_current == protocol_version_hdr_v4);
rsh.set(protocol_version_hdr, protocol_version_hdr_v3);
rsh.set(protocol_version_hdr, protocol_version_hdr_v4);
rsh.set(response_uri_hdr, rqh.target());
rsh.set(response_injection_hdr
, boost::format("id=%s,ts=%d") % injection_id % injection_ts);
......@@ -353,11 +353,13 @@ block_sig_from_exts(boost::string_view xs)
static
std::string
block_sig_str( boost::string_view injection_id
, size_t block_offset
, const block_digest_t& block_digest)
{
static const auto fmt_ = "%s%c%s";
static const auto fmt_ = "%s%c%d%c%s";
return ( boost::format(fmt_)
% injection_id % '\0'
% block_offset % '\0'
% util::bytes::to_string_view(block_digest)).str();
}
......@@ -386,10 +388,11 @@ block_chunk_ext( const opt_sig_array_t& sig
static
std::string
block_chunk_ext( boost::string_view injection_id
, size_t offset
, const block_digest_t& digest
, const util::Ed25519PrivateKey& sk)
{
auto sig_str = block_sig_str(injection_id, digest);
auto sig_str = block_sig_str(injection_id, offset, digest);
return block_chunk_ext(sk.sign(sig_str));
}
......@@ -642,6 +645,7 @@ struct SigningReader::Impl {
size_t body_length = 0;
size_t block_offset = 0;
size_t block_size_last = 0;
util::SHA256 body_hash;
util::SHA512 block_hash; // for first block
// Simplest implementation: one output chunk per data block.
......@@ -670,13 +674,15 @@ struct SigningReader::Impl {
if (do_inject) { // if injecting and sending data
if (block_offset > 0) { // add chunk extension for previous block
auto block_digest = block_hash.close();
ch.exts = block_chunk_ext(injection_id, block_digest, sk);
ch.exts = block_chunk_ext( injection_id
, block_offset - block_size_last
, block_digest, sk);
// Prepare chunk extension for next block: HASH[i]=SHA2-512(HASH[i-1] BLOCK[i])
block_hash = {};
block_hash.update(block_digest);
} // else HASH[0]=SHA2-512(BLOCK[0])
block_hash.update(block_buf);
block_offset += block_buf.size();
block_offset += (block_size_last = block_buf.size());
}
return http_response::Part(std::move(ch)); // pass data on, drop origin extensions
}
......@@ -710,7 +716,9 @@ struct SigningReader::Impl {
auto block_digest = block_hash.close();
auto last_ch = http_response::ChunkHdr(
0, block_chunk_ext(injection_id, block_digest, sk));
0, block_chunk_ext( injection_id
, block_offset - block_size_last
, block_digest, sk));
auto trailer = cache::http_injection_trailer( outh, std::move(trailer_in)
, body_length, body_hash.close()
, sk
......@@ -1103,7 +1111,7 @@ struct VerifyingReader::Impl {
// Complete hash for the data block; note that HASH[0]=SHA2-512(BLOCK[0])
block_hash.update(block_buf);
auto block_digest = block_hash.close();
auto bsig_str = block_sig_str(injection_id, block_digest);
auto bsig_str = block_sig_str(injection_id, block_offset, block_digest);
if (!bs_params->pk.verify(bsig_str, *block_sig)) {
LOG_WARN("Failed to verify data block with offset ", block_offset, "; uri=", uri);
return or_throw(y, sys::errc::make_error_code(sys::errc::bad_message), boost::none);
......
......@@ -32,8 +32,9 @@ static const std::string protocol_version_hdr_v0 = "0";
static const std::string protocol_version_hdr_v1 = "1";
static const std::string protocol_version_hdr_v2 = "2";
static const std::string protocol_version_hdr_v3 = "3";
static const std::string protocol_version_hdr_current = protocol_version_hdr_v3;
static const unsigned protocol_version_current = 3;
static const std::string protocol_version_hdr_v4 = "4";
static const std::string protocol_version_hdr_current = protocol_version_hdr_v4;
static const unsigned protocol_version_current = 4;
// The presence of this HTTP request header
// indicates that an error happened processing the request,
......
......@@ -96,7 +96,7 @@ static const string _rs_fields_origin = (
);
static const string _rs_head_injection = (
"X-Ouinet-Version: 3\r\n"
"X-Ouinet-Version: 4\r\n"
"X-Ouinet-URI: https://example.com/foo\r\n"
"X-Ouinet-Injection: id=d6076384-2295-462b-a047-fe2c9274e58d,ts=1516048310\r\n"
"X-Ouinet-BSigs: keyId=\"ed25519=DlBwx8WbSsZP7eni20bf5VKUH3t1XAF/+hlDoLbZzuw=\","
......@@ -109,7 +109,7 @@ static const string _rs_head_sig0 = (
"headers=\"(response-status) (created) "
"date server content-type content-disposition "
"x-ouinet-version x-ouinet-uri x-ouinet-injection x-ouinet-bsigs\","
"signature=\"tnVAAW/8FJs2PRgtUEwUYzMxBBlZpd7Lx3iucAt9q5hYXuY5ci9T7nEn7UxyKMGA1ZvnDMDBbs40dO1OQUkdCA==\"\r\n"
"signature=\"UvcvmTPLGnmG3Bk2xdIBZ2Mw5V6enCXqyS3jReRev/o7ZvtKrSujnyHUEpHQ3pM+axfjw1vAznE4+mhMXTVdAg==\"\r\n"
);
static const string _rs_head_framing = (
......@@ -130,7 +130,7 @@ static const string _rs_head_sig1 = (
"x-ouinet-version x-ouinet-uri x-ouinet-injection x-ouinet-bsigs "
"x-ouinet-data-size "
"digest\","
"signature=\"h/PmOlFvScNzDAUvV7tLNjoA0A39OL67/9wbfrzqEY7j47IYVe1ipXuhhCfTnPeCyXBKiMlc4BP+nf0VmYzoAw==\"\r\n"
"signature=\"nDUm3W0OCeygFTdVoH/6mEKt9S7xIL/EESCEFKNGxJy5zepJQjW38p3QUqycvZuc058vEuRa/CRLDdhc/KW7Ag==\"\r\n"
);
static const string rs_head_signed_s =
......@@ -150,9 +150,9 @@ static const array<string, 3> rs_block_hash_cx{
};
static const array<string, 3> rs_block_sig_cx{
";ouisig=\"AwiYuUjLYh/jZz9d0/ev6dpoWqjU/sUWUmGL36/D9tI30oaqFgQGgcbVCyBtl0a7x4saCmxRHC4JW7cYEPWwCw==\"",
";ouisig=\"c+ZJUJI/kc81q8sLMhwe813Zdc+VPa4DejdVkO5ZhdIPPojbZnRt8OMyFMEiQtHYHXrZIK2+pKj2AO03j70TBA==\"",
";ouisig=\"m6sz1NpU/8iF6KNN6drY+Yk361GiW0lfa0aaX5TH0GGW/L5GsHyg8ozA0ejm29a+aTjp/qIoI1VrEVj1XG/gDA==\"",
";ouisig=\"6gCnxL3lVHMAMSzhx+XJ1ZBt+JC/++m5hlak1adZMlUH0hnm2S3ZnbwjPQGMm9hDB45SqnybuQ9Bjo+PgnfnCw==\"",
";ouisig=\"647D/5afXUjP8jBWyfDQX2QTtLdshyawchxKm3eqhyJPC98DLcFbyC8ir8yciYgtPyN3yl7q88AwoMb7qURsBw==\"",
";ouisig=\"PAgvnzE20ypASNvxPbd/iBleipxmjJMD5cGxv0CbUjI/lsRlTdfNWDAXsb0V4a40ExkWqZc9Pe++2ZhQwRNMAQ==\"",
};
static const array<string, 4> rs_chunk_ext{
......@@ -222,7 +222,7 @@ BOOST_AUTO_TEST_CASE(test_http_sign) {
const auto sk = get_private_key();
const auto key_id = cache::http_key_id_for_injection(sk.public_key());
BOOST_REQUIRE(key_id == ("ed25519=" + inj_b64pk));
BOOST_REQUIRE_EQUAL(key_id, ("ed25519=" + inj_b64pk));
rs_head = cache::http_injection_head(req_h, std::move(rs_head), inj_id, inj_ts, sk, key_id);
......@@ -236,7 +236,7 @@ BOOST_AUTO_TEST_CASE(test_http_sign) {
std::stringstream rs_head_ss;
rs_head_ss << rs_head;
BOOST_REQUIRE(rs_head_ss.str() == rs_head_signed_s);
BOOST_REQUIRE_EQUAL(rs_head_ss.str(), rs_head_signed_s);
}
......
......@@ -67,7 +67,7 @@ static const string _rs_head_origin =
+ _rs_fields_origin);
static const string _rs_head_injection = (
"X-Ouinet-Version: 3\r\n"
"X-Ouinet-Version: 4\r\n"
"X-Ouinet-URI: https://example.com/foo\r\n"
"X-Ouinet-Injection: id=d6076384-2295-462b-a047-fe2c9274e58d,ts=1516048310\r\n"
"X-Ouinet-BSigs: keyId=\"ed25519=DlBwx8WbSsZP7eni20bf5VKUH3t1XAF/+hlDoLbZzuw=\","
......@@ -80,7 +80,7 @@ static const string _rs_head_sig0 = (
"headers=\"(response-status) (created) "
"date server content-type content-disposition "
"x-ouinet-version x-ouinet-uri x-ouinet-injection x-ouinet-bsigs\","
"signature=\"tnVAAW/8FJs2PRgtUEwUYzMxBBlZpd7Lx3iucAt9q5hYXuY5ci9T7nEn7UxyKMGA1ZvnDMDBbs40dO1OQUkdCA==\"\r\n"
"signature=\"UvcvmTPLGnmG3Bk2xdIBZ2Mw5V6enCXqyS3jReRev/o7ZvtKrSujnyHUEpHQ3pM+axfjw1vAznE4+mhMXTVdAg==\"\r\n"
);
static const string _rs_head_framing = (
......@@ -108,7 +108,7 @@ static const string _rs_head_sig1 = (
"x-ouinet-version x-ouinet-uri x-ouinet-injection x-ouinet-bsigs "
"x-ouinet-data-size "
"digest\","
"signature=\"h/PmOlFvScNzDAUvV7tLNjoA0A39OL67/9wbfrzqEY7j47IYVe1ipXuhhCfTnPeCyXBKiMlc4BP+nf0VmYzoAw==\"\r\n"
"signature=\"nDUm3W0OCeygFTdVoH/6mEKt9S7xIL/EESCEFKNGxJy5zepJQjW38p3QUqycvZuc058vEuRa/CRLDdhc/KW7Ag==\"\r\n"
);
static const string rs_trailer =
......@@ -139,9 +139,9 @@ static const array<string, 3> rs_block_hash{
};
static const array<string, 3> rs_block_sig{
"AwiYuUjLYh/jZz9d0/ev6dpoWqjU/sUWUmGL36/D9tI30oaqFgQGgcbVCyBtl0a7x4saCmxRHC4JW7cYEPWwCw==",
"c+ZJUJI/kc81q8sLMhwe813Zdc+VPa4DejdVkO5ZhdIPPojbZnRt8OMyFMEiQtHYHXrZIK2+pKj2AO03j70TBA==",
"m6sz1NpU/8iF6KNN6drY+Yk361GiW0lfa0aaX5TH0GGW/L5GsHyg8ozA0ejm29a+aTjp/qIoI1VrEVj1XG/gDA==",
"6gCnxL3lVHMAMSzhx+XJ1ZBt+JC/++m5hlak1adZMlUH0hnm2S3ZnbwjPQGMm9hDB45SqnybuQ9Bjo+PgnfnCw==",
"647D/5afXUjP8jBWyfDQX2QTtLdshyawchxKm3eqhyJPC98DLcFbyC8ir8yciYgtPyN3yl7q88AwoMb7qURsBw==",
"PAgvnzE20ypASNvxPbd/iBleipxmjJMD5cGxv0CbUjI/lsRlTdfNWDAXsb0V4a40ExkWqZc9Pe++2ZhQwRNMAQ==",
};
static const array<string, 4> rs_chunk_ext{
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment