You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
498 lines
22 KiB
498 lines
22 KiB
-----------------------------------------------
|
|
HTX API
|
|
Version 1.0
|
|
( Last update: 2019-06-20 )
|
|
-----------------------------------------------
|
|
Author : Christopher Faulet
|
|
Contact : cfaulet at haproxy dot com
|
|
|
|
1. Background
|
|
|
|
Historically, HAProxy stored HTTP messages in a raw fashion in buffers, keeping
|
|
parsing information separately in a "struct http_msg" owned by the stream. It was
|
|
optimized to the data transfer, but not so much for rewrites. It was also HTTP/1
|
|
centered. While it was the only HTTP version supported, it was not a
|
|
problem. But with the rise of HTTP/2, it starts to be hard to still use this
|
|
representation.
|
|
|
|
At the first age of the HTTP/2 in HAProxy, H2 messages were converted into
|
|
H1. This was terribly unefficient because it required two parsing passes, a
|
|
first one in H2 and a second one in H1, with a conversion in the middle. And of
|
|
course, the same was also true in the opposite direction. outgoing H1 messages
|
|
had to be converted back in H2 to be sent. Even worse, because the H2->H1
|
|
conversion, only client H2 connections were supported.
|
|
|
|
So, to address all these problems, we decided to replace the old raw
|
|
representation by a version-agnostic and self-structured internal HTTP
|
|
representation, the HTX. As an additional benefit, with this new representation,
|
|
the message parsing and its processing are now separated, making all the HTTP
|
|
analysis simpler and cleaner. The parsing of HTTP messages is now handled by
|
|
the multiplexers (h1 or h2).
|
|
|
|
|
|
2. The HTX message
|
|
|
|
The HTX is a structure containing useful information about an HTTP message
|
|
followed by a contiguous array with some parts of the message. These parts are
|
|
called blocks. A block is composed of metadata (htx_blk) and an associated
|
|
payload. Blocks' metadata are stored starting from the end of the array while
|
|
their payload are stored at the beginning. Blocks' metadata are often simply
|
|
called blocks. it is a misuse of language that's simplify explanations.
|
|
|
|
Internally, this structure is "hidden" in a buffer. This way, there are few
|
|
changes into intermediate layers (stream-interface and channels). They still
|
|
manipulate buffers. Only the multiplexer and the stream have to know how data
|
|
are really stored. From the HTX perspective, a buffer is just a memory
|
|
area. When an HTX message is stored in a buffer, this one appears as full.
|
|
|
|
* General view of an HTX message :
|
|
|
|
|
|
buffer->area
|
|
|
|
|
|<------------ buffer->size == buffer->data ----------------------|
|
|
| |
|
|
| |<------------- Blocks array (htx->size) ------------------>|
|
|
V | |
|
|
+-----+-----------------+-------------------------+---------------+
|
|
| HTX | PAYLOADS ==> | | <== HTX_BLKs |
|
|
+-----+-----------------+-------------------------+---------------+
|
|
| | | |
|
|
|<-payloads part->|<----- free space ------>|<-blocks part->|
|
|
(htx->data)
|
|
|
|
|
|
The blocks part remains linear and sorted. You may think about it as an array
|
|
with negative indexes. But, instead of using negative indexes, we use positive
|
|
positions to identify a block. This position is then converted to an address
|
|
relatively to the beginning of the blocks array.
|
|
|
|
tail head
|
|
| |
|
|
V V
|
|
.....--+----+-----------------------+------+------+
|
|
| Bn | ... | B1 | B0 |
|
|
.....--+----+-----------------------+------+------+
|
|
^ ^ ^
|
|
Addr of the block Addr of the block Addr of the block
|
|
at the position N at the position 1 at the position 0
|
|
|
|
|
|
In the HTX structure, 3 "special" positions are stored:
|
|
|
|
- tail : Position of the newest inserted block
|
|
- head : Position of the oldest inserted block
|
|
- first : Position of the first block to (re)start the analyse
|
|
|
|
The blocks part never wrap. If we have no space to allocate a new block and if
|
|
there is a hole at the beginning of the blocks part (so at the end of the blocks
|
|
array), we move back all blocks.
|
|
|
|
|
|
tail head tail head
|
|
| | | |
|
|
V V V V
|
|
...+--------------+---------+ blocks ...----------+--------------+
|
|
| X== HTX_BLKS | | defrag | <== HTX_BLKS |
|
|
...+--------------+---------+ =====> ...----------+--------------+
|
|
|
|
|
|
The payloads part is a raw space that may wrap. You never access to a block's
|
|
payload directly. Instead you get a block to retrieve the address of its
|
|
payload.
|
|
|
|
|
|
+------------------------( B0.addr )--------------------------+
|
|
| +-------------------( B1.addr )----------------------+ |
|
|
| | +-----------( B2.addr )----------------+ | |
|
|
V V V | | |
|
|
+-----+----+-------+----+--------+-------------+-------+----+----+----+
|
|
| HTX | P0 | P1 | P2 | ...==> | | <=... | B2 | B1 | B0 |
|
|
+-----+----+-------+----+--------+-------------+-------+----+----+----+
|
|
|
|
|
|
Because the payloads part may wrap, there are 2 usable free spaces:
|
|
|
|
- The free space in front of the blocks part. This one is used iff the other
|
|
one was not used yet.
|
|
|
|
- The free space at the beginning of the message. Once this one is used, the
|
|
other one is never used again, until a message defragmentation.
|
|
|
|
|
|
* Linear payloads part :
|
|
|
|
|
|
head_addr end_addr tail_addr
|
|
| | |
|
|
V V V
|
|
+-----+--------------------+-------------+--------------------+-------...
|
|
| HTX | | PAYLOADS | | HTX_BLKs
|
|
+-----+--------------------+-------------+--------------------+-------...
|
|
|<-- free space 2 -->| |<-- free space 1 -->|
|
|
(used if the other is too small) (used in priority)
|
|
|
|
|
|
* Wrapping payloads part :
|
|
|
|
|
|
head_addr end_addr tail_addr
|
|
| | |
|
|
V V V
|
|
+-----+----+----------------+--------+----------------+-------+-------...
|
|
| HTX | | PAYLOADS part2 | | PAYLOADS part1 | | HTX_BLKs
|
|
+-----+----+----------------+--------+----------------+-------+-------...
|
|
|<-->| |<------>| |<----->|
|
|
unusable free space unusable
|
|
free space free space
|
|
|
|
|
|
Finally, when the usable free space is not enough to store a new block, unusable
|
|
parts may be get back with a full defragmentation. The payloads part is then
|
|
realigned at the beginning of the blocks array and the free space becomes
|
|
continuous again.
|
|
|
|
|
|
3. The HTX blocks
|
|
|
|
An HTX block can be as well a start-line as a header, a body part or a
|
|
trailer. For all these types of block, a payload is attached to the block. It
|
|
can also be a marker, the end-of-headers, end-of-trailers or end-of-message. For
|
|
these blocks, there is no payload but it counts for a byte. It is important to
|
|
not skip it when data are forwarded.
|
|
|
|
As already said, a block is composed of metadata and a payload. Metadata are
|
|
stored in the blocks part and are composed of 2 fields :
|
|
|
|
- info : It a 32 bits field containing the block's type on 4 bits followed
|
|
by the payload length. See below for details.
|
|
|
|
- addr : The payload's address, if any, relatively to the beginning the
|
|
array used to store part of the HTTP message itself.
|
|
|
|
|
|
* Block's info representation :
|
|
|
|
0b 0000 0000 0000 0000 0000 0000 0000 0000
|
|
---- ------------------------ ---------
|
|
type value (1 MB max) name length (header/trailer - 256B max)
|
|
----------------------------------
|
|
data length (256 MB max)
|
|
(body, method, path, version, status, reason)
|
|
|
|
|
|
Supported types are :
|
|
|
|
- 0000 (0) : The request start-line
|
|
- 0001 (1) : The response start-line
|
|
- 0010 (2) : A header block
|
|
- 0011 (3) : The end-of-headers marker
|
|
- 0100 (4) : A data block
|
|
- 0101 (5) : A trailer block
|
|
- 0110 (6) : The end-of-trailers marker
|
|
- 0111 (7) : The end-of-message marker
|
|
- 1111 (15) : An unused block
|
|
|
|
Other types are unused for now and reserved for futur extensions.
|
|
|
|
An HTX message is typically composed of following blocks, in this order :
|
|
|
|
- a start-line
|
|
- zero or more header blocks
|
|
- an end-of-headers marker
|
|
- zero or more data blocks
|
|
- zero or more trailer blocks (optional)
|
|
- an end-of-trailers marker (optional but always set if there is at least
|
|
one trailer block)
|
|
- an end-of-message marker.
|
|
|
|
Only one HTTP request at a time can be stored in an HTX message. For HTTP
|
|
response, it is more complicated. Only one "final" response can be stored in an
|
|
HTX message. It is a response with status-code 101 or greater or equal to
|
|
200. But it may be preceded by several 1xx informational responses. Such
|
|
responses are part of the same HTX message, so there is no end-of-message marker
|
|
for them.
|
|
|
|
|
|
3.1. The start-line
|
|
|
|
Every HTX message starts with a start-line. Its payload is a "struct htx_sl". In
|
|
addition to the parts of the HTTP start-line, this structure contains some
|
|
information about the represented HTTP message, mainly in the form of flags
|
|
(HTX_SL_F_*). For instance, if an HTTP message contains the header
|
|
"conten-length", then the flag HTX_SL_F_CLEN is set.
|
|
|
|
Each HTTP message has its own start-line. So an HTX request has one and only one
|
|
start-line because it must contain only one HTTP request at a time. But an HTX
|
|
response may have more than one start-line if the final HTTP response is
|
|
precedeed by some 1xx informational responses.
|
|
|
|
In HTTP/2, there is no start-line. So the H2 multiplexer must create one when it
|
|
converts an H2 message to HTX :
|
|
|
|
- For the request, it uses the pseudo headers ":method", ":path" or
|
|
":authority" depending on the method and the hardcoded version "HTTP/2.0".
|
|
|
|
- For the response, it used the hardcoded version "HTTP/2.0", the
|
|
pseudo-header ":status" and an empty reason.
|
|
|
|
|
|
3.2. The headers and trailers
|
|
|
|
HTX Headers and trailers are quite similar. Different types are used to simplify
|
|
headers processing. But from the HTX point of view, there is no real difference,
|
|
except their position in the HTX message. The header blocks always follow an HTX
|
|
start-line while trailer blocks come after the data. If there is no data, they
|
|
follow the end-of-headers marker.
|
|
|
|
Headers and trailers are the only blocks containing a Key/Value payload. The
|
|
corresponding end-of marker must always be placed after each group to mark, as
|
|
it name suggests, the end.
|
|
|
|
In HTTP/1, trailers are only present on chunked messages. But chunked messages
|
|
do not always have trailers. In this case, the end-of-trailers block may or may
|
|
not be present. Multiplexers must be able to handle both situations. In HTTP/2,
|
|
trailers are only present if a HEADERS frame is sent after DATA frames.
|
|
|
|
|
|
3.3. The data
|
|
|
|
The payload body of an HTTP message is stored as DATA blocks in the HTX
|
|
message. For HTTP/1 messages, it is the message body without the chunks
|
|
formatting, if any. For HTTP/2, it is the payload of DATA frames.
|
|
|
|
The DATA blocks are the only HTX blocks that may be partially processed (copied
|
|
or removed). All other types of block must be entierly processed. This means
|
|
DATA blocks can be resized.
|
|
|
|
|
|
3.4. The end-of markers
|
|
|
|
These blocks are used to delimit parts of an HTX message. It exists three
|
|
markers:
|
|
|
|
- end-of-headers (EOH)
|
|
- end-of-trailers (EOT)
|
|
- end-of-message (EOM)
|
|
|
|
EOH and EOM are always present in an HTX message. EOT is optional.
|
|
|
|
|
|
4. The HTX API
|
|
|
|
|
|
4.1. Get/set HTX message from/to the underlying buffer
|
|
|
|
The first thing to do to process an HTX message is to get it from the underlying
|
|
buffer. There are 2 functions to do so, the second one relying on the first:
|
|
|
|
- htxbuf() returns an HTX message from a buffer. It does not modify the
|
|
buffer. It only initialize the HTX message if the buffer is empty.
|
|
|
|
- htx_from_buf() uses htxbuf(). But it also updates the underlying buffer so
|
|
that it appears as full.
|
|
|
|
Both functions return a "zero-sized" HTX message if the buffer is null. This
|
|
way, you are sure to always have a valid HTX message. The first function is the
|
|
default function to use. The second one is only useful when some content will be
|
|
added. For instance, it used by the HTX analyzers when HAproxy generates a
|
|
response. This way, the buffer is in a right state and you don't need to take
|
|
care of it anymore outside the possible error paths.
|
|
|
|
Once the processing done, if the HTX message has been modified, the underlying
|
|
buffer must be also updated, except you uses htx_from_buf() and you only add
|
|
data. For all other cases, the function htx_to_buf() must be called.
|
|
|
|
Finally, the function htx_reset() may be called at any time to reset an HTX
|
|
message. And the function buf_room_for_htx_data() may be called to know if a raw
|
|
buffer is full from the HTX perspective. It is used during conversion from/to
|
|
the HTX.
|
|
|
|
|
|
4.2. Helpers to deal with free space in an HTX message
|
|
|
|
Once you have an HTX message, following functions may help you to process it :
|
|
|
|
- htx_used_space() and htx_meta_space() return, respectively, the total
|
|
space used in an HTX message and the space used by block's metadata only.
|
|
|
|
- htx_free_space() and htx_free_data_space() return, respectively, the total
|
|
free space in an HTX message and the free space available for the payload
|
|
if a new HTX block is stored (so it is the total free space minus the size
|
|
of an HTX block).
|
|
|
|
- htx_is_empty() and htx_is_not_empty() are boolean functions to know if an
|
|
HTX message is empty or not.
|
|
|
|
- htx_get_max_blksz() returns the maximum size available for the payload,
|
|
not exceeding a maximum, metadata included.
|
|
|
|
- htx_almost_full() should be used to know if an HTX message uses at least
|
|
3/4 of its capacity.
|
|
|
|
|
|
4.3. HTX Blocks manipulations
|
|
|
|
Once you know how much space is available in an HTX message, the next step is to
|
|
add HTX blocks. First of all the function htx_nbblks() returns the number of
|
|
blocks allocated in an HTX message. Then, there is an add function per block's
|
|
type:
|
|
|
|
- htx_add_stline() adds a start-line. The type (request or response) and the
|
|
flags of the start-line must be provided, as well as its three parts
|
|
(method,uri,version or version,status-code,reason).
|
|
|
|
- htx_add_header() and htx_add_trailers() are similar. The name and the
|
|
value must be provided. The inserted HTX block is returned on success or
|
|
NULL if an error occurred.
|
|
|
|
- htx_add_endof() must be used to add any end-of marker. The block's type
|
|
(EOH, EOT or EOM) must be specified. The inserted HTX block is returned on
|
|
success or NULL if an error occurred.
|
|
|
|
- htx_add_all_headers() and htx_add_all_trailers() add, respectively, a list
|
|
of headers and a list of trailers, followed by the appropriate end-of
|
|
marker. On success, this marker is returned. Otherwise, NULL is
|
|
returned. Note there is no rollback on the HTX message when an error
|
|
occurred. Some headers or trailers may have been added. So it is the
|
|
caller responsibility to take care of that.
|
|
|
|
- htx_add_data() must be used to add a DATA block. Unlike previous
|
|
functions, this one returns the number of bytes copied or 0 if nothing was
|
|
copied. If possible, the data are appended to the last DATA block, if
|
|
any. Only a part of the payload may be copied because this function will
|
|
try to limit the message defragmentation and the wrapping of blocks as far
|
|
as possible. If you really need to add all data or nothing, the function
|
|
htx_add_data_atonce() must be used instead. Because it tries to insert all
|
|
the payload, this function returns the inserted block on success.
|
|
Otherwise it returns NULL.
|
|
|
|
When an HTX block is added, it is always the last one (the tail). But, if you
|
|
need to add a block at a specific place, it is not really handy. 2 functions may
|
|
help you (others could be added) :
|
|
|
|
- htx_add_last_data() adds a DATA block just after all other DATA blocks and
|
|
before any trailers and EOT or EOM markers. It relies on
|
|
htx_add_data_atonce(), so a defragmentation may be performed.
|
|
|
|
- htx_move_blk_before() moves a specific block just after another one. Both
|
|
blocks must already be in the HTX message and the block to move must
|
|
always be placed after the "pivot".
|
|
|
|
Once added, there are three functions to update the block's payload :
|
|
|
|
- htx_replace_stline() updates a start-line. The HTX block must be passed as
|
|
argument. Only string parts of the start-line are updated by this
|
|
function. On success, it returns the new start-line. So it is pretty easy
|
|
to update its flags. NULL is returned if an error occurred.
|
|
|
|
- htx_replace_header() fully replaces a header (its name and its value) by a
|
|
new one. The HTX block must be passed a argument, as well as its new name
|
|
and its new value. The new header can be smaller or larger than the old
|
|
one. This function returns the new HTX block on success, or NULL is an
|
|
error occurred.
|
|
|
|
- htx_replace_blk_value() replaces a part of a block's payload or its
|
|
totality. It works for HEADERS, TRAILERS or DATA blocks. The HTX block
|
|
must be provided with the part to remove and the new one. The new part can
|
|
be smaller or larger than the old one. This function returns the new HTX
|
|
block on success, or NULL is an error occurred.
|
|
|
|
Finally, You may remove a block using the function htx_remove_blk(). This
|
|
function returns the block following the one removed or NULL if it is the tail
|
|
block.
|
|
|
|
|
|
4.4. The HTX start-line
|
|
|
|
Unlike other HTX blocks, the start-line is a bit special because its payload is
|
|
a structure followed by its three parts :
|
|
|
|
+--------+-------+-------+-------+
|
|
| HTX_SL | PART1 | PART2 | PART3 |
|
|
+--------+-------+-------+-------+
|
|
|
|
Some macros and functions may help to manipulate these parts :
|
|
|
|
- HTX_SL_P{N}_LEN() and HTX_SL_P{N}_PTR() are macros to get the length of a
|
|
part and a pointer on it. {N} should be 1, 2 or 3.
|
|
|
|
- HTX_SL_REQ_MLEN(), HTX_SL_REQ_ULEN(), HTX_SL_REQ_VLEN(),
|
|
HTX_SL_REQ_MPTR(), HTX_SL_REQ_UPTR() and HTX_SL_REQ_VPTR() are macros to
|
|
get info about a request start-line. These macros only wrap HTX_SL_P*
|
|
ones.
|
|
|
|
- HTX_SL_RES_VLEN(), HTX_SL_RES_CLEN(), HTX_SL_RES_RLEN(),
|
|
HTX_SL_RES_VPTR(), HTX_SL_RES_CPTR() and HTX_SL_RES_RPTR() are macros to
|
|
get info about a response start-line. These macros only wrap HTX_SL_P*
|
|
ones.
|
|
|
|
- htx_sl_p1(), htx_sl_p2() and htx_sl_p2() are functions to get the ist
|
|
corresponding to the right part of a start-line.
|
|
|
|
- htx_sl_req_meth(), htx_sl_req_uri() and htx_sl_req_vsn() get the ist
|
|
corresponding to the right part of a request start-line.
|
|
|
|
- htx_sl_res_vsn(), htx_sl_res_code() and htx_sl_res_reason() get the ist
|
|
corresponding to the right part of a response start-line.
|
|
|
|
|
|
4.5. Iterate on the HTX message
|
|
|
|
To iterate on an HTX message, the first thing to do is to get the HTX block to
|
|
start the loop. There are three special blocks in an HTX message that may be
|
|
good candidates to start a loop :
|
|
|
|
* the head block. It is the oldest inserted block. Multiplexers always start
|
|
to consume an HTX message from this block. The function htx_get_head()
|
|
returns its position and htx_get_head_blk() returns the blocks itself. In
|
|
addition, the function htx_get_head_type() returns its block's type.
|
|
|
|
* the tail block. It is the newest inserted block. The function htx_get_tail()
|
|
returns its position and htx_get_tail_blk() returns the blocks itself. In
|
|
addition, the function htx_get_tail_type() returns its block's type.
|
|
|
|
* the first block. It is the block where to (re)start the analyse. It is used
|
|
as start point by HTX analyzers. The function htx_get_first() returns its
|
|
position and htx_get_first_blk() returns the blocks itself. In addition, the
|
|
function htx_get_first_type() returns its block's type.
|
|
|
|
For all these functions, if the HTX message is empty, -1 is returned for the
|
|
block's position, NULL instead of a block and HTX_BLK_UNUSED for its type.
|
|
|
|
Then to iterate on blocks, you may move foreword or backward :
|
|
|
|
* htx_get_prev() and htx_get_next() return, respectively, the position of the
|
|
previous block or the next block, given a specific position. Or -1 if an edge
|
|
is reached.
|
|
|
|
* htx_get_prev_blk() and htx_get_next_blk() return, respectively, the previous
|
|
block or the next one, given a specific block. Or NULL if an edge is
|
|
reached.
|
|
|
|
|
|
4.6. Advanced functions
|
|
|
|
Some more advanced functions may be used to do complex processing on the HTX
|
|
message. These functions are used by HTX analyzers or by multiplexers.
|
|
|
|
* htx_truncate() removes all blocks after the one containing a specific offset
|
|
relatively to the head block of the HTX message. If the offset is inside a
|
|
DATA block, it is truncated. For all other blocks, the removal starts to the
|
|
next block.
|
|
|
|
* htx_drain() tries to remove a specific amount of bytes of payload. If the
|
|
last block is a DATA block, it may be truncated if necessary. All other
|
|
block are removed at once or kept. This function returns a mixed value, with
|
|
the first block not removed, or NULL if everything was removed, and the
|
|
amount of data drained.
|
|
|
|
* htx_xfer_blks() transfers HTX blocks from an HTX message to another,
|
|
stopping on the first block of a specified type or when a specific amount of
|
|
bytes, including meta-data, was moved. If the last block is a DATA block, it
|
|
may be partially moved. All other block are transferred at once or
|
|
kept. This function returns a mixed value, with the last block moved, or
|
|
NULL if nothing was moved, and the amount of data transferred. When HEADERS
|
|
or TRAILERS blocks must be transferred, this function transfers all of
|
|
them. Otherwise, if it is not possible, it triggers an error. It is the
|
|
caller responsibility to transfer all headers or trailers at once.
|
|
|