Implement dynamic socket buffer sizes #1666
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cowboy will set the socket's buffer size dynamically to better fit the current workload. When the incoming data is small, a low buffer size reduces the memory footprint and improves responsiveness and therefore performance. When the incoming data is large, such as large HTTP request bodies, a larger buffer size helps us avoid doing too many binary appends and related allocations. Setting a large buffer size for all use cases is sub-optimal because allocating more than needed necessarily results in a performance hit (not just increased memory usage). By default Cowboy starts with a buffer size of 8192 bytes. It then doubles or halves the buffer size depending on the size of the data it receives from the socket. It stops decreasing at 8192 and increasing at 131072 by default. To keep track of the size of the incoming data Cowboy maintains a moving average. It allows Cowboy to avoid changing the buffer too often but still react quickly when necessary. Cowboy will increase the buffer size when the moving average is above 90% of the current buffer size, and decrease when the moving average is below 40% of the current buffer size. The current buffer size and moving average are propagated when switching protocols. The dynamic buffer is implemented in HTTP/1, HTTP/2 and HTTP/1 Websocket. HTTP/2 Websocket has it disabled because it doesn't interact directly with the socket; in that case it is HTTP/2 that has a dynamic buffer. The dynamic buffer provides a very large performance improvement in many scenarios, at minimal cost for others. Because it largely depend on the underlying protocol the improvements are no all equal. TLS and compression also impact the results. The improvement when reading a large request body, with the requests repeated in a fast loop are: * HTTP: 6x to 20x faster * HTTPS: 2x to 6x faster * H2: 4x to 5x faster * H2C: 20x to 40x faster I am not sure why H2C's performance was so bad, especially compared to H2, when using default buffer sizes. Dynamic buffers make H2C a lot more viable with default settings. The performance impact on "hello world" type requests is minimal, it goes from -5% to +5% roughly. Websocket improvements vary again depending on the protocol, but also depending on whether compression is enabled: * HTTP echo: roughly 2x faster * HTTP send: roughly 4x faster * H2C echo: roughly 2x faster * H2C send: 3x to 4x faster In the echo test we reply back, and Gun doesn't have the dynamic buffer optimisation, so that probably explains the x2 difference. With compression however there isn't much improvement. The results are roughly within -10% to +10% of each other. Zlib compression seems to be a bottleneck, or at least to modify the performance profile to such an extent that the size of the buffer does not matter. This happens to randomly generated binary data as well so it is probably not caused by the test data.
In rare cases it was possible for active mode to be disabled when there were no streams pipelined. This resulted in the dropping of the connection due to timeouts as no data could be received. We now enable active mode when necessary even if there are no streams pipelined. This was found while benchmarking and I have not been able to extract a test case.
|
Merged!!! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#1611