Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP Proxy: Fix corrupted hostname from partial connection read. #10454

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
31 changes: 26 additions & 5 deletions pkg/tcpproxy/tcp.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,31 @@ func (p *TCPProxy) Get(host string) *TCPServer {
// and open a connection to the passthrough server.
func (p *TCPProxy) Handle(conn net.Conn) {
defer conn.Close()
// See: https://www.ibm.com/docs/en/ztpf/1.1.0.15?topic=sessions-ssl-record-format
data := make([]byte, 16384)

length, err := conn.Read(data)
// [Documentation by @maxl99](https://github.com/kubernetes/ingress-nginx/pull/11843/files#diff-aef3e187fd37c68706ad582d7b89a2d9ad11691bd929a2158b86f93362244105R67-R79)
chotiwat marked this conversation as resolved.
Show resolved Hide resolved
// It appears that the ClientHello must fit into *one* TLSPlaintext message:
// When a client first connects to a server, it is REQUIRED to send the ClientHello as its first TLS message.
// Source: https://datatracker.ietf.org/doc/html/rfc8446#section-4.1.2
//
// length: The length (in bytes) of the following TLSPlaintext.fragment. The length MUST NOT exceed 2^14 bytes.
// An endpoint that receives a record that exceeds this length MUST terminate the connection with a "record_overflow" alert.
// Source: https://datatracker.ietf.org/doc/html/rfc8446#section-5.1
// bytes 0 : content type
// bytes 1-2: legacy version
// bytes 3-4: length
// bytes 5+ : message
// https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_record
// Thus, we need to allocate 5 + 16384 bytes
Gacko marked this conversation as resolved.
Show resolved Hide resolved
data := make([]byte, parser.TLSHeaderLength+16384)

// read the tls header first
_, err := io.ReadFull(conn, data[:parser.TLSHeaderLength])
if err != nil {
klog.V(4).ErrorS(err, "Error reading TLS header from the connection")
return
}
// get the total data length then read the rest
length := min(int(data[3])<<8+int(data[4])+parser.TLSHeaderLength, len(data))
_, err = io.ReadFull(conn, data[parser.TLSHeaderLength:length])
if err != nil {
klog.V(4).ErrorS(err, "Error reading data from the connection")
return
Expand Down Expand Up @@ -115,7 +136,7 @@ func (p *TCPProxy) Handle(conn net.Conn) {
} else {
_, err = clientConn.Write(data[:length])
if err != nil {
klog.Errorf("Error writing the first 4k of proxy data: %v", err)
klog.Errorf("Error writing the first %d bytes of proxy data: %v", length, err)
clientConn.Close()
}
}
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/framework/deployment.go
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,7 @@ func newDeployment(name, namespace, image string, port int32, replicas int32, co
{
Name: name,
Image: image,
Env: []corev1.EnvVar{},
Env: env,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a bug in the e2e framework as well. This was preventing httpbun from getting the environment variables so it always ran as an HTTP server even though HTTPBUN_SSL_CERT and HTTPBUN_SSL_KEY are specified.

Ports: []corev1.ContainerPort{
{
Name: "http",
Expand Down
35 changes: 27 additions & 8 deletions test/e2e/framework/httpexpect/request.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,22 +80,41 @@ func (h *HTTPRequest) ForceResolve(ip string, port uint16) *HTTPRequest {
h.chain.fail(fmt.Sprintf("invalid ip address: %s", ip))
return h
}
dialer := &net.Dialer{
Timeout: h.client.Timeout,
KeepAlive: h.client.Timeout,
DualStack: true,
}
resolveAddr := fmt.Sprintf("%s:%d", ip, int(port))

return h.WithDialContextMiddleware(func(next DialContextFunc) DialContextFunc {
return func(ctx context.Context, network, _ string) (net.Conn, error) {
return next(ctx, network, resolveAddr)
}
})
}

// DialContextFunc is the function signature for `DialContext`
type DialContextFunc func(ctx context.Context, network, addr string) (net.Conn, error)

// WithDialContextMiddleware sets the `DialContext` function of the client
// transport with a new function returns from `fn`. An existing `DialContext`
// is passed into `fn` so the new function can act as a middleware by calling
// the old one.
func (h *HTTPRequest) WithDialContextMiddleware(fn func(next DialContextFunc) DialContextFunc) *HTTPRequest {
oldTransport, ok := h.client.Transport.(*http.Transport)
if !ok {
h.chain.fail("invalid old transport address")
return h
}
newTransport := oldTransport.Clone()
newTransport.DialContext = func(ctx context.Context, network, _ string) (net.Conn, error) {
return dialer.DialContext(ctx, network, resolveAddr)
var nextDialContext DialContextFunc
if oldTransport.DialContext != nil {
nextDialContext = oldTransport.DialContext
} else {
dialer := &net.Dialer{
Timeout: h.client.Timeout,
KeepAlive: h.client.Timeout,
DualStack: true,
}
nextDialContext = dialer.DialContext
}
newTransport := oldTransport.Clone()
newTransport.DialContext = fn(nextDialContext)
h.client.Transport = newTransport
return h
}
Expand Down
Loading