Rust in Everything
基于 hyper 1.12.0

invalid HTTP version parsed 与 found HTTP2 preface

主线位置:两条主线的「解析消息头」一步。服务端主线上,role::Server::parse 用 httparse 解析请求行;客户端主线上,role::Client::parse 解析状态行。两边都只认 HTTP/1.0 和 HTTP/1.1,读到别的版本号(或者根本不是 HTTP 的字节)就是这个错误。

flowchart TB
  bytes["读缓冲里的字节"] --> who{"谁在解析"}
  who -->|"服务端"| req["请求行"]
  who -->|"客户端"| res["状态行"]
  req -->|"PRI * HTTP/2.0 且满 24 字节"| vh2["found HTTP2 preface,不回响应"]
  req -->|"HTTP/1.2 之类"| v400["invalid HTTP version,回 400"]
  req -->|"首字节不是 token"| m400["invalid HTTP method,回 400"]
  res -->|"不以 HTTP/1. 开头"| vcli["invalid HTTP version"]

两句话来自同一个枚举的两个变体:Parse::Version 是 invalid HTTP version parsed,Parse::VersionH2 是 invalid HTTP version parsed (found HTTP2 preface)。

这两句话从哪来

httparse 的 Error::Version 经 From<httparse::Error> for Parse 变成 Parse::Version。httparse 1.10.1 只接受 HTTP/1.0 和 HTTP/1.1,请求行末尾是别的版本号,或者响应开头不是 HTTP/1.,都是这个错误。

服务端多一步。解析出错后,Conn::on_parse_error 在回错误响应之前先看一眼读缓冲:

    fn on_parse_error(&mut self, err: crate::Error) -> crate::Result<()> {
        if let Writing::Init = self.state.writing {
            if self.has_h2_prefix() {
                return Err(crate::Error::new_version_h2());
            }
            if let Some(msg) = T::on_error(&err) {

has_h2_prefix 要求缓冲里至少 24 个字节且前 24 个正好是 PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n。是的话,不管原来是什么解析错误,都换成 VersionH2 直接返回,不写 400:对一个 HTTP/2 客户端写 HTTP/1 的错误响应没有意义,而且上层可能想把这条连接转给 HTTP/2。不是的话,问 on_error 要状态码,Parse::Version 回 400(解析与编码的「解析失败时回什么」)。

复现

一个只开 HTTP/1 的 hyper 服务端,分别接:HTTP/2 先验知识客户端(不经 TLS、不经 Upgrade,一上来就发前言);一个 TLS 客户端(rustls,模拟 https:// 打到明文端口);一个手写的 HTTP/1.2 请求。反过来,一个 HTTP/1 客户端分别连:只开 HTTP/2 的 hyper 服务端;一个 rustls 服务端(模拟 http:// 打到 TLS 端口)。

hyper = { version = "=1.12.0", features = ["client", "server", "http1", "http2"] }
hyper-util = { version = "=0.1.21", features = ["tokio"] }
http-body-util = "=0.1.5"
bytes = "1"
tokio = { version = "=1.54.1", features = ["macros", "rt", "net", "time", "io-util", "sync"] }
tokio-rustls = { version = "=0.26.6", default-features = false, features = ["ring"] }
rcgen = "=0.13.2"
use std::convert::Infallible;
use std::sync::Arc;
use std::time::Duration;

use bytes::Bytes;
use http_body_util::{Empty, Full};
use hyper::client::conn::{http1 as client_h1, http2 as client_h2};
use hyper::server::conn::{http1 as server_h1, http2 as server_h2};
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::oneshot;
use tokio_rustls::rustls;

async fn hello(_: Request<hyper::body::Incoming>) -> Result<Response<Full<Bytes>>, Infallible> {
  Ok(Response::new(Full::new(Bytes::from_static(b"ok"))))
}

fn chain(e: &(dyn std::error::Error + 'static)) -> String {
  let mut s = e.to_string();
  let mut src = e.source();
  while let Some(inner) = src {
    s.push_str(&format!(": {inner}"));
    src = inner.source();
  }
  s
}

// A plaintext HTTP/1 server for one connection; reports how serve_connection ended.
async fn h1_server() -> (std::net::SocketAddr, oneshot::Receiver<String>) {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  let (tx, rx) = oneshot::channel();
  tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    let r = server_h1::Builder::new().serve_connection(TokioIo::new(stream), service_fn(hello)).await;
    let _ = tx.send(match r {
      Ok(()) => "Ok(())".to_string(),
      Err(e) => format!("Err({}) is_parse_version_h2={}", chain(&e), e.is_parse_version_h2()),
    });
  });
  (addr, rx)
}

async fn h1_client(io: TcpStream) -> String {
  let (mut tx, conn) = match client_h1::handshake::<_, Empty<Bytes>>(TokioIo::new(io)).await {
    Ok(p) => p,
    Err(e) => return format!("Err({})", chain(&e)),
  };
  tokio::spawn(conn);
  match tx.send_request(Request::get("/").header("host", "x").body(Empty::new()).unwrap()).await {
    Ok(r) => r.status().to_string(),
    Err(e) => format!("Err({})", chain(&e)),
  }
}

fn tls_configs() -> (Arc<rustls::ServerConfig>, Arc<rustls::ClientConfig>) {
  let provider = Arc::new(rustls::crypto::ring::default_provider());
  let cert = rcgen::generate_simple_self_signed(vec!["localhost".to_string()]).unwrap();
  let der = cert.cert.der().clone();
  let key = rustls::pki_types::PrivateKeyDer::try_from(cert.key_pair.serialize_der()).unwrap();
  let server = rustls::ServerConfig::builder_with_provider(provider.clone())
    .with_safe_default_protocol_versions()
    .unwrap()
    .with_no_client_auth()
    .with_single_cert(vec![der.clone()], key)
    .unwrap();
  let mut roots = rustls::RootCertStore::empty();
  roots.add(der).unwrap();
  let client = rustls::ClientConfig::builder_with_provider(provider)
    .with_safe_default_protocol_versions()
    .unwrap()
    .with_root_certificates(roots)
    .with_no_client_auth();
  (Arc::new(server), Arc::new(client))
}

#[tokio::main(flavor = "current_thread")]
async fn main() {
  let (tls_server, tls_client) = tls_configs();

  println!("== server side: an HTTP/1 server receives ...");

  // 1. An HTTP/2 prior-knowledge client.
  let (addr, rx) = h1_server().await;
  let io = TokioIo::new(TcpStream::connect(addr).await.unwrap());
  let client = async {
    let (mut tx, conn) = client_h2::handshake::<_, _, Empty<Bytes>>(TokioExecutor::new(), io).await?;
    tokio::spawn(conn);
    tx.send_request(Request::get("http://x/").body(Empty::new()).unwrap()).await
  };
  let c = match client.await {
    Ok(r) => r.status().to_string(),
    Err(e) => format!("Err({})", chain(&e)),
  };
  println!("h2 prior knowledge\n  server: {}\n  client: {c}", rx.await.unwrap());

  // 2. A TLS ClientHello.
  let (addr, rx) = h1_server().await;
  let tcp = TcpStream::connect(addr).await.unwrap();
  let connector = tokio_rustls::TlsConnector::from(tls_client.clone());
  let name = rustls::pki_types::ServerName::try_from("localhost").unwrap();
  let c = match connector.connect(name, tcp).await {
    Ok(_) => "handshake ok".to_string(),
    Err(e) => format!("Err({})", chain(&e)),
  };
  println!("TLS ClientHello\n  server: {}\n  client: {c}", rx.await.unwrap());

  // 3. A request line with HTTP/1.2.
  let (addr, rx) = h1_server().await;
  let mut tcp = TcpStream::connect(addr).await.unwrap();
  tcp.write_all(b"GET / HTTP/1.2\r\nhost: x\r\n\r\n").await.unwrap();
  let mut out = String::new();
  let _ = tcp.read_to_string(&mut out).await;
  println!("GET / HTTP/1.2\n  server: {}\n  client: got {:?}", rx.await.unwrap(), out.lines().next().unwrap_or(""));

  println!("== client side: an HTTP/1 client talks to ...");

  // 4. An HTTP/2-only server.
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  let server = tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    server_h2::Builder::new(TokioExecutor::new())
      .serve_connection(TokioIo::new(stream), service_fn(hello))
      .await
      .map_err(|e| chain(&e))
  });
  let c = h1_client(TcpStream::connect(addr).await.unwrap()).await;
  println!("h2-only server\n  client: {c}\n  server: {:?}", server.await.unwrap());

  // 5. A TLS server (http:// URL on an https port).
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  let acceptor = tokio_rustls::TlsAcceptor::from(tls_server.clone());
  let server = tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    acceptor.accept(stream).await.map(|_| ()).map_err(|e| chain(&e))
  });
  let c = tokio::time::timeout(Duration::from_secs(2), h1_client(TcpStream::connect(addr).await.unwrap())).await;
  println!("TLS server\n  client: {}\n  server: {:?}", c.unwrap_or_else(|_| "timed out".into()), server.await.unwrap());
}

M2 Max、rustc 1.99.0、--release(rustls 0.23.46、tokio-rustls 0.26.6、rcgen 0.13.2),连跑 3 次输出相同:

== server side: an HTTP/1 server receives ...
h2 prior knowledge
  server: Err(invalid HTTP version parsed (found HTTP2 preface)) is_parse_version_h2=true
  client: Err(connection error: stream closed because of a broken pipe)
TLS ClientHello
  server: Err(invalid HTTP method parsed) is_parse_version_h2=false
  client: Err(received corrupt message of type InvalidContentType)
GET / HTTP/1.2
  server: Err(invalid HTTP version parsed) is_parse_version_h2=false
  client: got "HTTP/1.1 400 Bad Request"
== client side: an HTTP/1 client talks to ...
h2-only server
  client: Err(invalid HTTP version parsed)
  server: Err("http2 error: connection error detected: unspecific protocol error detected")
TLS server
  client: Err(invalid HTTP version parsed)
  server: Err("received corrupt message of type InvalidContentType")

客户端那两项,HTTP/1 客户端到底收到了什么?另一个小程序对同样的两个服务端用裸 TcpStream 发 GET / HTTP/1.1,打印回来的字节数和前 16 个字节。依赖同上:

use std::convert::Infallible;
use std::time::Duration;

use bytes::Bytes;
use http_body_util::Full;
use hyper::server::conn::http2 as server_h2;
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio_rustls::rustls;

async fn show(name: &str, target: std::net::SocketAddr) {
  let mut tcp = TcpStream::connect(target).await.unwrap();
  tcp.write_all(b"GET / HTTP/1.1\r\nhost: x\r\n\r\n").await.unwrap();
  let mut got = Vec::new();
  let mut buf = [0u8; 1024];
  let end = loop {
    match tokio::time::timeout(Duration::from_secs(1), tcp.read(&mut buf)).await {
      Ok(Ok(0)) => break "EOF",
      Ok(Ok(n)) => got.extend_from_slice(&buf[..n]),
      Ok(Err(_)) => break "reset",
      Err(_) => break "still open",
    }
  };
  let hex: Vec<String> = got.iter().take(16).map(|b| format!("{b:02x}")).collect();
  println!("{name:<15} {} bytes: {} then {end}", got.len(), hex.join(" "));
}

// What an HTTP/1 request gets back from an HTTP/2-only server and from a TLS server.
#[tokio::main(flavor = "current_thread")]
async fn main() {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    let svc = service_fn(|_: Request<hyper::body::Incoming>| async {
      Ok::<_, Infallible>(Response::new(Full::new(Bytes::from_static(b"ok"))))
    });
    let _ = server_h2::Builder::new(TokioExecutor::new()).serve_connection(TokioIo::new(stream), svc).await;
  });
  show("h2-only server", addr).await;

  let provider = std::sync::Arc::new(rustls::crypto::ring::default_provider());
  let cert = rcgen::generate_simple_self_signed(vec!["localhost".to_string()]).unwrap();
  let key = rustls::pki_types::PrivateKeyDer::try_from(cert.key_pair.serialize_der()).unwrap();
  let config = rustls::ServerConfig::builder_with_provider(provider)
    .with_safe_default_protocol_versions()
    .unwrap()
    .with_no_client_auth()
    .with_single_cert(vec![cert.cert.der().clone()], key)
    .unwrap();
  let acceptor = tokio_rustls::TlsAcceptor::from(std::sync::Arc::new(config));
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    let _ = acceptor.accept(stream).await;
  });
  show("rustls server", addr).await;
}

连跑 3 次输出相同:

h2-only server  33 bytes: 00 00 18 04 00 00 00 00 00 00 03 00 00 00 c8 00 then reset
rustls server   7 bytes: 15 03 03 00 02 02 32 then EOF

逐项看

场景报错的一方错误回了什么
HTTP/2 先验知识客户端 → HTTP/1 服务端服务端invalid HTTP version parsed (found HTTP2 preface)什么都不回,关连接
HTTP/1.2 请求 → HTTP/1 服务端服务端invalid HTTP version parsed400
HTTP/1 客户端 → HTTP/2 服务端客户端invalid HTTP version parsed服务端先发了 SETTINGS 帧
HTTP/1 客户端 → TLS 服务端客户端invalid HTTP version parsedrustls 回了一条 TLS alert
TLS 客户端 → HTTP/1 服务端服务端invalid HTTP method parsed400,TLS 客户端读不懂
  • HTTP/2 前言。 PRI * HTTP/2.0 的方法和路径都合法,httparse 卡在版本号上;解析失败时缓冲里已经有完整的 24 字节前言(输出是 VersionH2 说明了这一点),has_h2_prefix 成立,于是换成 VersionH2。只发了不到 24 个字节的前言(比如前 16 个),报的是普通的 invalid HTTP version parsed,并且回 400(auto 的动脑筋里 http1::Builder 那一组就是这样)。
  • 客户端那两项,第一个字节就不对。 HTTP/2 服务端不等对方说话就先发 SETTINGS 帧,帧头是 9 个字节,第一个是长度的高位 00;rustls 收到不是 TLS 记录的字节,回一条 alert,记录类型是 0x15。两者都不是 H,Client::parse 里 httparse 立刻报 Version。客户端开了 http09_responses 的话,这个错误会被当成一个 HTTP/0.9 响应接受下来(200,整段字节当 body),那就连错误都看不到了。
  • TLS 打到明文端口不是版本错误。 ClientHello 的第一个字节是 0x16(握手记录),不是合法的 token 字符,httparse 在方法名上就失败了,Server::parse 把方法名位置上的 Token 错误归为 Parse::Method。服务端照常回 400,TLS 客户端把 HTTP/1.1 400 当成 TLS 记录来读,报 InvalidContentType。

TLS 服务端怎么回应明文请求,取决于实现。上面的 rustls 回 alert,所以客户端看到 invalid HTTP version parsed。换成 OpenSSL 4.0.3 的 openssl s_server,用裸 socket 发同样的请求,3 次都是一个字节没收到就立刻读到 EOF,这时 hyper 客户端报的会是 connection closed before message completed。还有的服务器会回一个正常的 HTTP 400 响应,告诉你发错了端口。所以「http:// 打到 TLS 端口」不一定是本页的报错,本页的报错也不一定是 TLS。

📝NOTE
冷知识:VersionH2 原本是一个转换信号

has_h2_prefix 和 VersionH2 是 hyper 0.12 为自动识别协议加的。PR #1503(「Try h2 on h1 failure」)的提交说明:「If an HTTP/1 connection has a parse error, but it starts with the HTTP2 preface, converts the connection automatically into an HTTP2 server connection.」到 0.14 为止,server::conn::Http 都有一个 Fallback 模式,HTTP/1 连接报出 Kind::Parse(Parse::VersionH2) 就调用 upgrade_h2(),把同一个 IO 交给 HTTP/2 继续处理,用户看不到这个错误。第一个包含它的发布是 v0.12.0。

1.0 的 PR #3013(「feat(server): remove server::conn::{Http, Connection} types」,首见于 v1.0.0-rc.1)把 Http 拆成 http1 和 http2 两个模块,Fallback 和那段转换逻辑随之删除,提交说明让需要两种协议的人「look for the auto-version type in hyper-util」。has_h2_prefix 和 VersionH2 留了下来,从那以后,这个错误就直接交到用户手里。v1.9.0 加的 Error::is_parse_version_h2(PR #3996)让上层又能认出它,为的正是把「看到前言再转 HTTP/2」这条路重新接上,见 header_read_timeout 不生效 的冷知识。

修法

  • 服务端要同时接 HTTP/1 和 HTTP/2 先验知识客户端: 用 hyper-util 的 auto::Builder(auto:一个端口两个协议),它在交给 HTTP/1 之前先比对前言。注意 http1_only 加 serve_connection 会重新得到这个错误(同一页的实验)。
  • 服务端只想记录、不想报警: 用 is_parse_version_h2()(hyper v1.9.0 起)把它和真正的坏请求分开。
  • 客户端连 HTTP/2 服务(比如 gRPC 的明文端口): legacy Client 设 http2_only(true),或用 client::conn::http2。
  • 客户端连 TLS 服务: URL 用 https://,并给 legacy Client 一个支持 TLS 的连接器(HttpConnector 只做 TCP,见 HttpConnector)。
  • 服务端看到 invalid HTTP method parsed、而客户端说 TLS 出错: 多半是 https:// 打到了明文端口,查客户端的 scheme 或者前面代理的配置。

小结

  • invalid HTTP version parsed 是 httparse 在版本号位置失败:请求行不是 HTTP/1.0 / HTTP/1.1,或者响应不以 HTTP/1. 开头。服务端回 400。
  • 读缓冲以完整的 24 字节 HTTP/2 前言开头时,服务端改报 (found HTTP2 preface),不回响应;is_parse_version_h2() 判断它。这是 0.12 自动转 HTTP/2 留下的信号,1.0 起不再自动转。
  • 客户端看到它,通常是连到了 HTTP/2 服务端(第一个字节 00)或 TLS 服务端(15)。
  • TLS 握手打到明文 HTTP/1 端口报的是 invalid HTTP method parsed;明文打到 TLS 端口报什么取决于 TLS 实现,OpenSSL 直接关连接时是 IncompleteMessage。

💬 关联讨论

发起讨论
加载中...