Rust in Everything
基于 hyper-util 0.1.21

auto:一个端口两个协议

主线位置:服务端主线的入口多了一步。accept 得到 socket、TokioIo::new(stream) 之后,auto::Builder::serve_connection 先偷看连接最前面的字节,决定交给 http1::Builder 还是 http2::Builder,此后就是 HTTP/1 连接 或 HTTP/2 服务端 的主线。

flowchart TB
  acc["accept + TokioIo"] --> sc["auto serve_connection"]
  sc -->|"未指定版本"| rv["ReadVersion:读前 24 字节"]
  sc -->|http1_only| h1["http1 serve_connection"]
  sc -->|http2_only| h2["http2 serve_connection"]
  rv -->|"某字节不符或 EOF"| rw1["Rewind 塞回已读字节"]
  rv -->|"24 字节全符"| rw2["Rewind 塞回已读字节"]
  rw1 --> h1
  rw2 --> h2
  h1 --> loop1["Dispatcher 主线"]
  h2 --> loop2["h2 每流一个任务"]

hyper 本身的 http1::Builder 和 http2::Builder 各只认一种协议。浏览器走 TLS 时,协议在握手里用 ALPN 定好了,用哪个 Builder 应用自己知道。明文端口上没有 ALPN,客户端要用 HTTP/2 只能「先验知识」(prior knowledge):一连上就直接发 HTTP/2 的连接前言。hyper-util 的 server/conn/auto/mod.rs 做的就是在这种端口上分辨两种客户端。它不看 ALPN,也不处理 HTTP/1.1 的 Upgrade: h2c,只比对连接前言。axum 的 axum::serve 每条连接用的就是它。

Builder:两个 Builder 装在一起

Builder 里就是 hyper 的两个 Builder,加一个可选的版本:

pub struct Builder<E> {
    #[cfg(feature = "http1")]
    http1: http1::Builder,
    #[cfg(feature = "http2")]
    http2: http2::Builder<E>,
    #[cfg(any(feature = "http1", feature = "http2"))]
    version: Option<Version>,
    #[cfg(not(feature = "http2"))]
    _executor: E,
}

Builder::new(executor) 的执行器只交给 http2::Builder::new,HTTP/1 用不着它(见 Executor 与 Timer)。两边的选项通过 builder.http1()、builder.http2() 拿到的 Http1Builder、Http2Builder 设置,它们只是 &mut Builder 的包装,每个方法转发给里面的 hyper Builder。

转发是一个一个手写的,所以会落后于 hyper。拿 hyper 1.12.0 http1::Builder 的公开方法和 hyper-util 0.1.21 的 Http1Builder 对一遍,Http1Builder 少了两个:1.12.0 新加的 max_header_size,和 allow_multiple_spaces_in_request_line_delimiters。hyper-util 0.1.21 比 hyper 1.12.0 早发布十几天,用 auto 时这两个选项暂时设不了。

另一个和 http1::Builder 一样的地方:new 不设 timer,所以 header_read_timeout 的默认 30 秒不生效,要自己调 builder.http1().timer(TokioTimer::new())。设了 timer 也只管到 HTTP/1 那一段,下面会看到判断协议这一步不受它约束。

read_version:偷看 24 个字节

没有指定版本时,serve_connection 把 IO 交给 read_version,得到一个 ReadVersion future。它的栈上缓冲正好是 HTTP/2 连接前言的长度:

const H2_PREFACE: &[u8] = b"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n";

ReadVersion::poll:

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = self.project();
        if *this.cancelled {
            return Poll::Ready(Err(io::Error::new(io::ErrorKind::Interrupted, "Cancelled")));
        }

        let mut buf = ReadBuf::uninit(&mut *this.buf);
        // SAFETY: `this.filled` tracks how many bytes have been read (and thus initialized) and
        // we're only advancing by that many.
        unsafe {
            buf.unfilled().advance(*this.filled);
        };

        // We start as H2 and switch to H1 as soon as we don't have the preface.
        while buf.filled().len() < H2_PREFACE.len() {
            let len = buf.filled().len();
            ready!(Pin::new(this.io.as_mut().unwrap()).poll_read(cx, buf.unfilled()))?;
            *this.filled = buf.filled().len();

            // We starts as H2 and switch to H1 when we don't get the preface.
            if buf.filled().len() == len
                || buf.filled()[len..] != H2_PREFACE[len..buf.filled().len()]
            {
                *this.version = Version::H1;
                break;
            }
        }

        let io = this.io.take().unwrap();
        let buf = buf.filled().to_vec();
        Poll::Ready(Ok((
            *this.version,
            Rewind::new_buffered(io, Bytes::from(buf)),
        )))
    }

几个细节:

  • 先假定是 HTTP/2。 version 初始化成 Version::H2。每读进一段,只比较新读到的这一段([len..])和前言的对应位置,一个字节不符就改成 H1 并跳出。HTTP/1 请求以方法名开头,GET、HEAD 第一个字节就不是 P;POST、PUT、PATCH 要读到第二个字节才分出来。只有真正的 HTTP/2 客户端需要凑满 24 个字节。
  • EOF 也算 HTTP/1。 一次读返回 0 个字节(filled().len() == len),同样落到 H1,后面的事交给 HTTP/1 的连接去处理:一个字节都没收到的 EOF,hyper 当作空闲连接正常结束;收到半截再 EOF,就是 connection closed before message completed。
  • 跨 poll 记住进度。 缓冲是 [MaybeUninit<u8>; 24],读到多少记在 filled 里;Pending 之后再被 poll,先用 advance(filled) 恢复 ReadBuf 的位置。ReadBuf 的两条边界见 IO trait。
  • 没有超时。 循环里只有 poll_read。客户端连上不发字节,或者发了前言的前几个字节就不动了,这个 future 就一直 Pending。HTTP/1 的 header_read_timeout 计时从 http1::Connection 建好之后才开始,管不到这里。本页末尾的「动脑筋」量了这件事。

Rewind:把读过的字节塞回去

判断协议时读走的字节(最多 24 个)属于请求本身,不能丢。ReadVersion 把 IO 和这些字节一起包进 hyper-util 自己的 Rewind,它的 poll_read 先吐 pre,吐完再读底层 IO:

        if let Some(mut prefix) = self.pre.take() {
            // If there are no remaining bytes, let the bytes get dropped.
            if !prefix.is_empty() {
                let copy_len = cmp::min(prefix.len(), buf.remaining());
                buf.put_slice(&prefix[..copy_len]);
                prefix.advance(copy_len);
                // Put back what's left
                if !prefix.is_empty() {
                    self.pre = Some(prefix);
                }

                return Poll::Ready(Ok(()));
            }
        }
        Pin::new(&mut self.inner).poll_read(cx, buf)

写方向原样转发。这和 hyper 在升级时用的 Rewind 是同一个思路,两份代码各在各的 crate 里,hyper 那份见 协议升级。HTTP/1 或 HTTP/2 的连接看到的 IO 类型因此是 Rewind<I>,不是你传进去的 I;这一点在升级时会冒出来,见下文。

实验:一个端口,三种客户端

同一个 auto 端口上,先连一个 hyper 的 HTTP/1 客户端,再连一个 HTTP/2 先验知识客户端,最后用裸 TCP 发一个方法名恰好是 PRI 的 HTTP/1.1 请求。服务回显请求的版本和路径:

hyper = { version = "=1.12.0", features = ["client", "server", "http1", "http2"] }
hyper-util = { version = "=0.1.21", features = ["server-auto", "tokio"] }
http-body-util = "=0.1.5"
bytes = "1"
tokio = { version = "=1.54.1", features = ["rt-multi-thread", "macros", "net", "io-util", "time"] }
use std::convert::Infallible;

use bytes::Bytes;
use http_body_util::{BodyExt, Empty, Full};
use hyper::body::Incoming;
use hyper::client::conn::{http1 as client_h1, http2 as client_h2};
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo};
use hyper_util::server::conn::auto;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};

async fn handler(req: Request<Incoming>) -> Result<Response<Full<Bytes>>, Infallible> {
  let text = format!("{:?} {}", req.version(), req.uri().path());
  Ok(Response::new(Full::new(Bytes::from(text))))
}

#[tokio::main]
async fn main() {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  tokio::spawn(async move {
    loop {
      let (stream, _) = listener.accept().await.unwrap();
      tokio::spawn(async move {
        let builder = auto::Builder::new(TokioExecutor::new());
        let r = builder.serve_connection(TokioIo::new(stream), service_fn(handler)).await;
        println!("  server conn ended: {r:?}");
      });
    }
  });

  // 1. hyper HTTP/1 client
  let io = TokioIo::new(TcpStream::connect(addr).await.unwrap());
  let (mut tx, conn) = client_h1::handshake::<_, Empty<Bytes>>(io).await.unwrap();
  let conn_task = tokio::spawn(conn);
  let res = tx.send_request(Request::get("/one").body(Empty::new()).unwrap()).await.unwrap();
  let body = res.into_body().collect().await.unwrap().to_bytes();
  println!("h1 client  -> {:?}", body);
  drop(tx);
  conn_task.await.unwrap().unwrap();

  // 2. hyper HTTP/2 client, prior knowledge (no TLS, no ALPN, no Upgrade)
  let io = TokioIo::new(TcpStream::connect(addr).await.unwrap());
  let (mut tx, conn) = client_h2::handshake::<_, _, Empty<Bytes>>(TokioExecutor::new(), io).await.unwrap();
  let conn_task = tokio::spawn(conn);
  let res = tx
    .send_request(Request::get("http://x/two").body(Empty::new()).unwrap())
    .await
    .unwrap();
  let body = res.into_body().collect().await.unwrap().to_bytes();
  println!("h2 client  -> {:?}", body);
  drop(tx);
  conn_task.await.unwrap().unwrap();

  // 3. raw bytes: an HTTP/1.1 request whose method happens to be PRI
  let mut tcp = TcpStream::connect(addr).await.unwrap();
  tcp.write_all(b"PRI /three HTTP/1.1\r\nconnection: close\r\n\r\n").await.unwrap();
  let mut out = String::new();
  tcp.read_to_string(&mut out).await.unwrap();
  println!("raw PRI    -> {:?}", out.lines().next().unwrap_or(""));
  println!("              body {:?}", out.rsplit("\r\n").next().unwrap_or(""));
  tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}

M2 Max、rustc 1.99.0、--release,连跑 3 次输出相同:

h1 client  -> b"HTTP/1.1 /one"
  server conn ended: Ok(())
h2 client  -> b"HTTP/2.0 /two"
  server conn ended: Ok(())
  server conn ended: Ok(())
raw PRI    -> "HTTP/1.1 200 OK"
              body "HTTP/1.1 /three"

第三条连接以 PRI 开头,和前言的前 4 个字节一样,第 5 个字节 / 和前言的 * 不符,切到 HTTP/1。已经读进 ReadVersion 缓冲的字节经 Rewind 还给 HTTP/1 的解析器,请求行完整,路径 /three 一个字节没少。

Connection:三个状态

serve_connection 返回的 Connection 里只有一个状态枚举 ConnState:

    enum ConnState<'a, I, S, E>
    where
        S: HttpService<Incoming>,
    {
        ReadVersion {
            #[pin]
            read_version: ReadVersion<I>,
            builder: Cow<'a, Builder<E>>,
            service: Option<S>,
        },
        H1 {
            #[pin]
            conn: Http1Connection<I, S>,
        },
        H2 {
            #[pin]
            conn: Http2Connection<I, S, E>,
        },
    }
stateDiagram-v2
  state "ReadVersion:偷看字节" as ReadVersion
  state "H1:http1 Connection" as H1
  state "H2:http2 Connection" as H2
  [*] --> ReadVersion: 未指定版本
  [*] --> H1: http1_only
  [*] --> H2: http2_only
  ReadVersion --> H1: 不符或 EOF
  ReadVersion --> H2: 24 字节全符
  H1 --> [*]
  H2 --> [*]

Connection::poll 是一个 loop:在 ReadVersion 状态,ready! 等出版本和 Rewind<I>,从 service 里 take() 出服务,调对应 Builder 的 serve_connection 建出 hyper 的连接,state.set(...) 换状态,再转一圈去 poll 新状态;在 H1 / H2 状态,直接返回内层连接的 poll 结果,错误 map_err(Into::into) 成 Box<dyn Error + Send + Sync>。所以 auto 连接的错误类型不是 hyper::Error,要拿到 hyper::Error 得从 box 里向下转型(见 hyper::Error)。

ReadVersion 状态里存着 builder 和 service,是因为判断出版本之后才能建连接。builder 是一个自定义的 Cow:默认借用(Cow::Borrowed(self)),所以 Connection<'a, ...> 带着 Builder 的生命周期。要把连接 spawn 出去,有两种做法:在任务里建 Builder(axum 0.8.9 的 serve 每条连接在 spawn 出的任务里 Builder::new),或者调 into_owned 把 Builder 克隆一份放进 Cow::Owned。只有还在 ReadVersion 状态时才需要克隆,已经是 H1 / H2 就原样搬过去。

ReadVersion 里还有一个 PhantomPinned,注释说是为了让这个 future 是 !Unpin、和 async trait 方法兼容。

graceful_shutdown 按状态分派:H1 / H2 调内层连接的 graceful_shutdown;ReadVersion 调 cancel,只把 cancelled 置成 true,下次 poll 时返回 io::ErrorKind::Interrupted 的 Cancelled 错误。它不唤醒任务,要靠调用者调完之后接着 poll。多条连接的停机怎么协调见 优雅停机。

http1_only 与 http2_only

http2_only 和 http1_only 只是填上 version,开头各有一句 assert!(self.version.is_none()),两个都调会 panic。serve_connection 看到版本已定,就跳过 ReadVersion:

        let state = match self.version {
            #[cfg(feature = "http1")]
            Some(Version::H1) => {
                let io = Rewind::new_buffered(io, Bytes::new());
                let conn = self.http1.serve_connection(io, service);
                ConnState::H1 { conn }
            }
            #[cfg(feature = "http2")]
            Some(Version::H2) => {
                let io = Rewind::new_buffered(io, Bytes::new());
                let conn = self.http2.serve_connection(io, service);
                ConnState::H2 { conn }
            }
            #[cfg(any(feature = "http1", feature = "http2"))]
            _ => ConnState::ReadVersion {
                read_version: read_version(io),
                builder: Cow::Borrowed(self),
                service: Some(service),
            },
        };

用不着塞回任何字节,还是包了一层空的 Rewind:ConnState 的 H1 / H2 两个变体里连接的 IO 类型写死成 Rewind<I>,走哪条路都得是这个类型。

📝NOTE
冷知识:http1_only / http2_only 是为 ALPN 加的

hyper 1.0 刚把 auto 搬进 hyper-util 时,它总是先读版本。hyper 的 issue #3511 指出这在 TLS 下多余,还给了坏客户端可乘之机:「this decision should have already been made by ALPN when running over TLS (or via prior knowledge), so the ReadVersion logic ends up adding some overhead and allows bad clients to use an HTTP version they didn't negotiate.」提出者要的是 0.14 里那样的 http2_only / http1_only。

hyper-util 的 #111(「This PR adds an option to choose http1 or http2 only for auto connections.」)加了这两个方法,第一个包含它的发布是 v0.1.4。它改了 serve_connection,就是上面那段 match self.version;serve_connection_with_upgrades 没有跟着改,于是两个方法的文档都多了一句「Does not do anything if used with serve_connection_with_upgrades」。

serve_connection_with_upgrades 不看 self.version,无条件从 ReadVersion 开始:

        UpgradeableConnection {
            state: UpgradeableConnState::ReadVersion {
                read_version: read_version(io),
                builder: Cow::Borrowed(self),
                service: Some(service),
            },
        }

实验:只开一种协议,两种客户端

两种设置(http1_only、http2_only)× 两个入口(serve_connection、serve_connection_with_upgrades)× 两种客户端,每格一条新连接。依赖同上。

use std::convert::Infallible;
use std::time::Duration;

use bytes::Bytes;
use http_body_util::{BodyExt, Empty, Full};
use hyper::body::Incoming;
use hyper::client::conn::{http1 as client_h1, http2 as client_h2};
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo};
use hyper_util::server::conn::auto;
use tokio::net::{TcpListener, TcpStream};

async fn handler(req: Request<Incoming>) -> Result<Response<Full<Bytes>>, Infallible> {
  Ok(Response::new(Full::new(Bytes::from(format!("{:?}", req.version())))))
}

// One connection served by `builder`, with or without upgrades; returns the server's result.
async fn serve_one(
  builder: auto::Builder<TokioExecutor>,
  upgrades: bool,
  h2_client: bool,
) -> (String, String) {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  let server = tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    let io = TokioIo::new(stream);
    let r = if upgrades {
      builder.serve_connection_with_upgrades(io, service_fn(handler)).await
    } else {
      builder.serve_connection(io, service_fn(handler)).await
    };
    match r {
      Ok(()) => "Ok(())".to_string(),
      Err(e) => format!("Err({})", chain(&*e)),
    }
  });
  let io = TokioIo::new(TcpStream::connect(addr).await.unwrap());
  let client = async {
    if h2_client {
      let (mut tx, conn) = client_h2::handshake::<_, _, Empty<Bytes>>(TokioExecutor::new(), io).await?;
      tokio::spawn(conn);
      let res = tx.send_request(Request::get("http://x/").body(Empty::new()).unwrap()).await?;
      res.into_body().collect().await.map(|c| c.to_bytes())
    } else {
      let (mut tx, conn) = client_h1::handshake::<_, Empty<Bytes>>(io).await?;
      tokio::spawn(conn);
      let res = tx.send_request(Request::get("/").body(Empty::new()).unwrap()).await?;
      res.into_body().collect().await.map(|c| c.to_bytes())
    }
  };
  let client = match tokio::time::timeout(Duration::from_secs(1), client).await {
    Ok(Ok(b)) => format!("{b:?}"),
    // the client-side error text races (canceled vs. broken pipe); keep only the fact
    Ok(Err(_)) => "error".to_string(),
    Err(_) => "timed out".to_string(),
  };
  let server = match tokio::time::timeout(Duration::from_secs(1), server).await {
    Ok(r) => r.unwrap(),
    Err(_) => "still running".to_string(),
  };
  (client, server)
}

fn chain(e: &(dyn std::error::Error + 'static)) -> String {
  let mut s = e.to_string();
  let mut src = e.source();
  while let Some(inner) = src {
    s.push_str(&format!(": {inner}"));
    src = inner.source();
  }
  s
}

#[tokio::main]
async fn main() {
  let builders: [(&str, fn() -> auto::Builder<TokioExecutor>); 2] = [
    ("http1_only", || auto::Builder::new(TokioExecutor::new()).http1_only()),
    ("http2_only", || auto::Builder::new(TokioExecutor::new()).http2_only()),
  ];
  for (name, make) in builders {
    for upgrades in [false, true] {
      let method = if upgrades { "serve_connection_with_upgrades" } else { "serve_connection" };
      for h2_client in [false, true] {
        let (client, server) = serve_one(make(), upgrades, h2_client).await;
        let who = if h2_client { "h2" } else { "h1" };
        println!("{name} + {method:<30} {who} client: {client:<11} server: {server}");
      }
    }
  }
}

连跑 3 次输出相同:

http1_only + serve_connection               h1 client: b"HTTP/1.1" server: Ok(())
http1_only + serve_connection               h2 client: error       server: Err(invalid HTTP version parsed (found HTTP2 preface))
http1_only + serve_connection_with_upgrades h1 client: b"HTTP/1.1" server: Ok(())
http1_only + serve_connection_with_upgrades h2 client: b"HTTP/2.0" server: Ok(())
http2_only + serve_connection               h1 client: error       server: Err(http2 error: connection error detected: unspecific protocol error detected)
http2_only + serve_connection               h2 client: b"HTTP/2.0" server: Ok(())
http2_only + serve_connection_with_upgrades h1 client: b"HTTP/1.1" server: Ok(())
http2_only + serve_connection_with_upgrades h2 client: b"HTTP/2.0" server: Ok(())
  • serve_connection 下,设置起作用:http1_only 拒绝 HTTP/2 客户端,报错正是 hyper HTTP/1 解析器认出连接前言时的 found HTTP2 preface(解析与编码);http2_only 拒绝 HTTP/1 客户端,h2 把 GET / HTTP/1.1 当成坏前言,报 PROTOCOL_ERROR。
  • serve_connection_with_upgrades 下,两种设置都被忽略,四格全部成功。文档那句话属实:用 http2_only 加升级入口,HTTP/1 客户端照样进得来。
  • 客户端一列只记成不成功。被拒的 HTTP/2 客户端拿到的错误文本取决于时机:15 次运行里 14 次是 operation was canceled: connection closed,1 次是写 socket 时的 broken pipe,所以程序里只打印 error。

升级:UpgradeableConnection 与 upgrade::downcast

UpgradeableConnection 的状态枚举 UpgradeableConnState 和 ConnState 一样是三态,区别在 H1 那一支:判断出 HTTP/1 后调的是 serve_connection(io, service).with_upgrades(),所以入口要求 I: Send。H2 一支和 Connection 相同,HTTP/2 上的 CONNECT 升级由 hyper 的 HTTP/2 服务端自己处理(见 HTTP/2 服务端)。

问题出在 IO 的类型上。协议升级 一章讲过,Upgraded::downcast::<T> 按 TypeId 比对,T 必须和交给 hyper 的 IO 类型完全一致。经过 auto,hyper 拿到的是 Rewind<TokioIo<TcpStream>>,而 Rewind 是 hyper-util 的私有类型,用户写不出这个 T。hyper 的 issue #3704 报告的就是这个:「Upgraded::downcast_ref is unusable with hyper_util::server::conn::auto」。hyper-util 的解法是 auto::upgrade::downcast,文档自称「a kludge」:

pub fn downcast<T>(upgraded: Upgraded) -> Result<Parts<T>, Upgraded>
where
    T: Read + Write + Unpin + 'static,
{
    let hyper::upgrade::Parts {
        io: rewind,
        mut read_buf,
        ..
    } = upgraded.downcast::<Rewind<T>>()?;

    if let Some(pre) = rewind.pre {
        read_buf = if read_buf.is_empty() {
            pre
        } else {
            let mut buf = BytesMut::from(read_buf);

            buf.extend_from_slice(&pre);

            buf.freeze()
        };
    }

    Ok(Parts {
        io: rewind.inner,
        read_buf,
    })
}

它在 crate 内部能写出 Rewind<T>,替你做 hyper 的 downcast,再把两层「多读的字节」合并:hyper 读缓冲里剩下的 read_buf 在前,Rewind 里还没被 hyper 读走的 pre 在后。如果 pre 还有剩,说明 hyper 只读了它的一部分,read_buf 里的字节都来自 pre 的前半段,这个先后顺序就是字节在线上的顺序。

验证一下:用 serve_connection_with_upgrades 接一个升级请求,请求和新协议的第一批字节 hello 在同一次写里发过来,服务里先试 hyper 的 downcast,再试 hyper-util 的:

use std::convert::Infallible;

use bytes::Bytes;
use http_body_util::Empty;
use hyper::body::Incoming;
use hyper::service::service_fn;
use hyper::{Request, Response, StatusCode};
use hyper_util::rt::{TokioExecutor, TokioIo};
use hyper_util::server::conn::auto;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};

async fn handler(mut req: Request<Incoming>) -> Result<Response<Empty<Bytes>>, Infallible> {
  let on = hyper::upgrade::on(&mut req);
  tokio::spawn(async move {
    let up = on.await.unwrap();
    // 1. hyper's own downcast, with the type we handed to serve_connection_with_upgrades
    let up = match up.downcast::<TokioIo<TcpStream>>() {
      Ok(_) => unreachable!(),
      Err(up) => {
        println!("hyper::upgrade::Upgraded::downcast::<TokioIo<TcpStream>>: Err");
        up
      }
    };
    // 2. hyper-util's downcast
    match auto::upgrade::downcast::<TokioIo<TcpStream>>(up) {
      Ok(parts) => {
        println!("auto::upgrade::downcast::<TokioIo<TcpStream>>: Ok, read_buf = {:?}", parts.read_buf);
        let mut tcp = parts.io.into_inner();
        tcp.write_all(b"bye").await.unwrap();
      }
      Err(_) => println!("auto::upgrade::downcast: Err"),
    }
  });
  Ok(
    Response::builder()
      .status(StatusCode::SWITCHING_PROTOCOLS)
      .header("connection", "upgrade")
      .header("upgrade", "foo")
      .body(Empty::new())
      .unwrap(),
  )
}

#[tokio::main]
async fn main() {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  tokio::spawn(async move {
    let (stream, _) = listener.accept().await.unwrap();
    let builder = auto::Builder::new(TokioExecutor::new());
    builder
      .serve_connection_with_upgrades(TokioIo::new(stream), service_fn(handler))
      .await
      .unwrap();
  });
  let mut tcp = TcpStream::connect(addr).await.unwrap();
  // the request and the first bytes of the new protocol in one write
  tcp
    .write_all(b"GET / HTTP/1.1\r\nconnection: upgrade\r\nupgrade: foo\r\n\r\nhello")
    .await
    .unwrap();
  let mut out = String::new();
  tcp.read_to_string(&mut out).await.unwrap();
  println!("client read: {:?}", out.rsplit("\r\n").next().unwrap());
}

连跑 3 次输出相同:

hyper::upgrade::Upgraded::downcast::<TokioIo<TcpStream>>: Err
auto::upgrade::downcast::<TokioIo<TcpStream>>: Ok, read_buf = b"hello"
client read: "bye"

hyper 的 downcast 拿着你传进去的类型也失败;hyper-util 的成功,hello 在 read_buf 里。这次 pre 是空的:ReadVersion 第一次读就拿到 24 个字节,第一个字节 G 已经不符,这 24 个字节随后被 hyper 一次读走;hello 是 hyper 读请求头时多读进自己缓冲区的。

❗IMPORTANT
动脑筋:只发半个前言的客户端,什么时候被关掉?

auto::Builder 给 HTTP/1 设了 timer 和 500 ms 的 header_read_timeout。四种客户端:连上什么都不发;发 GET / HTTP/1.1\r\n 然后不动;发 PRI * HTTP/2.0\r\n(前言的前 16 个字节)然后不动;发 PRI 后关掉写方向。作为对照,同样的设置也在 http1::Builder 上跑一遍。客户端最多等 2 秒。

use std::convert::Infallible;
use std::time::{Duration, Instant};

use bytes::Bytes;
use http_body_util::Full;
use hyper::body::Incoming;
use hyper::server::conn::http1;
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::{TokioExecutor, TokioIo, TokioTimer};
use hyper_util::server::conn::auto;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::mpsc;

async fn handler(_: Request<Incoming>) -> Result<Response<Full<Bytes>>, Infallible> {
  Ok(Response::new(Full::new(Bytes::from("hi"))))
}

const TIMEOUT: Duration = Duration::from_millis(500);

async fn spawn_server(use_auto: bool) -> (std::net::SocketAddr, mpsc::UnboundedReceiver<String>) {
  let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
  let addr = listener.local_addr().unwrap();
  let (tx, rx) = mpsc::unbounded_channel();
  tokio::spawn(async move {
    loop {
      let (stream, _) = listener.accept().await.unwrap();
      let tx = tx.clone();
      tokio::spawn(async move {
        let io = TokioIo::new(stream);
        let r = if use_auto {
          let mut b = auto::Builder::new(TokioExecutor::new());
          b.http1().timer(TokioTimer::new()).header_read_timeout(TIMEOUT);
          b.serve_connection(io, service_fn(handler)).await.map_err(|e| e.to_string())
        } else {
          http1::Builder::new()
            .timer(TokioTimer::new())
            .header_read_timeout(TIMEOUT)
            .serve_connection(io, service_fn(handler))
            .await
            .map_err(|e| e.to_string())
        };
        let _ = tx.send(format!("{r:?}"));
      });
    }
  });
  (addr, rx)
}

async fn case(addr: std::net::SocketAddr, rx: &mut mpsc::UnboundedReceiver<String>, name: &str, bytes: &[u8], eof: bool) {
  let start = Instant::now();
  let mut tcp = TcpStream::connect(addr).await.unwrap();
  tcp.write_all(bytes).await.unwrap();
  if eof {
    tcp.shutdown().await.unwrap();
  }
  let mut got = Vec::new();
  let mut buf = [0u8; 1024];
  let end = loop {
    match tokio::time::timeout(Duration::from_secs(2) - start.elapsed(), tcp.read(&mut buf)).await {
      Ok(Ok(0)) => break format!("EOF at {} ms", start.elapsed().as_millis()),
      Ok(Ok(n)) => got.extend_from_slice(&buf[..n]),
      Ok(Err(e)) => break format!("{e} at {} ms", start.elapsed().as_millis()),
      Err(_) => break "still open at 2000 ms".to_string(),
    }
  };
  let first = String::from_utf8_lossy(&got).lines().next().unwrap_or("").to_string();
  drop(tcp);
  let srv = tokio::time::timeout(Duration::from_millis(200), rx.recv())
    .await
    .ok()
    .flatten()
    .unwrap_or_else(|| "(none)".into());
  println!("  {name:<22} {end:<22} got {first:?}  server: {srv}");
}

#[tokio::main]
async fn main() {
  for use_auto in [false, true] {
    println!("{}:", if use_auto { "auto::Builder" } else { "http1::Builder" });
    let (addr, mut rx) = spawn_server(use_auto).await;
    case(addr, &mut rx, "nothing", b"", false).await;
    case(addr, &mut rx, "\"GET / HTTP/1.1\\r\\n\"", b"GET / HTTP/1.1\r\n", false).await;
    case(addr, &mut rx, "\"PRI * HTTP/2.0\\r\\n\"", b"PRI * HTTP/2.0\r\n", false).await;
    case(addr, &mut rx, "\"PRI\" then EOF", b"PRI", true).await;
  }
}

依赖同第一个实验,tokio 再加 sync。auto::Builder 下,第一行和第三行的连接各在多少毫秒被服务端关掉?

动脑筋答案

M2 Max、rustc 1.99.0、--release,连跑 3 次,只有 auto::Builder 第二行的时间有一次是 502 ms,其余相同:

http1::Builder:
  nothing                EOF at 503 ms          got ""  server: Err("read header from client timeout")
  "GET / HTTP/1.1\r\n"   EOF at 503 ms          got ""  server: Err("read header from client timeout")
  "PRI * HTTP/2.0\r\n"   EOF at 0 ms            got "HTTP/1.1 400 Bad Request"  server: Err("invalid HTTP version parsed")
  "PRI" then EOF         EOF at 0 ms            got ""  server: Err("connection closed before message completed")
auto::Builder:
  nothing                still open at 2000 ms  got ""  server: Ok(())
  "GET / HTTP/1.1\r\n"   EOF at 503 ms          got ""  server: Err("read header from client timeout")
  "PRI * HTTP/2.0\r\n"   still open at 2000 ms  got ""  server: Err("invalid HTTP version parsed")
  "PRI" then EOF         EOF at 0 ms            got ""  server: Err("connection closed before message completed")

答案:都不会被关。2 秒后连接还开着,是客户端自己放弃的。

  • nothing:http1::Builder 在 503 ms 时按 header_read_timeout 关掉连接,不回任何字节(没有 408)。auto 下连接停在 ReadVersion,没有计时器,2 秒后还开着。客户端 drop 掉 socket 以后,ReadVersion 读到 EOF,落到 HTTP/1,hyper 看到一条一个字节都没收到就结束的连接,返回 Ok(())。
  • GET / HTTP/1.1\r\n:第一个字节 G 就不符,auto 立刻切到 HTTP/1,http1::Connection 建好后读头超时开始计,503 ms 关闭,和 http1::Builder 一样。
  • PRI * HTTP/2.0\r\n:16 个字节和前言前 16 个字节完全一致,ReadVersion 还在等剩下 8 个,2 秒后连接还开着。客户端 drop 之后 ReadVersion 读到 EOF,带着这 16 个字节切到 HTTP/1,解析出版本号 HTTP/2.0,报 invalid HTTP version parsed(这时 400 响应已经无处可写)。对照组 http1::Builder 直接解析请求行,0 ms 回 400。
  • PRI 加 EOF:前 3 个字节相符,接着读到 EOF,切到 HTTP/1,hyper 拿着半截请求遇到 EOF,报 connection closed before message completed,和对照组一样。

所以 auto 前面要防慢速连接,header_read_timeout 不够:判断协议这一步需要自己在外面套超时,或者交给前面的代理。

小结

  • auto::Builder 就是 hyper 的 http1::Builder 和 http2::Builder 加一个可选版本;子 Builder 的选项是逐个转发的,0.1.21 少转发了 max_header_size 和 allow_multiple_spaces_in_request_line_delimiters;默认不设 timer。
  • ReadVersion 用 24 字节的栈缓冲读连接前言,先假定 HTTP/2,新读的字节一个不符或者读到 EOF 就改判 HTTP/1;读过的字节由 hyper-util 的 Rewind 塞回去。这一步没有超时,header_read_timeout 管不到。
  • Connection 是 ReadVersion / H1 / H2 三态枚举,借用 Builder,spawn 前用 into_owned;在 ReadVersion 里 graceful_shutdown 会让连接以 Cancelled 结束。
  • http1_only / http2_only 跳过 ReadVersion(hyper-util #111,v0.1.4,为 ALPN 场景加的);serve_connection_with_upgrades 不看这两个设置,实测四种组合全部放行。
  • 经过 auto 的升级连接,IO 是 Rewind<I>,hyper 的 downcast::<I> 失败,要用 auto::upgrade::downcast,它合并 read_buf 和 pre 两层多读的字节。

💬 关联讨论

发起讨论
加载中...