Loader 与 N+1
主线位置:查询主线的「第二趟」。第一条语句照常走
Select→Selector→query_all拿回一批父Model;Loader 从这批Model里读出外键(关系的描述来自RelationDef),拼成一个WHERE (col) IN (...)再走一遍查询主线,最后在内存里按键把子记录分给各自的父记录。2.0 的 Entity Loader 是#[sea_orm::model]生成的一层包装,它决定哪些关系并进第一条语句(join),哪些交给 Loader。
flowchart TB
parents["父 Model 切片"] --> keys["按 from_col 取键"]
keys --> empty{"键为空"}
empty -->|是| none["直接返回 空 Vec"]
empty -->|否| uniq["unique 去重"]
uniq --> inq["一条 IN 查询"]
inq --> map["按 to_col 建 HashMap"]
map --> out["按父的顺序装回"]
实验 crate 依赖 sea-orm =2.0.4(sqlx-postgres、runtime-tokio-rustls、macros、with-json、postgres-array、serde,加上默认 feature),平台 M2 Max、macOS、rustc 1.99.0、Postgres 16.14。实体用 2.0 的 dense 写法:users(has_one profile、has_many posts)、profile(user_id 唯一)、post(belongs_to user、has_many comments、经 post_tag 多对多到 tag)、comment。数据:三个用户 ann、bob、cat;ann 写了 a1、a2,bob 写了 b1;a1 有两条评论、b1 有一条;cat 没有 profile。SQL 都是 sqlx 的语句日志(sqlx=info),即实际发给服务端的文本。
N+1:先看问题
从三篇文章出发,逐篇 find_related 查评论(src/bin/n_plus_1.rs):
let posts = post::Entity::find().order_by_asc(post::Column::Id).all(&db).await?;
for p in &posts {
let cs = p.find_related(comment::Entity).all(&db).await?;
println!("{} -> {}", p.title, cs.len());
}
INFO sqlx::query: summary="SELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" …" db.statement="\n\nSELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" FROM \"post\" ORDER BY \"post\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=1.494125ms elapsed_secs=0.001494125
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" INNER JOIN \"post\" ON \"post\".\"id\" = \"comment\".\"post_id\" WHERE \"post\".\"id\" = $1\n" rows_affected=2 rows_returned=2 elapsed=1.445333ms elapsed_secs=0.001445333
a1 -> 2
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" INNER JOIN \"post\" ON \"post\".\"id\" = \"comment\".\"post_id\" WHERE \"post\".\"id\" = $1\n" rows_affected=0 rows_returned=0 elapsed=274.583µs elapsed_secs=0.000274583
a2 -> 0
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" INNER JOIN \"post\" ON \"post\".\"id\" = \"comment\".\"post_id\" WHERE \"post\".\"id\" = $1\n" rows_affected=1 rows_returned=1 elapsed=224.042µs elapsed_secs=0.000224042
b1 -> 1
一篇文章一条语句,N 篇就是 1 + N 条。每条都要从池里借一次连接(一次 find 的全过程),往返时间按 N 线性增长。find_related 本身的 SQL 怎么来的见 join 与 find_related;本页讲怎么把 N 条压成一条。
LoaderTrait:一批父记录,一条 IN
LoaderTrait 实现在 Vec<M> 和 &[M] 上(Vec 的实现只是转给 as_slice()),常用的三个方法:
| 方法 | 返回 | 要求 | 语句数 |
|---|---|---|---|
load_one(R, db) | Vec<Option<R::Model>> | Related<R>::to() 是 HasOne(belongs_to 或 has_one) | 1 |
load_many(R, db) | Vec<Vec<R::Model>> | 任何 Related<R>,包括带 via 的多对多 | 1 |
load_many_to_many(R, V, db) | Vec<Vec<R::Model>> | 多对多,V 必须是中间表 | 2 |
另外四个 load_self* 给自引用:自引用不生成 Related<Entity> for Entity(见关系的自引用一节),所以 load_self / load_self_many 要你点名关系分支(relation_enum),多对多的 load_self_via* 走 RelatedSelfVia。
第一个参数是 EntityOrSelect<R>:可以直接传实体,也可以传一个已经加了条件的 Select<R>。返回值和输入一一对应:第 i 个元素属于第 i 个父记录。
三个方法最后都落到 loader_impl_impl。没有中间表的那一支:
let keys = items
.map(|model| get_key_from_model(&rel_def.from_col, model))
.collect::<Result<Vec<_>, _>>()?;
if keys.is_empty() {
return Ok(Vec::new());
}
let condition = prepare_condition::<Model>(
&rel_def.to_tbl,
&rel_def.from_col,
&rel_def.to_col,
&keys,
db.get_database_backend(),
)?;
let stmt = QueryFilter::filter(stmt, condition);
let data = stmt.all(db).await?;
let mut hashmap: HashMap<ValueTuple, T> = Default::default();
for item in data {
let key = get_key_from_model(&rel_def.to_col, &item)?;
let holder = hashmap.entry(key).or_default();
holder.add(item.into());
}
(keys, hashmap)
// ...
let result: Vec<T> = keys
.iter()
.map(|key: &ValueTuple| hashmap.get(key).cloned().unwrap_or_default())
.collect();
四步:从每个父 Model 按 rel_def.from_col 取出键(get_key_from_model 用 Column::from_str 把列名变回 Column,再 model.get(col)),拼条件、查一次,按子记录的 to_col 建 HashMap<ValueTuple, T>,最后按父记录的顺序逐个查表。T 是 Vec<_>(load_many)或 Option<_>(load_one),由私有 trait Container 统一成一个 add。
from_col / to_col 的方向就是 RelationDef 的方向。post 的 Related<user>::to() 是 from_col = post.user_id、to_col = users.id,所以 load_one(user) 收集 post.user_id,条件写在 users.id 上;user 的 Related<post>::to() 是反过来的 has_many,load_many(post) 收集 users.id,条件写在 post.user_id 上。
实测(src/bin/loader.rs,posts 是 a1、a2、b1):
let authors = posts.load_one(user::Entity, &db).await?;
let comments = posts.load_many(comment::Entity, &db).await?;
let tags = posts.load_many(tag::Entity, &db).await?;
let tags = posts.load_many_to_many(tag::Entity, post_tag::Entity, &db).await?;
--- load_one(user) [belongs_to]
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2)) ORDER BY \"users\".\"id\" ASC\n" rows_affected=2 rows_returned=2 elapsed=1.183459ms elapsed_secs=0.001183459
a1 -> Some("ann")
a2 -> Some("ann")
b1 -> Some("bob")
--- load_many(comment) [has_many]
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" WHERE (\"comment\".\"post_id\") IN (($1), ($2), ($3)) ORDER BY \"comment\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=685.583µs elapsed_secs=0.000685583
a1 -> ["c1", "c2"]
a2 -> []
b1 -> ["c3"]
--- load_many(tag) [via post_tag]
INFO sqlx::query: summary="SELECT \"tag\".\"id\", \"tag\".\"name\", \"post_tag\".\"post_id\" …" db.statement="\n\nSELECT \"tag\".\"id\", \"tag\".\"name\", \"post_tag\".\"post_id\" FROM \"tag\" INNER JOIN \"post_tag\" ON \"post_tag\".\"tag_id\" = \"tag\".\"id\" WHERE (\"post_tag\".\"post_id\") IN (($1), ($2), ($3)) ORDER BY \"tag\".\"id\" ASC\n" rows_affected=4 rows_returned=4 elapsed=1.178042ms elapsed_secs=0.001178042
a1 -> ["rust", "sql"]
a2 -> ["rust"]
b1 -> ["orm"]
--- load_many_to_many(tag, post_tag) [legacy]
INFO sqlx::query: summary="SELECT \"post_tag\".\"post_id\", \"post_tag\".\"tag_id\" FROM …" db.statement="\n\nSELECT \"post_tag\".\"post_id\", \"post_tag\".\"tag_id\" FROM \"post_tag\" WHERE (\"post_tag\".\"post_id\") IN (($1), ($2), ($3))\n" rows_affected=4 rows_returned=4 elapsed=501.084µs elapsed_secs=0.000501084
INFO sqlx::query: summary="SELECT \"tag\".\"id\", \"tag\".\"name\" FROM …" db.statement="\n\nSELECT \"tag\".\"id\", \"tag\".\"name\" FROM \"tag\" WHERE (\"tag\".\"id\") IN (($1), ($2), ($3)) ORDER BY \"tag\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=402.041µs elapsed_secs=0.000402041
a1 -> ["rust", "sql"]
a2 -> ["rust"]
b1 -> ["orm"]
四次调用都没有 N+1,而且都不 join 父表:父记录已经在内存里,Loader 只需要它们的键。区别在多对多:
load_many(tag)走via分支:一条语句,tag INNER JOIN post_tag,额外选出"post_tag"."post_id"(代码里叫select_also_dyn_model,一个只有键列的动态模型),用它来分组。所以返回的tag行是「每对 (tag, post) 一行」,rust 这一行被 a1、a2 各拿到一份。load_many_to_many是 1.x 留下的写法,load_many_to_many先查中间表、在内存里建「post 键 → tag 键」的表,再按 tag 键查一次tag。两条语句,第二条的IN只有 3 个值(四行中间表里 tag 1 出现两次)。trait 的文档注释写着「In SeaORM 2.0load_manyalready support M-N relations so this method is not needed, only kept as legacy」。
load_one 要求关系是 HasOne,在 user 上调 load_one(post::Entity) 不是编译错误,而是运行时返回 query_err("Relation is HasMany instead of HasOne"):Related<R> 不区分基数,检查只能放在运行期。
条件长什么样:Postgres 的元组 IN
日志里的条件是 ("users"."id") IN (($1), ($2)):左边一个单元素元组,右边每个键一个元组。这是 Postgres 专有的分支,prepare_condition 只在 DbBackend::Postgres 时走 prepare_condition_with_save_as,其他后端走普通的 col IN (...):
let keys = keys.iter().unique();
let (from_cols, to_cols) = resolve_column_pairs::<Model>(table, from, to)?;
// ...
let value_tuples = keys
.map(|key| {
let key_arity = key.arity();
if arity != key_arity {
return Err(arity_mismatch(arity, key));
}
// For Postgres, we need to use `AS` to cast the value to the correct type
Ok(apply_save_as::<Model>(&from_cols, key.clone()))
})
.collect::<Result<Vec<_>, DbErr>>()?;
// Build `(c1, c2, ...) IN ((v11, v12, ...), (v21, v22, ...), ...)`
let expr = Expr::tuple(create_table_columns(table, to)).is_in(value_tuples);
每个值都经过 from 那一列的 save_as(apply_save_as)。普通整数列的 save_as 原样返回,所以日志里只是 ($1);Postgres enum 列的 save_as 会给值加上转换到 enum 类型的 cast。这是 #2759 修的:它关掉的 issue #2756 报告 load_many 在键是 enum 时失败,报 42883 operator does not exist: tea = text(键值按 text 绑定,和 enum 列比较不了)。单列、复合键走同一段代码,复合键就是 ("a", "b") IN (($1, $2), ($3, $4))。
去重、顺序、空输入
去重:keys.iter().unique()(itertools)保留第一次出现的顺序去掉重复的键。上面 load_one(user) 有三篇文章、两个作者,IN 里就是两个参数。去重只发生在拼条件这一步,keys 本身没去重,装回结果时 a1、a2 各自查一次 HashMap,都拿到 ann 的一份 clone。
#2569(标题「fix: In the prepare_condition function, if the value is hashable, only unique values are retained.」)之前,prepare_condition 的开头是一行 // TODO when value is hashable, retain only unique values 和 let keys = keys.to_owned();:一千篇文章出自同一个作者,IN 里就有一千个相同的参数。PR 说明写的是「I found the TODO in the codebase and have responded to it.」,当时的做法是把键收进 HashSet 再转回 Vec,同时加了测试 test_load_one_duplicate_keys。首次发布于 1.1.11,CHANGELOG 记为「Loader: retain only unique key values in the query condition」。本站用的 1.1.20 已经包含它;2.0.4 换成了上面的 unique(),效果相同,只是参数顺序不再随 HashSet 变化。
顺序:外层顺序就是父记录的顺序(keys.iter().map(...)),和数据库返回的顺序无关。每组里子记录的顺序是第二条语句的顺序:日志里自动带了 ORDER BY "comment"."id" ASC,因为直接传实体时 EntityOrSelect for E 是 E::find().order_by_id_asc()。这是 2.0 的改动,1.1.20 的同一个 impl 只是 E::find(),组内顺序由数据库决定。传 Select 时原样使用,排序、过滤都是你的:
let comments = posts
.load_many(
comment::Entity::find().filter(comment::Column::Body.ne("c3")).order_by_desc(comment::Column::Id),
&db,
)
.await?;
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" WHERE \"comment\".\"body\" <> $1 AND (\"comment\".\"post_id\") IN (($2), ($3), ($4)) ORDER BY \"comment\".\"id\" DESC\n" rows_affected=2 rows_returned=2 elapsed=472.917µs elapsed_secs=0.000472917
a1 -> ["c2", "c1"]
a2 -> []
b1 -> []
你的条件和 Loader 的 IN 用 AND 连起来,$1 是 "c3",IN 的参数排在后面。
空输入:键为空时在拼 SQL 之前就 return Ok(Vec::new())。实验里 Vec::<post::Model>::new().load_one(user::Entity, &db) 返回长度 0,sqlx 日志里没有任何一行。所以 列与值 里 is_in([]) 生成的 $1 = $2 在 Loader 里碰不到。
问: 想要「每篇文章最新的一条评论」,给 Loader 传 comment::Entity::find().order_by_desc(comment::Column::Id).limit(1) 行不行?
答: 不行。Loader 只往这个 Select 上加一个 IN 条件,LIMIT 作用在整条语句上,不是每组一条。src/bin/loader_limit.rs 的实际输出:
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" WHERE (\"comment\".\"post_id\") IN (($1), ($2), ($3)) ORDER BY \"comment\".\"id\" DESC LIMIT $4\n" rows_affected=1 rows_returned=1 elapsed=2.526292ms elapsed_secs=0.002526292
a1 -> []
a2 -> []
b1 -> ["c3"]
三篇文章一共只拿到一条评论,a1 的两条评论都丢了。每组取前 N 条要用窗口函数或 LATERAL,Loader 做不到。
很大的输入:不分批
loader_impl_impl 里没有任何分批逻辑,所有去重后的键都放进同一条语句,每个键占一个参数(复合键占 arity 个)。Postgres 扩展协议的 Bind 消息用 16 位整数表示参数个数,sqlx 的 PgConnection::run 在写任何字节之前先做 u16::try_from(arguments.len())。src/bin/loader_big.rs 在内存里造 n 篇文章,每篇指向一个不同的用户(库里只有三个用户),再 load_one(user):
for n in [65_535_i32, 65_536, 70_000] {
let posts: Vec<post::Model> = (1..=n)
.map(|i| post::Model { id: i, user_id: i, title: String::new() })
.collect();
let r = posts.load_one(user::Entity, &db).await;
// ...
}
// 70 000 parents but only 3 distinct foreign keys
let posts: Vec<post::Model> = (1..=70_000)
.map(|i| post::Model { id: i, user_id: i % 3 + 1, title: String::new() })
.collect();
输出(db.statement 有六万多个占位符,从第 17 个开始截掉,标为 ...):
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2), ($3), ($4), ($5), ($6), ($7), ($8), ($9), ($10), ($11), ($12), ($13), ($14), ($15), ($16), ($17 ... rows_affected=3 rows_returned=3 elapsed=67.523917ms elapsed_secs=0.067523917
n=65535: Ok, 65535 slots, 3 found
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2), ($3), ($4), ($5), ($6), ($7), ($8), ($9), ($10), ($11), ($12), ($13), ($14), ($15), ($16), ($17 ... rows_affected=0 rows_returned=0 elapsed=140.667µs elapsed_secs=0.000140667
n=65536: Err(Query(SqlxError(Protocol("PgConnection::run(): too many arguments for query: 65536 (sqlx_postgres::connection::executor:222)"))))
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2), ($3), ($4), ($5), ($6), ($7), ($8), ($9), ($10), ($11), ($12), ($13), ($14), ($15), ($16), ($17 ... rows_affected=0 rows_returned=0 elapsed=154.917µs elapsed_secs=0.000154917
n=70000: Err(Query(SqlxError(Protocol("PgConnection::run(): too many arguments for query: 70000 (sqlx_postgres::connection::executor:222)"))))
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2), ($3)) ORDER BY \"users\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=1.081625ms elapsed_secs=0.001081625
n=70000, 3 distinct keys: Ok, 70000 slots, 70000 found
- 65535 个不同的键正好能发出去,67 ms。
- 65536 和 70000 都是
DbErr::Query(SqlxError(Protocol(..)))。sqlx 照样打了一行语句日志(日志在语句结束时记,失败的也记),但 0.14 ms、rows_returned=0,什么也没发到服务端。sea-orm 不分批、不降级,错误原样往上抛。 - 限制按去重后的键数算:七万篇文章只有三个作者时一切正常,
IN里只有三个参数。
这个错误本身和两种绕法(分批、绑数组用 = ANY($1))在 sqlx 卷的 too many arguments for query 里。用 Loader 时最直接的办法是自己把父记录切片,每片几千条分别 load_*,再把结果按顺序拼起来;返回值和输入一一对应,拼接不会错位。另外,每种 IN 长度都是一条不同的 SQL 文本,sqlx 的语句缓存按文本做键,批次大小不固定时会缓存很多条只差参数个数的预备语句。
Entity Loader:2.0 的 load().with(..)
#[sea_orm::model] 给每个实体多生成一个 EntityLoader(DeriveEntityModel 与 dense 格式 里列过)。cargo expand --lib ent::user 的关键几段:
pub struct EntityLoader {
select: sea_orm::Select<Entity>,
with: EntityLoaderWith,
nest: EntityLoaderNest,
}
// ...
pub struct EntityLoaderWith {
/// Generated by sea-orm-macros
pub profile: bool,
/// Generated by sea-orm-macros
pub posts: bool,
}
// ...
impl Entity {
/// Generated by sea-orm-macros
pub fn load() -> EntityLoader {
EntityLoader {
select: Entity::find(),
with: Default::default(),
nest: Default::default(),
}
}
}
EntityLoaderWith 是每个关系字段一个 bool,EntityLoaderNest 是每个关系字段再挂一个对方实体的 EntityLoaderWith,所以嵌套只有两层。with(param) 收 R(要求 Entity: Related<R>)或二元组 (R, S)(再要求 R: Related<S>),按目标表的 table_ref() 去匹配字段,把对应的 bool 置位;没有 Related 的实体在类型上就进不来。EntityLoader 实现了 QueryFilter、QueryOrder,filter、order_by 改的是里面那个 select。
真正干活的是 fetch:
async fn fetch<C: sea_orm::ConnectionTrait>(
mut self,
db: &C,
page: u64,
page_size: u64,
) -> Result<Vec<ModelEx>, sea_orm::DbErr> {
let select = self.select;
let mut loaded = EntityLoaderWith::default();
let select = if self.with.profile && self.nest.profile.is_empty() {
self.with.profile = false;
loaded.profile = true;
select.find_also(Entity, super::profile::Entity)
} else {
select.select_also_fake(super::profile::Entity)
};
let models = if page_size != 0 {
select.paginate(db, page_size).fetch_page(page).await?
} else {
select.all(db).await?
};
// ...
let models = Self::load(models, &self.with, &self.nest, db).await?;
Ok(models)
}
profile 是一对一,而且没有嵌套时,把 with.profile 清掉,改用 find_also(left_join + select_also)并进主查询;否则用 select_also_fake,只给主表起 A_ 前缀,不 join。接着 Self::load 处理还亮着的 with 位:
if with.posts {
let posts = models
.as_slice()
.load_many_ex(super::post::Entity, db)
.await?;
let posts = <super::post::EntityLoader>::load_nest_nest(
posts,
&nest.posts,
db,
)
.await?;
for (model, posts) in models.iter_mut().zip(posts) {
model.posts = posts.into();
}
}
load_many_ex 就是上一节的 loader_impl_impl,只是产出 ModelEx。嵌套的第二层在 load_nest_nest 里对 Vec<Vec<ModelEx>> 再调一次(NestedLoaderTrait 先把二维展平成一批键)。
哪些关系 join、哪些走 Loader,由宏在 from_schema 里决定:
if matches!(&field.kind, EntityLoaderFieldKind::HasOne) && !duplicate_entity {
select_arity += 1;
if select_arity <= 3 {
field.expand_select_one_into(&mut output);
}
}
field.expand_load_into(&mut output, duplicate_entity);
| 关系字段 | 主查询里 LEFT JOIN | 交给 Loader |
|---|---|---|
has_one / belongs_to,前两个,没有嵌套 | 是 | 否 |
has_one / belongs_to,带了嵌套 (R, S) | 否 | 是(load_one_ex) |
has_one / belongs_to,第三个起 | 否 | 是 |
| 两个字段指向同一个实体 | 否 | 是(要写 relation_enum) |
has_many,含 via 多对多 | 否 | 是(load_many_ex) |
上限是 3 列「实体组」(主实体加两个),对应 SelectThree。每个字段都生成了 expand_load_into(expand_load_into),join 不了的一对一照样能加载,只是多一条语句。
src/bin/entity_loader.rs 的实测:
let users = user::Entity::load().all(&db).await?; // A
let users = user::Entity::load().with(profile::Entity).all(&db).await?; // B
let users = user::Entity::load() // C
.with(profile::Entity)
.with((post::Entity, comment::Entity))
.with((post::Entity, tag::Entity))
.all(&db)
.await?;
let users = user::Entity::load().with((profile::Entity, user::Entity)).all(&db).await?; // D
let p = post::Entity::load() // E
.with(user::Entity)
.filter(post::Column::Title.eq("b1"))
.one(&db)
.await?;
--- A. load() alone
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\" FROM \"users\"\n" rows_affected=3 rows_returned=3 elapsed=5.238709ms elapsed_secs=0.005238709
profile unloaded=true posts unloaded=true posts.is_empty()=true posts.len()=0
--- B. with(profile)
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\", \"profile\".\"id\" AS \"B_id\", \"profile\".\"picture\" AS \"B_picture\", \"profile\".\"user_id\" AS \"B_user_id\" FROM \"users\" LEFT JOIN \"profile\" ON \"users\".\"id\" = \"profile\".\"user_id\"\n" rows_affected=3 rows_returned=3 elapsed=1.537542ms elapsed_secs=0.001537542
ann: profile=Some("ann.png")
bob: profile=Some("bob.png")
cat: profile=None
--- C. with(profile).with((post, comment)).with((post, tag))
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\", \"profile\".\"id\" AS \"B_id\", \"profile\".\"picture\" AS \"B_picture\", \"profile\".\"user_id\" AS \"B_user_id\" FROM \"users\" LEFT JOIN \"profile\" ON \"users\".\"id\" = \"profile\".\"user_id\"\n" rows_affected=3 rows_returned=3 elapsed=408.625µs elapsed_secs=0.000408625
INFO sqlx::query: summary="SELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" …" db.statement="\n\nSELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" FROM \"post\" WHERE (\"post\".\"user_id\") IN (($1), ($2), ($3)) ORDER BY \"post\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=2.169084ms elapsed_secs=0.002169084
INFO sqlx::query: summary="SELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" …" db.statement="\n\nSELECT \"comment\".\"id\", \"comment\".\"post_id\", \"comment\".\"body\" FROM \"comment\" WHERE (\"comment\".\"post_id\") IN (($1), ($2), ($3)) ORDER BY \"comment\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=912.542µs elapsed_secs=0.000912542
INFO sqlx::query: summary="SELECT \"tag\".\"id\", \"tag\".\"name\", \"post_tag\".\"post_id\" …" db.statement="\n\nSELECT \"tag\".\"id\", \"tag\".\"name\", \"post_tag\".\"post_id\" FROM \"tag\" INNER JOIN \"post_tag\" ON \"post_tag\".\"tag_id\" = \"tag\".\"id\" WHERE (\"post_tag\".\"post_id\") IN (($1), ($2), ($3)) ORDER BY \"tag\".\"id\" ASC\n" rows_affected=4 rows_returned=4 elapsed=1.75325ms elapsed_secs=0.00175325
ann / a1: comments=["c1", "c2"] tags=["rust", "sql"]
ann / a2: comments=[] tags=["rust"]
bob / b1: comments=["c3"] tags=["orm"]
--- D. with((profile, user)) [nested on a 1-1]
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\" FROM \"users\"\n" rows_affected=3 rows_returned=3 elapsed=294.375µs elapsed_secs=0.000294375
INFO sqlx::query: summary="SELECT \"profile\".\"id\", \"profile\".\"picture\", \"profile\".\"user_id\" …" db.statement="\n\nSELECT \"profile\".\"id\", \"profile\".\"picture\", \"profile\".\"user_id\" FROM \"profile\" WHERE (\"profile\".\"user_id\") IN (($1), ($2), ($3)) ORDER BY \"profile\".\"id\" ASC\n" rows_affected=2 rows_returned=2 elapsed=871.875µs elapsed_secs=0.000871875
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\" WHERE (\"users\".\"id\") IN (($1), ($2)) ORDER BY \"users\".\"id\" ASC\n" rows_affected=2 rows_returned=2 elapsed=700.917µs elapsed_secs=0.000700917
ann: profile.user loaded=true
--- E. post::load().with(user).filter(..).one()
INFO sqlx::query: summary="SELECT \"post\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"post\".\"id\" AS \"A_id\", \"post\".\"user_id\" AS \"A_user_id\", \"post\".\"title\" AS \"A_title\", \"users\".\"id\" AS \"B_id\", \"users\".\"name\" AS \"B_name\" FROM \"post\" LEFT JOIN \"users\" ON \"post\".\"user_id\" = \"users\".\"id\" WHERE \"post\".\"title\" = $1 LIMIT $2\n" rows_affected=1 rows_returned=1 elapsed=876.5µs elapsed_secs=0.0008765
Some(("b1", Some("bob")))
| 调用 | 语句数 | join 的 | IN 的 |
|---|---|---|---|
A load() | 1 | — | — |
B with(profile) | 1 | profile | — |
C with(profile) + (post, comment) + (post, tag) | 4 | profile | post、comment、tag(经 post_tag) |
D with((profile, user)) | 3 | — | profile、user |
E post::load().with(user).one() | 1 | users | — |
几点:
- C 里
(post, comment)和(post, tag)两次with都落在posts这一位上,post只查了一次,nest.posts里亮了comments和tags两位。语句数 = 1 + 每一层亮着的 Loader 位数,与行数无关。 - D 只是给一对一加了嵌套,
profile就从 join 退回到 Loader,语句从 1 条变成 3 条。 cat没有文章,C 的第二条语句仍然带着 cat 的 id(IN里三个参数)。父记录不知道自己有没有子记录,Loader 只能都问一遍。- E 的
one是EntityLoader::one生成的self.select.limit(1)再all。LIMIT加在主查询上,join 的只是一对一,限制的就是父记录数。 - 主查询里没有
ORDER BY:load()是Entity::find(),不像LoaderTrait那样自动按主键排序。要稳定的顺序得自己order_by。
ModelEx 里的 Unloaded 和 Loaded
with 没点到的关系字段是 Unloaded,不是空。HasMany<E> 是:
#[derive(Debug, Default, Clone)]
pub enum HasMany<E: EntityTrait> {
#[default]
Unloaded,
Loaded(Vec<E::ModelEx>),
}
HasOne 是 Unloaded | Loaded(Option<Box<E::ModelEx>>),BelongsTo 见关系。Loader 装回时用 posts.into() 把 Vec 变成 Loaded(vec),所以加载过但没有子记录的是 Loaded([]),和 Unloaded 能区分开。
问: A 里 posts.is_empty() 是 true、len() 是 0,这不就说明 ann 没写过文章吗?
答: 不能这么读。HasMany::is_empty 的注释写着「Return true if variant is Unloaded or underlying vector is empty」,len() 对 Unloaded 也返回 0。这两个方法是为了让 HasMany 用起来像 Vec,代价是「没加载」和「没有」在它们眼里一样。要区分,先看 is_loaded() / is_unloaded(),或者直接 match。A 的输出里 ann 的 posts unloaded=true,而实际上她有两篇文章。
分页加载
EntityLoader 实现了 EntityLoaderTrait,有 paginate。上面 fetch 的 page_size != 0 分支把 LIMIT / OFFSET 加在主查询上,Loader 再只为这一页的父记录取子记录(src/bin/el_dup.rs 的最后一段):
--- load().with(post).paginate(2).fetch_page(0)
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\" FROM \"users\" LIMIT $1 OFFSET $2\n" rows_affected=2 rows_returned=2 elapsed=311.083µs elapsed_secs=0.000311083
INFO sqlx::query: summary="SELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" …" db.statement="\n\nSELECT \"post\".\"id\", \"post\".\"user_id\", \"post\".\"title\" FROM \"post\" WHERE (\"post\".\"user_id\") IN (($1), ($2)) ORDER BY \"post\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=436.625µs elapsed_secs=0.000436625
1 ann: 2 posts
2 bob: 1 posts
这正是 find_with_related(..).limit(n) 做不到的:后者在 join 之后的行上 LIMIT,父记录会被截断(见 join 与 find_related);这里 LIMIT 只数父记录,子记录一条不少。同样要注意主查询没有 ORDER BY,LIMIT / OFFSET 的页与页之间没有保证,见分页与游标。
profile.user_id 原本是 UNIQUE。去掉这个约束,再给 ann(id 1)插第二条 profile ann-2.png,然后:
let users = user::Entity::load().with(profile::Entity).all(&db).await?;
let users2 = user::Entity::find().all(&db).await?;
let profiles = users2.load_one(profile::Entity, &db).await?;
users 有几个元素?load_one 给 ann 的是哪一张图?平台:M2 Max、macOS、rustc 1.99.0、Postgres 16.14。答案在页末。
动脑筋答案
src/bin/el_dup.rs(DDL 之后执行 ALTER TABLE profile DROP CONSTRAINT profile_user_id_key; INSERT INTO profile (picture, user_id) VALUES ('ann-2.png', 1);):
--- load().with(profile) [join]
INFO sqlx::query: summary="SELECT \"users\".\"id\" AS \"A_id\", …" db.statement="\n\nSELECT \"users\".\"id\" AS \"A_id\", \"users\".\"name\" AS \"A_name\", \"profile\".\"id\" AS \"B_id\", \"profile\".\"picture\" AS \"B_picture\", \"profile\".\"user_id\" AS \"B_user_id\" FROM \"users\" LEFT JOIN \"profile\" ON \"users\".\"id\" = \"profile\".\"user_id\"\n" rows_affected=4 rows_returned=4 elapsed=1.274125ms elapsed_secs=0.001274125
1 ann: Some("ann.png")
2 bob: Some("bob.png")
1 ann: Some("ann-2.png")
3 cat: None
--- load_one(profile) [loader]
INFO sqlx::query: summary="SELECT \"users\".\"id\", \"users\".\"name\" FROM …" db.statement="\n\nSELECT \"users\".\"id\", \"users\".\"name\" FROM \"users\"\n" rows_affected=3 rows_returned=3 elapsed=348.416µs elapsed_secs=0.000348416
INFO sqlx::query: summary="SELECT \"profile\".\"id\", \"profile\".\"picture\", \"profile\".\"user_id\" …" db.statement="\n\nSELECT \"profile\".\"id\", \"profile\".\"picture\", \"profile\".\"user_id\" FROM \"profile\" WHERE (\"profile\".\"user_id\") IN (($1), ($2), ($3)) ORDER BY \"profile\".\"id\" ASC\n" rows_affected=3 rows_returned=3 elapsed=644.875µs elapsed_secs=0.000644875
1 ann: Some("ann-2.png")
2 bob: Some("bob.png")
3 cat: None
users有 4 个元素,ann 出现两次。Entity Loader 把一对一 LEFT JOIN 进主查询,库里有两条 profile 就有两行,每行变成一个ModelEx,不做任何合并。行顺序也不保证(主查询没有ORDER BY),这里两个 ann 中间夹着 bob。load_one给 ann 的是ann-2.png。Loader 每个用户只占一个槽,Container for Option<T>的add是self.replace(item):同一个键来了两行,后一行覆盖前一行。ORDER BY "profile"."id" ASC让 id 3 的ann-2.png排在后面,于是它赢了。
两种方式都不报错。一对一靠的是数据库里的 UNIQUE 约束,sea-orm 不检查;约束缺了,join 的路径多出父记录,Loader 的路径静默丢掉子记录。
小结
- N+1 是逐个父记录发一条查询;
LoaderTrait把整批父记录的键放进一条WHERE (col) IN ((..), ..),再按键在内存里分组,返回值与输入一一对应。 load_one要求HasOne(运行时检查),load_many覆盖一对多和带via的多对多(一条 join 中间表的语句),load_many_to_many是两条语句的旧写法。- 键在拼条件时
unique()去重(1.1.11 起),空输入直接返回空Vec、不发语句;直接传实体会加ORDER BY主键(2.0 起),传Select则沿用你的条件、排序和LIMIT,LIMIT作用于整条语句而不是每组。 - 不分批:去重后的键超过 65535 个,sqlx 在发送前报
too many arguments for query,要自己切片。 - Entity Loader 的
load().with(..)把前两个没有嵌套的一对一 LEFT JOIN 进主查询,其余一对一和所有一对多走 Loader;语句数是 1 加每层亮着的 Loader 位,paginate只对父记录分页。 HasMany::is_empty()/len()把Unloaded当成空;一对一缺少UNIQUE约束时,join 会重复父记录,Loader 会让后来的子记录覆盖前面的。