一张A4纸,拆解任何一堆用户query
面对几千条乱成一团的用户query,我最后靠一张A4纸和最朴素的5W1H理出了范式。这套框架后来成了评测集的骨架,也是我分析任何一堆用户需求的起点。
距离ddl还有3个小时,我面前是几千条用户query,脑袋像一团乱麻。
那是三年前的2023年,我在百度做AI地图(完整的经历写在这一篇里)。项目刚开始,老板给我的第一个任务是:给研发总结出用户query的范式。
我试了一整天,换了好几种分类方式,都觉得不够有代表性。最后我跑到公司楼下的花园,从打印机拿了一张A4纸和一支笔,把脑海里乱蹦的想法全写下来,一条一条找它们之间的关联。
在最后一个小时,我突然发现,是我之前想得太复杂了。
那张纸上最后留下的,是一个很朴素的框架。后来不管是搭评测集,还是上线后query蜂拥而至,我都是从它出发。这篇就把它完整地拆开讲一遍,也算是给自己在产品路上的经历做一次总结归纳。
01为什么要给query找范式
“范式”听起来很虚,说白了就是:把像一团散沙的用户query和session(行为路径),总结成一种或几种pattern。
它至少有三个用处:
- 对研发:清楚地知道核心优化哪些场景,每个场景下用户怎么问、问题里带了哪些信息
- 对产品:这是组织一个合理评测集的重要依据,评测集覆盖什么、不覆盖什么,都从这里来
- 对之后:上线后真实query大量涌进来,有一个框架接着,才不会乱了阵脚
在找范式之前,只能先用笨办法。先是从一周的搜索日志里抽了5000条query,全组分工,一条一条地看、判断、分类;后来AI对话开始内测,又把三千多条对话query、近两千个session逐条标了一遍。《从需求到产品》讲需求采集时强调过一件事:记录用户的原话和场景,而不是记录用户给你的解决方案。几千条原话看下来,你对用户是怎么说话的会有一种体感,这是后面所有抽象的地基。
但看完原话,最最最容易掉进的坑是:按业务逻辑去分类。
这也是之后的每一次产品设计里,我都会反复提醒自己的一点:分析用户需求的时候,不要带入太多脑海里已有的业务逻辑。业务逻辑虽然是产品一路发展演变出来的,但它带着很强的组织管理倾向,也有滞后性。比如按地图的功能模块分——搜索/导航/路线,或者按品类分——餐饮/景点/酒店。业务逻辑是“我们怎么组织产品”,而用户提问的时候,并不知道、也不关心我们怎么组织产品。
如果按当时的业务逻辑来分,分着分着就会发现,这些分类都不够有代表性。一个用户可能在同一条query里,问去某个品类里某一种地方、要走某种特定的路线、还顺手发起导航——一条query就横跨了所有的业务分类。按业务逻辑找范式,一定是一种有迷惑性的方式。
02第一层:用户在问什么
于是我退回到第一性原理:抛开地图,一个人去问一个问题,一共就这几种询问方式——【who】【what】【when】【where】【why】【how】。
这就是5W1H,入行时在产品书里学过的最朴素的工具。它朴素,但有一个好处:它是从“人怎么提问”出发的,而不是从“我们的产品长什么样”出发的。
用法很简单:先把六种全列出来,再删掉在你的场景里不存在的。
放到地图里:
- who:基本不存在,很少有人问地图“谁”
- what / why:问一个地点的详情,比如“故宫几点开门?儿童需要买门票吗?”
- when:问时间,比如什么时候能到(ETA)
- where:找地点
- how:问怎么去
六种问法,收敛成了三个核心场景:
- where → 泛化找地点
- what / why → 地点详情问询
- when / how → 复杂的出行需求(导航、ETA、交通方式等)
这三个场景,后来就是整个AI地图核心优化的方向。
回头再对照当时那三千多条对话query的标注,分类几乎是一一对应的:找地点的泛需求、单个地点的内容问询、出行和行程规划,三类加起来占了九成。其中泛化找地点最多,接近一半。
03第二层:问题里带了什么
知道用户在问什么还不够,还要知道每一种问法里,用户带了哪些信息。
同样从第一性原理推导:一个问题里的信息,基本就是【时间】【地点】【人物】,在此基础上,再加上对每个元素的形容——【什么样的】。
还是先列全再删:在地图场景里,【人物】是不重要的。剩下的就是【时间】【地点】和【什么样的】。
这三样里,最关键的是【什么样的】。
“导航到西湖”,没有形容词,地图很擅长。“导航到杭州有名的、适合亲子游玩的景点”,多出来的全是形容词。地图原本的标签matching和地点打分接不住它,因为地点本身没有被打上“适合亲子”这样的内容标签。长尾需求,几乎全藏在【什么样的】里。
数据也印证了这一点。在搜索框里,八成多的query是“导航到西湖”这样的精确需求,泛需求只占一成左右;可到了对话里,比例几乎倒过来,泛需求接近一半。用户一旦发现可以“说人话”,形容词就全冒出来了。
所以后面很多功夫都花在【什么样的】上。比如把“娃、亲子、小朋友、小孩子、宝宝们”这些说法聚成同一个主题词“遛娃”,再把它变成地点身上的标签“适合带娃”。有了标签,地图才接得住这类问题,甚至能顺着往下问一句:附近有没有适合宝宝的餐厅?
《俞军产品方法论》里有一个观点:用户不是自然人,而是需求的集合,而需求总是依赖于具体的场景。回头看,这一层拆的其实就是场景:什么时候、在哪里、要什么样的。
04两层交叉,得到一张query地图
把两层放在一起:横轴是问法,纵轴是信息。一条query,就是在某一列问法下面,把几行信息拼起来。

每一条query都能这样拆开。一种问法加上几样信息,就是一类query;所有的类合在一起,就是最终的用户query框架。拆开之后,它要什么、缺什么就很清楚了:比如第一列,难点不在“北京”“周末”,而在“适合亲子、有历史”——那正是需要AI去理解、去给地点生成内容标签的部分。
也有不那么听话的query:
“帮我找从故宫去天坛顺路的餐厅”
询问方式:where(找餐厅)+ how(从故宫到天坛的路线)
组成部分:【地点:故宫、天坛】【什么样的:顺路】
它横跨了两个场景。这也意味着,地图原本的路线、导航、ETA数据,也会变成被搜索的一种内容标签:“顺路”本身,就是餐厅身上的一个属性。
当时我们的处理是:把“判断顺路度”“沿途搜”这类query,归进出行和行程规划,而不是找地点。原因是用户真正要完成的是这条路线,餐厅只是路线上的一个点。横跨两类的时候,按用户最终要完成的那个动作归类。
还有一种不听话,是多轮对话。近两千个session里,有三分之一不止一轮。用户常常先问一句很宽的:
“北京好玩的地方”
下一轮再补:“哪些地方拍出来的游客照最好看”
第一轮只有【地点:北京】,第二轮才补上【什么样的:拍照好看】。所以框架不能只看单条query,要看整个session:信息是一轮一轮拼起来的。
05从框架到评测集
这套框架真正发挥作用,是在设计评测集的时候。评测集的底子,就是前面那两份人工标注:搜索的5000条,和对话的三千多条。
没有框架的评测集,很容易变成“想到什么测什么”:有的场景测了一百条,有的场景一条都没覆盖。有了这套框架,评测集就有了骨架:
- 按类抽样:每一类按真实流量里的比例去取query,让评测集长得像真实用户
- 按类定标准:不同的类,好答案的标准不一样。找地点,先看找得准不准;问详情,最不能容忍的是幻觉
- 按类看问题:效果变差的时候,先看是哪几类在掉分,而不是盯着一个总分发愁
打分用的是GSB:把新旧两个模型对同一条query的回答放在一起比,判Good、Same还是Bad,最后看Good的比例。之所以只能自己跟自己比,是因为那时市面上还没有可以对标的竞品,唯一的参照就是上一版的自己。判的时候看三个维度:意图有没有理解对,信息是否准确,回答是否简洁、说在点上。前两条相对客观,第三条就很主观了——同一条回答,有人觉得刚好,有人觉得啰嗦。
它还给了我们一个额外的信号器。上线以后,新的query进来,先按框架归类:某一类突然变多,说明用户的心智在往这个场景走;出现了归不进任何一类的query,往往就是一个新需求。
评测集本身也会过期。当时我们遇到过fine-tune导致的过拟合:用户换一些提问方式,或者问同类型的新问题,模型就答不好了。这时就要重新审视,测试集和一个月前相比是否还成立。框架在这里的作用是:你知道该往哪几类补新的样本。
三年后再看行业里的做法,评测这件事的骨架几乎没变:一份测试集、分维度的标准、两两对比。GSB现在多叫胜率(win rate),变的是打分的人——大部分换成了模型当评委,人退到两个位置:写评分细则,和抽查模型评委判得准不准。
当年最让我们头疼的“简洁、说在点上”,现在的做法是拆成几个能回答“是”或“否”的小问题:第一句话有没有直接回答?有没有塞用户没问的信息?拆得越细,人和模型的判断就越容易对齐。测试集也不再是一次性的,线上的新query和badcase会持续补进来,每次更新模型都要再跑一遍。
但有一件事没变:什么叫“好答案”,还是得人先想清楚。模型可以替你打分,但不能替你决定该按什么打。
06换一个产品,框架怎么变
这张A4纸上的框架,不只属于地图。
方法不变:先把5W1H和【时间】【地点】【人物】【什么样的】全列出来,再删掉在你的场景里不存在的。变的是删掉了什么。
在地图里,who和【人物】都被删掉了。但换一个产品,被删掉的可能恰恰是最重要的。比如后来我在一家硅谷公司做Enterprise AI产品,【who】就变成了不可或缺的一部分。(做Enterprise AI产品的经历,之后也会单独写一写。)
所以与其说这是一个地图的框架,不如说它是一个拆问题的顺序:先问人怎么问,再问问题里有什么,最后才轮到我们的业务逻辑。
尾声有了大模型,还需要这张A4纸吗
现在再遇到几千条query,我会直接扔给大模型。几秒钟,它就能分好类、找出规律,比我当时一整天的效率高太多了。
那这张A4纸还有意义吗?
我觉得有。模型很擅长“分”,但“按什么分”仍然是人的判断。决定在地图里删掉who,决定【什么样的】才是长尾的关键——这些不是从数据里自动长出来的,而是对“什么对这个产品重要”的判断。分类本身是体力活,分类的维度才是产品经理的观点。
我在上一篇的结尾写过,现在着急快速iterate,有时反而心里没那么笃定踏实。回头看那个下午,笃定恰恰来自亲手推导过一遍:每一类为什么存在,每一个被删掉的维度为什么被删掉,我都说得出来。
离开百度之后,我没有再做模型优化相关的产品,而是把模型能力用到不同的领域里。交付的方式也变了:不再是一问一答,而是以agentic的方式,帮用户把一件具体的工作做完。
但这张A4纸一直跟着我。用户不再只是问一个问题,而是交给你一件事。拆的顺序还是一样:这件事是谁的、要交付什么、什么时候要、在哪里做、怎么做,然后看里面带了哪些信息,最后才轮到我们的产品逻辑。问题变成了任务,5W1H依然是拆开它的第一刀。
本文内容均为个人回忆与观点,不代表任何公司立场
产品笔记 · 系列(一)
One Sheet of A4: How I Break Down Any Pile of User Queries
Faced with thousands of messy user queries, I ended up making sense of them with a sheet of A4 paper and the plainest tool there is: 5W1H. That framework became the backbone of our evaluation set, and it's still where I start whenever I need to understand what users are asking for.
Three hours before the deadline, I was staring at thousands of user queries, and my head was in knots.
It was 2023, three years ago, and I was building an AI map at Baidu (the full story is in this piece). The project had just kicked off, and my manager's first assignment for me was to distill our users' queries into a handful of patterns the engineers could build against.
I'd spent the whole day trying one way of grouping them after another, and none of them felt right. Finally I went down to the garden outside the office with a sheet of A4 paper from the printer and a pen, wrote down every idea bouncing around in my head, and started looking for the connections between them.
In the last hour, it suddenly hit me: I had been overcomplicating it.
What was left on that page was a very plain framework. Later, whether I was building an evaluation set or dealing with the flood of queries after launch, I always started from it. This piece takes it apart step by step. It's also my way of looking back on what I've learned as a product manager.
01Why look for a pattern in queries at all
"Pattern" sounds abstract. Put simply, it means taking a shapeless heap of user queries and sessions (the sequence of things a user does) and boiling it down to a few recurring shapes.
It's useful in at least three ways:
- For engineers: they know exactly which scenarios to optimize, how users phrase their questions in each one, and what information those questions carry
- For product: it's the basis for a sensible evaluation set; what the set covers, and what it leaves out, comes from here
- For later: when real queries pour in after launch, you have a framework to catch them instead of getting swamped
Before you can find a pattern, you have to do it the slow way. First we pulled 5,000 queries from a week of search logs, split them across the whole team, and read, judged and labeled them one by one. Later, when the AI chat entered internal testing, we did the same for over three thousand chat queries across nearly two thousand sessions. One idea from From Requirements to Product, a Chinese PM handbook by Quan Li, stuck with me: record what users actually said and the situation they were in, not the solution they suggested. After reading thousands of real queries, you develop a feel for how users talk. That feel is the foundation for every abstraction that comes after.
But once you've read them, the easiest trap to fall into is grouping them by business logic.
It's something I've reminded myself of in every product I've designed since: when you analyze user needs, don't bring in too much of the business logic already in your head. Business logic grew out of how the product evolved, but it mirrors how the organization is structured, and it always lags behind what users actually want. Think of splitting by feature (search, navigation, routes) or by category (restaurants, attractions, hotels). Business logic is "how we organize the product," and when users ask a question, they neither know nor care how we organize the product.
Sort by that logic and you soon find that none of the categories really hold. A single query might ask for a particular kind of place in a particular category, via a particular route, and then start navigation on top of that. One query cuts across every business category. As a lens for finding patterns, business logic is seductive, and misleading.
02Layer one: what is the user asking?
So I went back to first principles. Forget maps for a moment: when a person asks a question, there are only a handful of ways to ask it: who, what, when, where, why, how.
That's 5W1H, the plainest tool I picked up from product books when I started out. It's plain, but it has one big advantage: it starts from "how people ask questions," not from "what our product looks like."
Using it is simple. List all six first, then cross out the ones that don't exist in your scenario.
Applied to maps:
- who: barely exists; almost nobody asks a map "who"
- what / why: asking about the details of a place, like "What time does the Forbidden City open? Do kids need a ticket?"
- when: asking about time, like when you'll arrive (the ETA)
- where: finding a place
- how: asking how to get there
Six ways of asking collapsed into three core scenarios:
- where → open-ended place finding
- what / why → questions about a specific place
- when / how → complex trip needs (navigation, ETA, modes of transport, and so on)
These three scenarios became the core optimization directions for the whole AI map.
Looking back at how those three thousand-plus chat queries were labeled, the categories line up almost one to one: open-ended place finding, questions about a single place, and trip and itinerary planning together covered about 90% of queries. Open-ended place finding was the largest, close to half.
03Layer two: what does the question contain?
Knowing what the user is asking isn't enough. You also need to know what information they put into each kind of question.
Again from first principles: the information in a question is basically time, place, and people, plus a description attached to any of them, which I call "what kind."
Same rule: list everything first, then cross out. In a map, "people" doesn't matter much. That leaves time, place, and "what kind."
Of the three, "what kind" is the one that matters most.
"Navigate to West Lake" has no adjectives, and maps are great at it. "Navigate to a well-known attraction in Hangzhou that's good for families with kids" is almost all adjectives. The map's existing tag matching and place scoring can't handle it, because places don't carry content tags like "kid-friendly." The long tail of user needs hides almost entirely in "what kind."
The data backed this up. In the search box, over 80% of queries were precise requests like "navigate to West Lake," and open-ended ones were only about 10%. In chat, the ratio nearly flipped: open-ended requests were close to half. Once users realize they can talk to it like a person, the adjectives come pouring out.
That's why so much of our later work went into "what kind." For example, we merged different phrasings (kids, children, little ones, toddlers, family) into one theme, "outings with kids," and turned it into a tag on places: "good for kids." With that tag, the map could handle these questions, and even offer a natural follow-up: is there a kid-friendly restaurant nearby?
Yu Jun's Product Methodology, by Baidu's former head of product, makes the point that a user isn't a person but a bundle of needs, and needs always depend on a specific situation. Looking back, this layer is really about breaking down the situation: when, where, and what kind.
04Cross the two layers and you get a query map
Put the two layers together: the horizontal axis is how users ask, the vertical axis is what information they include. A query is one column of asking, with a few rows of information filled in.

Every query can be broken down this way. One way of asking plus a few pieces of information makes one kind of query; all the kinds together make up the final query framework. Once a query is broken down, it's clear what it wants and what's missing. In the first column, the hard part isn't "Beijing" or "weekend"; it's "good for kids, with history." That's exactly the part that needs AI to understand it and to generate content tags for places.
And then there are the queries that don't play by the rules:
"Find me a restaurant on the way from the Forbidden City to the Temple of Heaven"
How it's asked: where (find a restaurant) + how (the route from the Forbidden City to the Temple of Heaven)
What it contains: place: Forbidden City, Temple of Heaven; what kind: on the way
It spans two scenarios. It also means the map's own route, navigation and ETA data become a kind of searchable content tag: "on the way" is itself an attribute of the restaurant.
At the time, we filed queries like "is it on the way" and "search along the route" under trips and itinerary planning, not place finding. The reason is that what the user really wants to complete is the route; the restaurant is just a point on it. When a query spans two kinds, file it under the action the user ultimately wants to complete.
Multi-turn conversations break the mold in a different way. Among nearly two thousand sessions, a third had more than one turn. Users often start broad:
"Fun places in Beijing"
and add in the next turn: "Which spots make for the best tourist photos?"
The first turn only has "place: Beijing"; the second adds "what kind: great for photos." So the framework can't look at single queries alone. It has to look at the whole session, because the information is pieced together turn by turn.
05From framework to evaluation set
The framework really proved its worth when we designed the evaluation set. The foundation of that set was the two rounds of manual labeling above: 5,000 search queries and more than three thousand chat queries.
Without a framework, an evaluation set easily turns into "test whatever comes to mind": a hundred cases for one scenario and none for another. With the framework, the set has a backbone:
- Sample by kind: draw queries for each kind in proportion to real traffic, so the set looks like real users
- Set standards by kind: a good answer means different things for different kinds. For finding places, accuracy comes first; for questions about a place, hallucination is the one thing you can't tolerate
- Diagnose by kind: when quality drops, look at which kinds are losing points first, instead of agonizing over one overall score
For scoring we used GSB: put the old and new models' answers to the same query side by side, judge each as Good, Same or Bad, and track the Good rate. We could only compare ourselves with ourselves, because back then there were no comparable products on the market; the only reference point was our own previous version. Raters judged three things: did it understand the intent, was the information accurate, and was the answer concise and to the point. The first two were fairly objective. The third was very subjective: the same answer felt just right to one person and long-winded to another.
The framework also gave us an extra signal. After launch, every new query was first sorted into a kind. If one kind suddenly grew, users' expectations were shifting toward that scenario. If a query didn't fit any kind, it was usually a new need.
Evaluation sets also go stale. We ran into overfitting from fine-tuning: when users phrased things differently, or asked new questions of the same type, the model's answers got worse. That's when you have to ask whether the test set still holds compared with a month ago. The framework tells you which kinds need fresh samples.
Three years on, looking at how the industry approaches this, the skeleton of evaluation has barely changed: a test set, standards broken into dimensions, and side-by-side comparison. GSB is now usually called win rate. What changed is who does the scoring. It's mostly models acting as judges now, and people have moved into two roles: writing the grading rubric, and spot-checking whether the model judges are getting it right.
"Concise and to the point," the criterion that gave us the most headaches, is now usually broken into a few yes-or-no questions: does the first sentence answer the question directly? Does it include anything the user didn't ask for? The finer you break it down, the easier it is for people and models to agree. Test sets aren't one-off anymore either; new queries and bad cases from production keep flowing in, and every model update runs through them again.
But one thing hasn't changed: someone still has to decide what a "good answer" is. A model can score for you, but it can't decide what to score on.
06Change the product, change the framework
The framework on that sheet of A4 doesn't belong only to maps.
The method stays the same: list 5W1H and time, place, people and "what kind" in full, then cross out what doesn't exist in your scenario. What changes is what you cross out.
In maps, "who" and "people" were both crossed out. In a different product, what you crossed out might be exactly what matters most. When I later worked on enterprise AI products at a Silicon Valley company, "who" became indispensable. (I'll write about building enterprise AI products separately.)
So rather than a framework for maps, it's really an order for breaking down a question: first ask how people ask, then ask what's in the question, and only then bring in our business logic.
EpilogueWith large models, do I still need the A4 sheet?
If I faced thousands of queries today, I'd hand them straight to a large model. In a few seconds it would sort them and find the patterns, far faster than my whole day back then.
So does the sheet of A4 still matter?
I think it does. Models are very good at sorting, but deciding what to sort by is still a human judgment. Crossing out "who" for maps, deciding that "what kind" is where the long tail lives: none of that grows out of the data on its own. It's a judgment about what matters for this product. Sorting is manual labor; the dimensions you sort by are the product manager's point of view.
At the end of the previous piece, I wrote that in my rush to iterate quickly, I sometimes feel less sure of my footing. Looking back at that afternoon, the certainty came from having worked it out by hand: I could explain why every kind existed and why every crossed-out dimension was crossed out.
After Baidu, I stopped working on model-optimization products and started applying model capabilities to different domains. The way we deliver changed too: not question and answer anymore, but an agentic approach that helps users get a specific piece of work done.
But that sheet of A4 never really left me. Users no longer just ask a question; they hand you a task. The order for breaking it down is the same: whose task is it, what needs to be delivered, when is it due, where does the work happen, and how should it be done. Then look at what information it carries, and only then bring in our product logic. The question has become a task, and 5W1H is still where I make the first cut.
All content in this article reflects personal memories and opinions, and does not represent the position of any company.
Product Notes · Part I