A Plain-English Guide to Picking an AI Product

You’ve probably heard two completely opposite claims about AI. One says it can’t be trusted — it’ll state something with total confidence and describe data that doesn’t exist as if it were staring right at it. The other is your own lived experience: yesterday you asked it for an industry number and it came back with three sourced links; the day before, you asked about something it wasn’t sure of and it told you plainly, “I’m not confident about this — you should double-check.”

Both claims are true, and the gap between them hides something important: the thing you deal with every day isn’t the model. It’s the product.

This piece is about making that gap — everything “beyond the model” — legible, so that by the end you’ll have a working yardstick: faced with a screen full of AI products, you’ll know how to tell which one actually fits you. One boundary up front: this piece covers chat products, from ChatGPT’s launch in late 2022 through the apps on your phone today that can converse, search, and write. The products that can go do things on their own — agents — are the subject of the next piece, and section four explains exactly why the line falls where it does.

1. The Engine and the Car

Let’s separate two things first, because nothing after this makes sense until we do.

The model is the engine. Whatever sits behind ChatGPT, Claude, Gemini, Doubao, Qwen, or DeepSeek is, at its core, a text-prediction engine: given what came before, it calculates the most likely next stretch of text and keeps going. The product is the whole car — the app you actually open, which is the model plus a whole apparatus wrapped around it: system instructions, retrieval tools, memory, safety policy, an interface, and a pile of judgment calls you never see.

The engine sets the ceiling on power, but whether a car is actually pleasant to drive — quiet, responsive brakes, doesn’t skid in the rain — comes down mostly to everything outside the engine. And the engine itself, the model, carries four native weaknesses. These aren’t bugs in any one model — they’re just what a “text-prediction engine” inherently is.

First, it doesn’t know you, and it doesn’t know “now.” It’s never seen your name, your company, or the preference you mentioned last week; its training stopped at some fixed point, called the knowledge cutoff, and anything after that is a blank to it.

Second, it has no memory. You tell it your name, it acknowledges it, and a moment later that sentence no longer exists anywhere in its “mind.” The reason a conversation feels continuous is that something, behind the scenes, quietly hands the whole transcript back to it fresh at every turn. This property is called stateless — what feels to you like an ongoing conversation is, from the model’s side, a series of separate, brand-new reading-comprehension exercises. And there’s a limit to how much text it can look at in any one pass — that limit is the context window, best pictured as a desk of fixed size.

Third, it’s exactly as confident when it’s wrong as when it’s right. Its job is to produce a plausible next stretch of text, not to check whether that text is true — so it can produce a number that’s beautifully formatted, sounds airtight, and simply doesn’t exist. This is usually called hallucination, though I prefer the term the U.S. National Institute of Standards and Technology uses: confabulation. It isn’t an occasional malfunction — it’s what happens when the mechanism fills a gap it doesn’t have an answer for with a story that sounds plausible. That’s not a glitch in the system; it’s the system working as designed.

Fourth, it has no hands. It can draft a beautiful email but can’t send it, calculate a number but can’t update your spreadsheet. Everything it produces is, in the end, just text.

Since the day ChatGPT launched, what the entire industry has been doing is building outward from those four weaknesses, one layer at a time.

2. From ChatGPT to Now: A History of Patching the Gaps

This history is worth walking through in full, because it isn’t a string of product launches — it’s a clean logical line, where each step patches one of the four weaknesses above, just further and further from the core.

Late 2022 to 2023: patching “it doesn’t know ‘now.’” ChatGPT launched in November 2022 and crossed 100 million users within two months. What set it off wasn’t a new model — the underlying capability had existed for a while — it was that the product layer got one small thing right: it turned into a chat box. Before that, the same capability sat behind a developer API that ordinary people couldn’t touch; the form a technology takes determines whether it actually gets used, and that idea is the whole subject of this piece. But back then it was well-read and stuck in the past — ask it today’s exchange rate and it either said it didn’t know or made one up. So starting in February 2023, web search entered the product; in March, ChatGPT shipped plugins (later phased out, but the direction stuck), and GPT-4 launched the same month, adding vision and a noticeable jump in reasoning. The keyword is “plugging into the outside world” — AI went from a well-read student to one who could actually go look things up.

Late 2023 to 2024: patching “it can’t see much, and it can’t remember.” The next weakness surfaced fast: the context window was too small — it couldn’t read a thirty-page contract in one pass. So in November 2023 context windows jumped to the low hundreds of thousands of tokens; by February 2024 Gemini pushed into the millions, enough to read a whole book in one go; in China, Kimi broke into the market on the strength of ultra-long context. In the same window, GPT-4o made real-time voice and vision a routine feature, memory started rolling out broadly, and answers began carrying citation links — that last one looks small but was actually the dividing line for trustworthiness. The keyword is “the desk got bigger, and it started taking notes.”

Late 2024 to 2025: patching “it thinks too shallowly.” It answered too fast, running straight ahead without a pause, and anything requiring multi-step reasoning would veer off course — like someone writing an answer without ever drafting it first. The turning point was reasoning models: in September 2024, the o1 series turned “think it through before answering” into an actual product capability for the first time; in January 2025, DeepSeek released R1 and open-sourced the weights, proving this kind of “thinking” wasn’t something only a handful of companies with deep pockets could build — and from that point on, access to frontier capability changed completely. That same year, “deep research” features appeared: give it a topic, and it goes and reads dozens of web pages on its own and hands back a sourced report. The keyword is “slow down and think one more step” — the industry’s center of gravity shifted from “make the model bigger” to “let the model spend more compute at answer time.”

Late 2025 into 2026: patching “it has no hands.” AI started being able to look at a screen, click buttons, fill out forms. A protocol called MCP got widely adopted and handed off to a neutral foundation to steward — effectively giving AI a standard “USB port.” By 2026, agents exploded across the board — from the installation rush kicked off by open-source projects, to desktop office agents from the major labs, to products that live in the cloud around the clock and keep working after you close your laptop. The keyword is “from talking to doing.”

Four years in, it’s a remarkably clean line: the product layer was never trying to make the model smarter. It was scaffolding around each weakness, one at a time, from the outside. And the chat products this piece covers stop right at the end of the third stage.

3. The Six Patches Every Chat Product Wears Today

Bringing that history up to the present: mainstream chat products today carry roughly six patches. One caveat up front: none of these patches are free, so each one below comes with its cost attached.

Search before answering. The system searches, or reads the files you uploaded, and hands the model that material to answer from and cite. The industry calls this retrieval-augmented generation (RAG) — plainly put, it turns a closed-book exam into an open-book one: instead of answering from memory, it looks things up first, and has to tell you which page it looked at. This is the single most effective patch, and the more careful implementations add one more rule on top: if a sentence isn’t backed by the retrieved material, don’t say it. The cost is speed and money, and worse, if retrieval pulls the wrong thing, the resulting answer comes with more, not less, confidence — because now it has a “source.”

Remember you. The product stores your preferences, project context, and past corrections somewhere else, and pulls them back into the conversation when needed — which is why it feels like “it’s gotten to know me.” It hasn’t gotten smarter; someone else took notes on your behalf. The cost is that memory can go stale or simply be wrong, and a bad piece of long-term memory quietly poisons every conversation after it without you noticing — harder to catch than a hallucination, precisely because it’s so consistent.

Think longer. Turn on “extended thinking” and it slows down noticeably, because it’s breaking the question apart and trying a few paths before answering. The cost is speed and money, and it can actually do worse on questions that don’t need much thought — overthinking can turn a simple question complicated.

Check its own work. This usually means generating several candidate answers and picking the best one, or comparing each sentence back against the source material and flagging anything unsupported. The cost is doubled compute, and the same model tends to approve of its own answers — like proofreading your own draft, where you can read past the same typo ten times and never see it.

Post a guard. The product runs a check both before and after the model, looking for adversarial input or output it shouldn’t produce. This layer is usually a separate, smaller model or a rule set — it doesn’t rely on the large model having a “conscience,” because it doesn’t have one; it’s just calculating the next word. The cost is false positives: a perfectly legitimate question gets refused, and it’s usually this layer that’s responsible.

Route by task. Most products run more than one model behind the scenes — an easy question goes to something cheap and fast, a hard one goes to something expensive and slow — so two messages in the same chat box might not be answered by the same model at all. The cost is that routing gets it wrong sometimes: a question that looks simple but is actually hard gets handed to the cheap model, and you get the sense that “it got dumber today” — it didn’t get dumber, you just got handed off to a different model, and the interface never tells you.

4. Where the Patches Run Out, Agents Begin

If you’ve read anything about “agents,” this might already feel familiar: the six patches above are made of almost the same parts as an agent’s skeleton — and that’s not a coincidence, they were always on the same line. The industry calls everything wrapped around an agent’s core model the harness, and it’s typically built from seven pieces:

The Seven Pieces of a Harness	Does a Chat Product Have It?
Context and memory	✅
Retrieval and verification	✅
Self-checking	✅
Guardrails and permissions	✅ Weak version: governs what it says, not what it does
Goals and planning	🟡 Half: it breaks a question down within one turn, but has no plan across turns
Tools and action (hands)	❌
The think–act–observe loop (a heartbeat)	❌

(Task routing isn’t in this table — that’s the product’s own cost optimization, not part of the harness.)

So a chat product has four and a half of these seven pieces, and it’s missing exactly two — hands, and a heartbeat — and those two are the real dividing line.

“Hands” change what the output actually is. A chat product’s output is still just text — no matter how well-written, you still have to copy it, paste it, send it yourself. Once it has hands, the output becomes a change in the state of the outside world — a file that actually got created, an email that actually got sent. A “heartbeat” means you don’t have to be present. A conversation is turn-based — you’re there for every exchange. A loop runs on its own: think a step, act a step, look at the result, think the next step, repeated dozens of times, with you absent for most of them. Put the two together and the whole nature of the thing changes: getting a sentence wrong and getting an action wrong are not the same order of magnitude of cost.

So that’s exactly where this piece draws its boundary — the overwhelming majority of AI apps on your phone belong to the category of “most of the harness, but still no hands or heartbeat.” However good they get, they remain a conversational partner: one that searches, remembers, and thinks before it speaks. There’s a reassuring corollary here: the cost of an error in this category mostly stays confined to the screen — which is exactly why it’s the best place to learn AI. Once you’ve built your judgment at this layer, you’re ready to move on to the kind that can actually act, without getting burned.

5. Walking Through It: Six Steps Behind One Answer

The patches are a bit abstract taken one at a time, so let’s walk a real question through them. You type: “What was our industry’s market size last year, roughly?” You hit enter. Before you see the first character of the reply, roughly this happens:

Step one is routing. The system judges what kind of question this is — “needs current data” — so it needs retrieval, and possibly a stronger model.

Step two is assembling the desk. The system lays out everything in front of the model: your message, the preceding turns of this conversation, whatever it remembers about you (say, that you mentioned working in renewable energy), and a chunk of system instructions you never see, like “cite your sources” or “say so if you’re not sure.” Notice this step — you typed a dozen words, but the model is looking at a great deal more than that.

Step three is going to look. The model doesn’t answer directly — it first requests a search tool, the system actually goes online, and whatever it finds gets fed back in verbatim.

Step four is answering with citations, and ideally, for anything the material didn’t cover, it says “I couldn’t find a reliable source for this.” Step five is the guard — the output passes through a safety check before it reaches you. In more careful products, there’s a step six: every generated sentence gets checked back against the source material, with anything unsupported flagged or removed.

By the time all six steps finish, all you see is that final screen of text. Two things are worth noting here. First, you control more of this than you’d think — in step two, part of what the model sees comes from you, and the same question phrased as “I mean mainland China, by shipment volume, calendar year 2025” versus just “our industry” produces answers on entirely different levels of quality. A lot of “AI’s answer isn’t accurate” complaints actually trace back to step two. Second, the error could be at any step: routing sent you to the cheap model, memory held something stale, the search pulled up a wrong article, the paraphrasing drifted from the source — and the interface will tell you none of it.

6. Four Holes No Patch Can Fix

Having covered what can be patched, it’s only fair to cover what can’t — otherwise this piece turns into “don’t worry, just use it,” which would be irresponsible.

Retrieval solves “is there a source,” not “is the source right.” What it finds might be an inaccurate news report, stale data, or even content another AI generated — and it hands that back to you dressed in proper citation formatting, which looks a hundred times more credible than something it made up on its own.

A citation can be a real link with a fake summary. The link is real, the page exists, but the model’s paraphrase of it is off, sometimes saying things the source never said — and seeing a link makes you feel safe, which is exactly the moment you should be clicking it. My own rule is simple: for any number or claim that matters, click the link — it costs thirty seconds and blocks most of the trouble before it starts.

More patches means harder debugging when something goes wrong — with six steps in the pipeline, you have no way of knowing which one misfired.

And one hole can’t be patched in principle: anything you can’t verify yourself. A product can lay the evidence in front of you, but it can’t judge that evidence for you. Which gives us a genuinely useful way to sort tasks:

This kind of task	How well it’s patched	How to use it
Public sources, and you can check them	Patched well	Use it freely — click the links
A verifiable ground truth (arithmetic, code, formatting)	Patched well	Use it freely — let it self-verify
You have material and it needs to read it	Patched decently	Use it freely — don’t overload the material
A prediction about the future, or a guess about your company’s internals	Barely patched	Treat it as an opinion, not a conclusion
Obscure data, non-public information	It can find it, but you can’t verify it	⚠️ The most dangerous category
Something that needs your own value judgment	Shouldn’t be patched, and can’t be	Don’t ask it to decide for you

The second-to-last row is the riskiest: it hands you a number and a source, looking complete and credible — but you have no way to actually check that source. The gap between “looks credible” and “is credible” is exactly the part you can’t verify.

7. Five Ways to Live: Where Product Philosophies Split

The six patches above are the common move every product makes. But you’ve probably noticed that even with the same six patches, different products feel very different to use — the difference isn’t how much they’ve patched, it’s where each one stands when facing the same set of trade-offs.

I went through the chat products that have genuinely broken out over the past few years and sorted them into five ways of living, each viewed through four lenses: what it’s trying to be (positioning), which side it takes on a trade-off (design philosophy), what it can actually see and touch (source of capability), and where it’s strong and weak in practice (performance).

The generalist, represented by ChatGPT, and by Doubao in China. These aren’t designed for any specific scenario — they’re designed for everyone’s every question, and when faced with the trade-off of “sacrifice generality for a specialized use case,” they consistently choose generality. Doubao pushes this even further, lowering the barrier to almost zero — many people’s first-ever AI experience started here. It’s the all-purpose tool, and the value of an all-purpose tool is precisely that you don’t have to choose.

The restrained one, represented by Claude. This is, I’d argue, the most distinctly opinionated of the five: caught between “being helpful” and “being honest,” it clearly leans honest — it says it’s not sure when it isn’t, and if your premise is flawed it’ll say so directly rather than run with it. This isn’t incidental — its training is built around an explicitly written-down set of behavioral principles. It also pioneered a form that’s underrated: turning a conversation’s output into something you can edit and take with you directly, instead of scraping it out of a chat log. The cost is that it declines more often than others, and sometimes even a perfectly legitimate question gets a bit of pushback — that’s not a bug, it’s the cost of where it stands. Do you want a collaborator who’ll say “this is wrong,” or an assistant who says “sounds good” to everything? That’s what determines whether it’s the right fit for you.

The source-tracing type — same philosophy, “don’t say it without a source,” pointed in opposite directions, forming a pair. Pointed outward is Perplexity, which isn’t building a chatbot but an answer engine, reinventing search: every claim comes with a citation, and the goal isn’t a pleasant chat, it’s letting you follow the trail down to the source. Its ceiling is whatever’s actually on the internet — if the internet’s wrong, it hands you the same wrong thing. Pointed inward is NotebookLM, a research assistant that only reads the material you hand it — where most AI’s default move is to go search the entire internet for an answer, this one does the opposite: if it’s not in the material, it says so, and every sentence carries a small tag you can click to jump straight back to that spot in the source. It also has one genuinely clever feature: turning a stack of material into an audio conversation between two people, so you can “listen” through your material on a walk — solving the very real problem that most people aren’t unwilling to read, they just never have time to sit down and do it.

Here’s an analogy: most AI is like a well-read friend — ask him anything and he can talk about it, but you never know which parts he actually read and which he just heard somewhere. NotebookLM is like an assistant who’s only read the handful of books you gave him — he knows far less, but ask “which page was that on” and he can flip right to it. When you have to answer for every word you write, the ability to trace a source matters a hundred times more than knowing more.

The ecosystem type, represented by Gemini and Microsoft Copilot, and by Qwen in China. These aren’t building new apps — they’re growing inside things you’re already using: the search box, your documents, your inbox, maps. The philosophy is that distribution is the advantage — the bet isn’t “I’m the smartest,” it’s “you’re going to open this thing anyway, every day.”

Their real moat sits in the source of capability, and it’s two layers deep. The layer you can see: Gemini can check flights, compare prices, look up hotels, because it’s plugged into its own company’s map, flight, and hotel data. The layer you can touch: Qwen connected to Alibaba’s travel and e-commerce systems in early 2026, and can go book the ticket for you directly. The hard part here has nothing to do with the model — for an AI to book you a flight, it needs three things at once: visibility into live inventory and pricing, your account and identity, and the ability to complete payment — and a pure chat product can’t get any of those three, not because it’s incapable, but because it’s simply out of reach. It’s like asking a friend to book your flight — how smart he is matters less than whether he actually has your personal information.

There’s one special case in this category: Grok, whose edge doesn’t come from an ecosystem but from where it’s standing — living on X gives it a direct view of the real-time public discussion stream and community annotations, which gives it a natural advantage in judging whether a spreading claim is true: the most useful signal often isn’t a mainstream outlet’s report (which takes hours to arrive), it’s how many people are already questioning it right now, or whether anyone’s posted the raw footage. It’s like judging whether a restaurant has a line — you could check reviews, or you could just stand at the door and look. The person standing at the door doesn’t need to be smarter. The shared cost of the ecosystem type is this: the moment it can actually spend your money, the nature of the risk changes — which is why these features usually build in a human confirmation step. That confirmation isn’t friction — it’s your only brake.

The open type, represented by DeepSeek and a cluster of open-weight models. These don’t make money from an app — they release the model itself, and anyone can download it, modify it, run it on their own machine. In early 2025, DeepSeek open-sourced something on par with the strongest reasoning model at the time, and its significance wasn’t “who’s stronger” — it was that it changed who could get access at all. Its biggest strength is that “you can take it home with you” — abstract-sounding for a casual user, but critical for anyone handling client files, contracts, or unpublished financial data: what runs locally never leaves your machine; what calls an API sends your data out — and that difference outranks any feature comparison. What it’s selling isn’t the best product. It’s the lowest barrier.

Lined up side by side, the five ways of living look like this, across the same four dimensions:

Way of living	Positioning	Design philosophy	Source of capability	Performance (strong / weak)
Generalist	Everyone’s every question	Go broad first, refine later	Broadest coverage, deepest ecosystem	Handles most daily needs / nothing pushed to the extreme
Restrained	A rigorous writing and thinking partner	Between helpful and honest, leans honest	Long context and writing; output you can edit directly	Best at long-form writing and reasoning / declines more often
Source-tracing	Answer engine / research assistant	Don’t say it without a source	Outward = whole-web retrieval; inward = only your material	Traceable / outward limited by what’s online, inward blind once you’re out of material
Ecosystem	Lives inside the tools you already use	Distribution is the advantage	Sees its own company’s data + can touch accounts and payment	Can actually get things done / limited to advice outside its ecosystem
Open	Turns frontier capability into a public good	Openness and cost first	Deployable locally, data never leaves the machine	Best value for cost / least polished as a product

Two columns in this table get misread most often. “Source of capability” isn’t about brand positioning — it’s two very concrete things: what data it can actually see, and what systems it can actually touch — neither is a measure of intelligence; both are structural differences that come from where it’s standing. “Design philosophy” is where “personality” comes from: helpful, harmless, and honest constantly pull against each other — lean too hard into harmless and it gets cautious and refuses too much; lean too hard into helpful and it starts agreeing with everything you say; lean too hard into honest and it says “I don’t know” more often, which can feel to the user like it “isn’t very good.” Each product sits at a different point in that triangle, and that’s the personality difference you feel — it isn’t a difference in ability, it’s a difference in trade-offs. So the way you should pick a product needs to change: don’t look for the best one — look for the one whose trade-offs actually fit what you’re doing.

8. How to Choose — and Back to “Beyond the Model”

No recommendation list here — products change every month, but the criteria hold for years. Three steps are enough.

Step one: be clear about what kind of work you actually need it for. Most wrong choices trace back to skipping this step. The method is simple: ask where the difficulty in the task actually lies, then ask whose structure happens to solve exactly that.

What you need to do	Where the difficulty lies	Which type to look at
Everyday drafting, rewriting, brainstorming	No real difficulty — needs to be handy and broadly capable	Generalist
Writing something you’ll be held to every word of	Being right, and being able to explain why	Restrained + source-tracing
Working through material, doing research	Whether you can jump straight back to the source	Source-tracing (inward)
Getting up to speed on an unfamiliar topic	Whether the sources can be traced back further	Source-tracing (outward)
You need it actually done, not just advised	Ecosystem, account access, payment	Ecosystem
Handling sensitive data	Where the data actually goes	Open (locally deployable)

Step two: measure its skeleton with four questions. No technical knowledge required — ten minutes of testing does it. Does it search before answering, and cite sources? Can you see, edit, and delete its memory of you? Does it say when it’s not sure? Does it check with you before doing anything for real?

Step three: ask the question that actually determines whether you can use it at all — where does my data go. The first two steps determine how good it is to use — this step determines whether you can use it at all. Don’t do them out of order: plenty of people spend ages comparing features only to find this last box unchecked, and all that comparison was wasted.

One last note on cost. AI is billed on how much work it does, not on whether you’re actively using it — which produces a counterintuitive effect: the smoother it feels, the faster the bill climbs, and a lot of the waste is invisible — a conversation stuffed with irrelevant material pays, every single turn, for material you never needed. So it’s worth asking one more question: does it let you see how much this is actually costing you?

Back to the contradiction we opened with. “AI confidently makes things up” is a statement about the model, and it’s still true — just obscured. “It works great for me” is a statement about the product, and it’s also true, because what you’re enjoying is the scaffolding built up, layer by layer, over the last few years. The value of understanding this gap is this: once you know how thick the scaffolding under you is, you also know exactly where it gets thin.

Three takeaways to carry with you:

1. What you’re using is a product, not a model. The good experience comes from the six patches, not from the model becoming more honest.
2. The differences between products come from positioning, design philosophy, and source of capability — none of which is a measure of intelligence. NotebookLM isn’t dumber for only using your material, Grok isn’t smarter for seeing the real-time stream, and Gemini and Qwen aren’t better models just because they can book your ticket.
3. Pick a product by first naming the kind of work you need, then finding whose structure solves that specific difficulty, and only then asking where your data goes. Don’t start from a feature list.

One memory card: the model is the engine, the product is the whole car · searching before answering is the single strongest patch · always click the citation · the differences are in positioning, philosophy, and source of capability · ask whether it can see, before you ask whether it’s smart.

Next time, we cross that dividing line and talk about the agents that can actually go do things on their own — why one kind of AI can only keep you company, and another is trusted to take on the whole job.

Sources

• Timeline (ChatGPT’s launch and its two-month sprint to 100 million users, plugins and web browsing, context-window jumps, multimodality, reasoning models, deep research, computer use, tool-interface standards, the agent boom): public announcements from various vendors and public reporting, 2022–2026
• DeepSeek R1’s open-source release in January 2025, matching the strongest reasoning model of the time: public release and widespread reporting
• The effect of retrieval grounding and citation constraints on reducing confabulation rates: multiple engineering write-ups and evaluation surveys, 2025–2026 (the reported reduction varies widely by methodology; this piece only takes the direction as given)
• Individual product design and capabilities (NotebookLM’s source anchoring and audio overviews, Grok’s access to X’s real-time stream and community notes, Gemini’s connection to its own maps/flights/hotel data, Qwen’s integration with Alibaba’s travel and e-commerce systems): each company’s official documentation and 2026 reporting
• The breakdown of harness components: compiled from multiple 2026 agent-engineering analyses, with some variation in how each source draws the lines
• The term “confabulation”: U.S. National Institute of Standards and Technology (NIST) Generative AI risk management framework

A note: anywhere this piece is marked “I think” or “my own approach” is opinion; everything else is traceable public information. Every product mentioned is used strictly as an example of a form factor — none of this is a recommendation or a ranking of which is better. This space moves fast, so check the official information at the time you’re reading for current features and boundaries.