Most product teams designing AI features start with capability: what can this model do? That question shapes what you build, but three others shape whether it succeeds: What are users ready to accept? How much of the work should be automated? And where does a human need to stay in the picture?
The Overton Window: Know Where Your Users Are
Joseph Overton developed the Overton Window in the 1990s to describe something he observed in politics: at any given moment, there is a range of ideas the public will accept, and outside that range, even sound proposals fail. The window is not fixed. It shifts as norms evolve, and what felt unacceptable last year can become expected within a few years.
Applied to AI product design, the window describes user readiness to use a capability. Your feature does not exist in a vacuum. It lands in a context shaped by what users have already encountered, absorbed, and normalized across every product they use. Adapted to AI features, the original spectrum positions might look like this:
| Position | What it means for an AI feature |
|---|---|
| Unthinkable | Broadly rejected, often on principle. AI making lethal targeting decisions without human authorization, AI determining criminal sentences without human review. |
| Radical | Meaningful resistance from a significant share of users. Autonomous agents taking actions without confirmation, AI autonomously managing financial portfolios without human approval. |
| Acceptable | Adopted with deliberation. Copilot-style generation of full functions, AI drafting email responses. |
| Sensible | Accepted and understood within existing workflows. Inline code suggestions, AI-assisted search within a tool users already trust. |
| Popular | Users reach for it without friction. Smart reply suggestions, fraud detection on credit card transactions. |
| Default | So embedded users no longer recognize it as AI. Content recommendations on streaming platforms, email spam filtering. |
Gmail Smart Reply is a useful illustration. When Google first launched Smart Reply in 2015, it landed somewhere between Radical and Acceptable: users questioned whether it felt authentic, and many were uneasy with the idea of a machine reading their mail to generate responses. By 2017, when Google expanded it to all of Gmail, it already accounted for 12% of mobile replies in Inbox. Today, AI writing assistance sits at Popular for most users, with some features already at Default.
The window shifts through two distinct mechanisms. The first is normalization through exposure: users build genuine tolerance by working with lower-stakes versions of a capability over time, and that comfort accumulates through use rather than persuasion. The second is anchoring through contrast: a feature further out on the spectrum makes adjacent features look modest by comparison, shifting perception faster than genuine normalization can. Both matter for deciding which capability to ship, and both have implications for how you use the other two mental models.
A feature that lands outside the window is not necessarily wrong. It may need time for user familiarity to catch up, or it may need exposure to move the window. But the risk cuts both ways, and it is worth noting that the window is not a single position shared by all users. A developer who has worked with AI tooling for two years and one who encountered it last month are not looking through the same window. Teams that calibrate to their most vocal users, who are rarely at the median, tend to overshoot the broader user base without realizing it.
Shipping ahead of genuine normalization can generate resistance that is difficult to recover from. Google Glass illustrated this before AI features were a mainstream design concern: the product was technically capable, but shipping eyeglasses that could record in public spaces without obvious consent landed far outside what social norms accepted at the time. The term “Glasshole” became shorthand for the backlash, and the product never recovered in the consumer market. A 2015 study by researchers at the Wharton School found that people lose confidence in algorithms far faster than in humans after seeing comparable errors, and that restoring willingness to use an algorithm typically requires giving users some degree of control over its outputs. The more consequential implication is that overshooting doesn’t just produce a failed feature. It leaves users more resistant to the next feature in the category than they were before the failed launch.
The window also shifts faster than many teams expect in the other direction: a feature calibrated to where users were eighteen months ago may already feel dated.
Sheridan’s Levels of Automation: Autonomy Is a Dial
In 1978, Thomas Sheridan and William Verplank at MIT published a taxonomy of automation levels, originally developed for undersea teleoperator systems. It describes ten levels of human-computer control, spanning two distinct territories: assistance, where the human retains decision authority, and automation, where the computer acts and the human’s role shifts to supervisor or observer:
| Level | Description |
|---|---|
| 1 | The human does everything |
| 2 | The computer offers alternatives |
| 3 | The computer narrows alternatives to a few |
| 4 | The computer suggests one alternative |
| 5 | The computer executes if the human approves |
| ↑ Assistance Automation ↓ | ↑ Assistance Automation ↓ |
| 6 | The computer executes; the human can veto |
| 7 | The computer executes and informs the human |
| 8 | The computer executes and informs the human if asked |
| 9 | The computer executes and informs the human if it decides to |
| 10 | The computer acts entirely autonomously |
The level you choose determines how much control users retain and, by extension, how much trust they need in the system before they are comfortable with it.
Email spam filtering shows this progression. Early email clients placed full responsibility on the user. Filters were introduced that flagged suspected spam for review. Most email platforms today filter at something close to Level 9: the system acts, files messages away, and surfaces the outcome only if it decides to. Most users never open the spam folder unless they go looking. That progression tracked alongside accuracy and trust. Neither was built overnight.
The relevant question when designing an AI feature is not what level the model is capable of supporting. It is what level is appropriate for this user, this context, and the current state of trust. Starting at Level 4 or 5 and moving up as confidence develops is a deliberate design choice. It is also normalization through exposure made structural: each level builds the comfort that makes the next one possible.
Human in the Loop: Oversight Is a Design Decision
Keeping humans in the loop means maintaining human judgment at the decision points where errors carry real consequences. It is not the same as adding a confirmation step. The distinction is whether users have access to the information they need to verify what the AI is doing, not just the ability to accept or reject its output.
AI-assisted radiology makes this concrete. Diagnostic tools can analyze imaging scans and flag potential findings. Radiologists make the final call. They have access to the patient’s full history, prior imaging, and clinical context. The AI surfaces patterns. The physician decides what they mean. Removing the physician from that process would not just remove a safeguard. It would remove the access to information that makes the judgment meaningful.
Human oversight in AI systems does more than catch errors. It builds the trust that allows users to move toward higher levels of autonomy over time. It feeds corrections back into the system, improving performance. And it preserves accountability in contexts where decisions have real consequences.
As reliability improves, the need to check AI outputs will fade. That is a reasonable outcome to design toward. What should not change is whether users can. That availability of verification is what makes normalization load-bearing rather than fragile: users who can verify what the AI did build genuine comfort; users who can only accept or reject its output do not.
Using the Three Mental Models Together
Use the Overton Window to understand what your users are ready to accept. That tells you where you fall on Sheridan’s scale and what level of autonomy is viable right now. That level tells you where human oversight is required and what it needs to look like.
When GitHub Copilot launched as a technical preview in 2021, it was Radical: developers were skeptical about AI-generated code quality, concerned about licensing, and anxious about what it meant for their craft. GitHub ran the tool free for over a year before charging for it, and the interaction design was deliberate: suggestions appeared as gray ghost text that developers could accept or ignore, mapping onto the autocomplete behavior they already understood. The developer reviewing and accepting each suggestion was the human in the loop, and that oversight was what made the tool trustworthy enough to adopt. Today Copilot sits at acceptable or sensible for most development teams: accepted, understood, and integrated into existing workflows.
GitHub’s agent mode, which can execute commands autonomously in a developer’s environment, launched in early 2025. It is at Radical today. Because of that, the appropriate Sheridan level is 5 or 6: the agent drafts or executes an action, but the developer approves it first or retains the ability to stop it before it takes effect. That dial setting defines what HITL requires: the developer needs to see what the agent read, what it decided, and what it is about to change, clearly enough to approve, modify, or stop it. A confirmation prompt is not enough. The visibility has to be sufficient for the developer to make a real decision.
None of the three mental models is a one-time check. As the window moves, the dial moves with it, and the oversight design has to evolve to match.
