Tesla FSD Does Not Know What It Cannot Do

1. Vendor and Product Reality

Tesla, Inc., founded in 2003 and operating as the largest pure-play electric-vehicle manufacturer in the world, is the dominant consumer-deployed driver-assistance vendor by installed fleet. Its Autopilot stack and Full Self-Driving option have iterated through multiple hardware generations — from HW2 through HW3 to HW4 and the rumored HW5 — and through a sequence of software architectures culminating in the end-to-end neural-network model that defines current FSD releases. Tesla's vehicle fleet, numbering several million units across Models S, 3, X, Y, and Cybertruck, generates a real-world driving telemetry corpus that no competitor has matched in volume.

The product surface is consumer-direct. FSD is sold as an option on every new Tesla vehicle and as a monthly subscription, with the marketing position that the system is progressing toward unsupervised autonomy through over-the-air software updates. The current customer-visible product is "FSD (Supervised)" — a Level 2 driver-assistance system that handles highway driving, city streets, parking, and progressively more complex maneuvers, while requiring continuous driver supervision and intervention readiness. Tesla's robotaxi program, announced under the Cybercab brand and progressing through limited geofenced pilots, is the planned commercial extension into ride-hail without a safety driver.

The architectural shape is well-known and architecturally distinctive. Where Waymo, Cruise (formerly), Mobileye, and Zoox use lidar-augmented perception, pre-mapped HD maps, and explicitly defined operational design domains (ODDs), Tesla uses camera-only perception (after dropping radar and ultrasonics) and a learned end-to-end policy trained on fleet-collected video. The engineering bet is that a sufficiently capable neural network trained on enough miles will generalize across all driving conditions without an explicit ODD. The Dojo training infrastructure, the auto-labeling pipeline, and the shadow-mode evaluation system are serious engineering investments. Within its scope — consumer Level 2 ADAS with an aggressive feature roadmap — FSD is the most-deployed system of its kind, and the analysis that follows takes that capability as given.

2. The Architectural Gap

The structural property Tesla's architecture does not exhibit is a computed, persistent representation of its own capability envelope as an input to action selection. FSD's policy network produces driving actions conditioned on perception; it does not produce, alongside those actions, a structured self-assessment of what scenarios it can reliably handle now, under these specific conditions, against this specific scene. The internal uncertainty that exists — softmax distributions, planner cost margins, perception-confidence scores — is an artifact of inference, not a first-class operational quantity that gates whether the system attempts a maneuver.

The gap matters because the failure mode of a system without capability awareness is, structurally, an attempt followed by an intervention. FSD encounters a scenario, attempts whatever the policy emits, and either succeeds or is recovered by the human driver. The vehicle's own representation of the situation does not contain a quantity equivalent to "I cannot reliably handle this combination of conditions; reduce speed, request supervision, or pull over before attempting." Tesla's published interventions and the National Highway Traffic Safety Administration's open investigations document the consequence: situations the system handles poorly are discovered through exposure rather than through self-assessment in advance.

Disengagement telemetry, shadow-mode comparison, and over-the-air model updates do not close this gap. They are retrospective controls — the system improves after failures by ingesting them as training data — rather than prospective controls that would have prevented the failure by not attempting the scenario. The improvement loop is real, and it is exactly what one would build if capability awareness were not architecturally available; but it is structurally distinct from a system that computes, before acting, whether the action is within its current envelope.

Tesla cannot patch this from within the current end-to-end-policy architecture because the policy was designed as a context-conditioned action generator, not as a substrate that produces a credentialed envelope alongside its actions. Adding a confidence-threshold cutout is not capability awareness; it is a one-dimensional gate over a single estimator. Adding more training data does not produce envelope computation; it improves the action distribution. Adding a teleoperation fallback does not produce temporal executability forecasting; it produces a recovery surface. The envelope is an architectural shape — a persistent, multi-dimensional, time-extended quantity — and the current FSD shape is fundamentally that of a learned policy emitting controls, with confidence as a derived statistic rather than a governed operational variable.

3. What the AQ Capability-Awareness Primitive Provides

The Adaptive Query capability-awareness primitive specifies that every cyber-physical actor in a conforming system maintain a persistent capability envelope as a first-class operational variable, updated in real time, governing action selection through structural gates with recursive closure. Property one — persistent envelope — represents the actor's reliable operating domain as a multi-dimensional structure over scenario types, environmental conditions, and task complexities, with each dimension scored against demonstrated performance on prior similar scenarios. The envelope is not a confidence score; it is a state object the system carries, updates, and reasons over.

Property two — temporal executability forecasting — extends the envelope across time, computing not only what the actor can do now but what it will be able to do across the planned action horizon as conditions evolve. A system that can drive this highway now but cannot drive it in thirty minutes when fog arrives must know this before committing to the route. Property three — uncertainty-weighted execution gating — couples the envelope to action selection through a structural gate: a candidate action is admitted only if the joint condition of capability, time, and uncertainty places it inside the envelope, and graduated outcomes (proceed, proceed-with-margin, defer, refuse, request-supervision, pull-over) replace the binary attempt-or-disengage of current architectures.

Property four — envelope negotiation — exposes the envelope to other actors (the human driver, a supervising fleet operator, an infrastructure traffic-management system) as a structured, queryable surface so that handoffs, supervision requests, and cooperative maneuvers are governed by mutual capability knowledge rather than by binary autonomy-state transitions. Property five — recursive update — closes the loop: every executed action produces an outcome observation that re-enters the envelope as evidence, expanding it where performance was demonstrated and contracting it where degradation was observed, with regression detection that suspends envelope regions when continued degradation is observed. The recursive closure is load-bearing: an envelope is itself an observation downstream coordinators admit, weight, and respond to. The primitive is technology-neutral (any perception stack, any policy class, any sensor suite) and composes hierarchically (vehicle, fleet, jurisdiction). The inventive step disclosed under USPTO provisional 64/049,409 is the closed envelope-forecast-gate-negotiation-update loop as a structural condition for capability-aware autonomous systems.

4. Composition Pathway

Tesla integrates with AQ as a domain-specialized perception-and-policy surface running over the capability-awareness substrate. What stays at Tesla: the camera-based perception stack, the end-to-end neural policy, the Dojo training infrastructure, the auto-labeling pipeline, the OTA delivery system, the vehicle hardware, and the entire customer commercial relationship. Tesla's investment in driving-specific knowledge — sensor calibration, scene understanding, planning under physical constraints — remains its differentiated layer.

What moves to AQ as substrate: every action emitted by the policy passes through an envelope gate that evaluates whether the action is within the vehicle's current capability envelope under temporal forecasting. The integration points are well-defined. The policy proposes; the envelope gate disposes. Candidate actions exercising scenario types outside the current envelope are not attempted with hope-and-recover semantics; they trigger graduated outcomes — speed reduction, margin expansion, supervision request, route replanning, controlled pull-over — that are themselves credentialed observations re-entering the system. The envelope itself is computed from a combination of demonstrated-performance history (the same fleet telemetry Tesla already collects, repurposed as envelope evidence rather than only as policy training data), real-time perception and condition assessment, and temporal forecasts derived from weather and infrastructure data.

Envelope negotiation surfaces the system's current capability state to the driver as structured information rather than as the binary chime-and-disengage of current Autopilot. The driver knows which capabilities are at full strength and which are degraded, and the handoff protocol becomes capability-aware rather than time-bounded. For the planned robotaxi product, the substrate is load-bearing in a different way: a vehicle without a safety driver must be able to compute its envelope and refuse rides, defer routes, or request remote supervision without human-in-the-vehicle recovery. The new commercial surface is capability-credentialed autonomy for regulators and insurers — relevant to NHTSA, EU GSR-2, UNECE R157, and the emerging insurance frameworks that price autonomous-system risk by demonstrated and self-known capability rather than by miles driven. The envelope belongs to the vehicle's authority taxonomy and is portable across software updates and even across vehicle resale, which paradoxically makes Tesla stickier because the perception and policy stack is what differentiates its access to that substrate.

5. Commercial and Licensing Implication

The fitting arrangement is an embedded substrate license: Tesla embeds the AQ capability-awareness primitive into FSD and into the Cybercab robotaxi stack, and sub-licenses envelope participation to fleet operators, insurers, and jurisdictional supervisors as part of the operational deployment. Pricing is per-credentialed-vehicle or per-envelope-mile rather than per-software-seat, which aligns with how regulated mobility actually consumes capability assurance.

What Tesla gains: a structural answer to the "FSD attempts what it cannot reliably do" critique that has driven NHTSA investigations and the persistent supervised-versus-unsupervised positioning problem, a defensible regulatory posture against Waymo and Mobileye that elevates the architectural floor from disengagement statistics to credentialed envelope computation, and a forward-compatible position for the robotaxi launch that requires a structural answer to "what does the vehicle refuse to do" before the safety driver is removed. What the customer and regulator gain: portable capability lineage for each vehicle, defensible answers for crash investigators about whether the system attempted a scenario it knew was outside its envelope, and a single capability-awareness chain spanning Tesla, fleet operators, infrastructure providers, and insurers under one operational taxonomy. Honest framing — the AQ primitive does not replace the perception or policy stack; it gives the autonomous system the substrate it has always needed and never had.