Making Invalid States Unrepresentable: The Billion-Dollar Mistake That Wasn't

Making Invalid States Unrepresentable: The Billion-Dollar Mistake That Wasn't

●1 ●29 ●58
calendar_today • schedule12 min read
— Originally published at lucentowl.com

The billion-dollar mistake. That's what Tony Hoare, in a 2009 talk, called his invention of the null reference for ALGOL W in 1965. The quote gets repeated so often that "null is dangerous" has become conventional wisdom. Entry-level and intermediate developers especially hear it as dogma, without understanding the context or the alternatives that can be far worse.

But I think we're blaming the wrong villain. When developers avoid null, they often reach for a default instead of the fix Hoare himself had in mind, references the compiler checks. Compared with those defaults, null may have saved far more than it ever cost. In a managed runtime, a null reference exception stops the system at the moment it tries to use a value nobody supplied. A default would let it proceed with corrupted data and invalid logical decisions. The billion-dollar mistake framing counts the crashes but ignores the corruption that never happened.

Information security professionals value the CIA triad of Confidentiality, Integrity, and Availability. Software developers tend to obsess over availability, and that's understandable since a crashed service is visible, embarrassing, and can violate business SLAs. But in any system whose data outlives the request, integrity failures are often far worse. Data that looks valid but isn't can corrupt your system just as surely as SQL injection or a man-in-the-middle attack. The corruption just compounds slower and is harder to detect. This is the lens through which the null debate should be understood.

In managed runtimes like the CLR and the JVM, null crashes loudly at the point of misuse, which is an availability problem you can see and fix. A default value that masks missing data? That proceeds quietly until it causes a security vulnerability, a financial miscalculation, or data corruption that might require weeks to detect and more to fix.

What "Making Invalid States Unrepresentable" Actually Means

Yaron Minsky popularized the idea as "make illegal states unrepresentable" in his Effective ML material at Jane Street, including the 2011 post "Effective ML Revisited." It comes from functional programming, but the concept is practical: structure your data so that invalid combinations cannot exist. Invalid states should fail at construction time, not at runtime deep in business logic.

Consider a user registration:

public class UserRegistration
{
    public string Email { get; set; } = "";
    public string Password { get; set; } = "";
}

This class allows every invalid state imaginable. Empty email, empty password, any combination. The defaults make it easy to construct an object that looks valid but isn't. Code that receives this object has no way to know whether the empty string represents "not provided" or "explicitly set to empty" or "bug in upstream code."

Compare:

public class UserRegistration
{
    public string Email { get; }
    public string Password { get; }

    public UserRegistration(string email, string password)
    {
        if (string.IsNullOrWhiteSpace(email))
            throw new ArgumentException("Email is required", nameof(email));

        if (string.IsNullOrWhiteSpace(password))
            throw new ArgumentException("Password is required", nameof(password));

        Email = email;
        Password = password;
    }
}

Now invalid states cannot be constructed, and there's no default email to mask a missing value.

C# 11 introduced the required keyword, which moves this enforcement to compile time for simpler cases:

public class UserRegistration
{
    public required string Email { get; init; }
    public required string Password { get; init; }
}

The compiler rejects any object initializer for UserRegistration that skips either property.

required enforces presence, and constructors enforce validity. required guarantees assignment, not a non-null value, so Email = null! still compiles. Use required when presence is all you need. Use constructors when you need validation logic, like checking that the email contains an @ or that the password meets complexity requirements.

At system boundaries, objects are often built by reflection rather than by code the compiler checks, so required can't protect them. Many serializers, ORMs, some dependency injection containers, and mocking frameworks can create the object without ever setting the property. For API contracts and external data, you still need runtime validation with [Required] attributes or explicit checks. System.Text.Json partly closes this gap. It has honored required since .NET 7, but it accepts an explicit null unless you enable RespectNullableAnnotations, added in .NET 9.

Where the Problem Usually Starts: API Contracts

The domain model above is clean, but most developers encounter this tension at the API boundary first. Consider a typical request DTO:

public class CreateUserRequest
{
    [Required]
    public string Email { get; set; } = "";

    [Required]
    public string Password { get; set; } = "";

    [Required]
    public bool RegisterForAlerts { get; set; } = false;
}

The [Required] attribute signals intent, but the developer adds = "" out of habit, a misguided sense of defensive coding, or to silence the compiler's CS8618 warning about an uninitialized non-nullable property. Now there's a contradiction: the attribute says "required" while the code says "default to empty string."

Which half wins depends on how the object gets created. A test or internal caller that builds the object directly never runs validation, so its empty string passes silently. Through the serializer, the default does no harm. The serializer creates the object before reading the payload, so a missing Email becomes "", and [Required] still rejects it because it treats empty strings as missing by default.

The bool is worse. Microsoft's ASP.NET Core model validation documentation notes that a non-nullable field is always valid, so [Required] on a bool can never fail, and a client that omits RegisterForAlerts gets false with no error.

The fix is simple. Don't add the default, and declare every field nullable so its absence is visible.

public class CreateUserRequest
{
    [Required]
    public string? Email { get; set; }

    [Required]
    public string? Password { get; set; }

    //The business could decide that this should default to false but test that assumption first!
    [Required]
    public bool? RegisterForAlerts { get; set; }
}

The [Required] attribute ensures the framework validates these fields before your code ever touches them, and bool? finally gives it a null to reject. If validation is bypassed, calling a method on the null Email throws a NullReferenceException and reading RegisterForAlerts.Value throws an InvalidOperationException, rather than handing back values that look valid.

But null is loud only where code dereferences it. Coalescing with ?? false, string interpolation, and a nullable database column all let it pass quietly. So null needs a partner. Validate at the boundary, then hand the value to a non-nullable domain type or database column, which turns any null that slips through back into a loud failure.

Why Defaults Are More Dangerous Than Null

Default values create four categories of problems that a null checked at construction avoids.

Silent propagation of invalid state. When a required field defaults to an empty string or zero, the invalid state propagates through the system. Each layer assumes the previous layer validated the data. Nobody validated it because it never looked invalid. The corruption accumulates until something finally breaks far from the source.

Consider a payment processing system:

public class PaymentRequest
{
    public decimal Amount { get; set; } = 0m;
    public string Currency { get; set; } = "USD";
    public string MerchantId { get; set; } = "";
}

A bug upstream fails to set the amount. The payment proceeds with Amount = 0, and an internal ledger that allows zero-value entries for credits and free tiers records it without complaint. No crash, no exception, no alert. The transaction logs show a valid-looking payment. Days later, someone notices revenue is wrong. The investigation takes hours because nothing obviously failed. Deleting = 0m wouldn't help, because a decimal is zero anyway. The fix is decimal? or required, which gives absence somewhere to show.

Ambiguous semantics. Does Amount = 0 mean "free transaction," "not set," or "bug"? Does Email = "" mean "user declined to provide" or "form field wasn't rendered"? Null keeps absence separate from every legitimate value.

A default value claims knowledge it doesn't have. Null admits ignorance.

This ambiguity becomes critical in update operations. When a client submits an update request, the API needs to distinguish between "set this field to empty" and "don't touch this field." In a typed object, null is the simplest way to express that distinction. The alternatives, such as JSON Merge Patch documents, protobuf field masks, or an Optional<T> wrapper, add machinery to every update contract. That machinery earns its place when a client must also be able to set a field to null, because plain null can't tell that request apart from "leave it alone."

Validation bypass. Code that checks if (amount != null) correctly identifies missing data. Code that checks if (amount != 0) conflates "missing" with "zero." Legitimate zero values become impossible to represent.

Security vulnerabilities. Consider a RateLimitPerMinute field that defaults to 0. In some systems, zero means "no limit" (nginx reads client_max_body_size 0 that way), so a malformed request that should be rejected instead gets unlimited access. Or a Permissions string that defaults to empty, which a downstream parser interprets as "inherit all permissions from parent." With null, the missing field stays distinguishable from a deliberate zero, so a validator can reject the request, require the field, or make a conscious decision about what absence means. A parser that reads null as "unlimited" repeats the same mistake.

Hoare's Mistake Was Unchecked Null, Not Null

Hoare called null his billion-dollar mistake, and the criticism was valid for its time. His own account says his goal was references checked automatically by the compiler, and the mistake he confessed was adding a null the compiler didn't check. The slogan dropped the compiler-checking half. His confession holds up. What doesn't hold up is the slogan's reading of it, that null itself is the danger.

ALGOL W and its descendants treated every reference as implicitly nullable. The compiler couldn't help you, and nothing forced developers to consider absence. In C and C++, where dereferencing null is undefined behavior rather than a guaranteed crash, unchecked null also produced the silent failures and vulnerabilities Hoare's talk describes.

But modern type systems address this problem without eliminating null. C# 8.0 introduced nullable reference types, compiler analysis that distinguishes string (expected never to be null) from string? (might be null) and warns when code dereferences a value that might be null. Kotlin distinguishes String from String? and refuses to compile the unchecked dereference. TypeScript has strict null checks. The billion-dollar mistake wasn't null itself; it was nullable references in type systems that never flagged missing handling.

Hoare's mistake wasn't inventing null. It was inventing null without inventing string?.

The mistake we keep making today is different. It's the pattern of masking errors with defaults instead of failing fast. Every system that returned -1 instead of throwing an exception. Every API that substituted empty arrays for error responses. Every constructor that initialized required fields to placeholder values. Null can mask an error too, when a function returns it in place of the error, and it raises the alarm only where presence was required. So the line isn't null versus default. It's whether something forces the absence to be handled.

Counterarguments and When Defaults Make Sense

"Null reference exceptions are the most common runtime error." They're among the most common, and that's the point. The frequency of null reference exceptions reflects how often code fails to handle absent values, not a flaw in null itself. Non-nullable types do cut that frequency, by moving the failure to compile time rather than hiding it. Replacing null with defaults doesn't. It turns the same bugs into data corruption instead of crashes.

The frequency also says something about the team. Most of those exceptions are ordinary oversights. But some keep recurring, in places where a field everyone assumed was required turns up missing. Those can signal something deeper, a continuous misalignment between the development team and stakeholders about what the system should accept and produce. Unit tests exist to test assumptions and prove agreement in both application logic and API contracts. If null reference exceptions keep appearing, the team hasn't captured those agreements in tests, or the agreements themselves are unclear. In those cases, the exceptions are symptoms of a collaboration problem, not just a coding problem.

"Option/Maybe types are strictly better than null." For representing intentional absence, they are better. Option<User> makes it explicit that a user might not exist, and pattern matching forces you to handle both cases. That's also why Option.getOrElse(default), used to silence a None the code should have handled, defeats the purpose. The whole point is to force handling, not to provide an escape hatch.

But this proves my argument rather than refuting it. In a language with exhaustive matching, like Rust, a match that ignores the None case won't compile. That's the same principle I'm advocating: force handling, don't mask absence. Most runtimes depend on null, and used correctly with modern type systems, it comes close to the guarantee Option types give in functional languages.

C#'s analysis is advisory, and !, default, and deserializers can all defeat it, so its guarantee is weaker than Rust's. Treating nullable warnings as errors closes the gaps the compiler can see. Arrays, struct defaults, and reflection-based construction stay outside its analysis, and boundary validation has to cover them.

"Defensive programming means providing safe defaults." This conflates two different concerns. Resilience at system boundaries means handling malformed external input gracefully, but that's different from masking bugs internally. Providing "safe" defaults inside the system just moves the failure somewhere harder to diagnose.

"Users shouldn't see crashes." Correct, which is why you handle errors at system boundaries. But the crash should still happen internally. Catch exceptions at the API layer, log the details, return a user-friendly error. The internal crash gave you the information to fix the bug. A silent default would have hidden it. In a queue or batch job, outside any request, the crash should send the one record to a dead-letter queue, not stall the records behind it.

"Some fields genuinely have sensible defaults." True. A CreatedAt timestamp defaulting to DateTime.UtcNow makes sense. A RetryCount defaulting to 0 represents legitimate initial state, and so does an empty order list for a customer who has never ordered. The same empty list returned for a failed fetch masks the failure. The distinction is between defaults that represent valid initial state versus defaults that mask missing required data. User-provided data, external inputs, and required business fields typically don't have one.

Exceptions for Bugs, Result Types for Expected Outcomes

If failing loudly is the goal, why not use exceptions everywhere?

A null on a required field represents a violated constraint, something the system was promised it wouldn't receive. That's a bug. The correct response is to crash, log, and fix the code. A Result type represents an expected domain outcome: "user not found" or "validation failed" aren't bugs. They're legitimate results that correct code produced from valid input.

If correct code with valid input could produce this result, use a Result type. If not, fail fast with an exception. Both approaches force handling; neither lets you ignore failure and proceed with corrupted state. The danger is when either mechanism gets misused to mask absence: catching exceptions and substituting defaults, or calling Result.GetValueOrDefault() without handling the failure case.

Validate at Construction, Trust at Consumption

The confusion around null often stems from conflating two different phases.

At construction time, invalid states should fail immediately. Validation should happen once, at the boundary, with clear errors for invalid input. Objects that exist should be valid by construction.

At consumption time, code shouldn't need to check validity. Internal code that receives a UserRegistration shouldn't need to re-validate the email because the constructor already guarantees it's present and valid.

This unsettles developers who've been taught to validate defensively at every layer. But spreading validation across layers is itself a source of bugs. When validation logic lives in the controller, the service, the repository, and the domain model, you've scattered what should be encapsulated business rules across your entire codebase. When validation rules change, you update three places and miss the fourth.

This doesn't mean a single validation layer. Systems have multiple trust boundaries: the API gateway, service boundaries, aggregate roots, database constraints. Each boundary validates what it needs to trust. Validate at each door, trust everyone inside that room.

Practical Guidelines

Enforce requirements at the correct layer. At ingress boundaries (API DTOs, deserialization), fields may be nullable because input might be missing. After validation, domain objects should have non-nullable required fields because their existence proves validity. Inside the domain, a nullable int? signals "this is optional."

Reserve defaults for genuinely optional fields with valid initial states. Retry counts, timestamps, tuning settings like timeouts, and accumulators qualify.

Prefer crashes to silent corruption. A null reference exception fails at the first dereference of the missing value instead of at an unknown later point. A default value that hides the bug lets it reach production and corrupt data.

Failing Loudly Is a Feature

The fear of null comes from the pain of decades of null reference exceptions in production, and systems that crashed when they should have kept running in theory. But crashes are symptoms, not the disease. The disease is code that doesn't handle absence properly. The cure isn't eliminating null; it's using null correctly as part of a system that makes invalid states unrepresentable.

Default values do the opposite. When you substitute a default for missing data, you're creating records that claim to represent reality but don't. Every downstream system that trusts that data inherits the lie.

The billion-dollar mistake we're still making isn't null. It's the widespread practice of substituting defaults for validation, prioritizing code that runs over code that runs correctly. In a system whose data outlives the request, given the choice between an availability problem you can see and fix, and an integrity problem that compounds invisibly until something important breaks, I'll take the availability problem every time.

2 Comments

1 vote
1 vote
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

Local-First: The Browser as the Vault

Pocket Portfolio - Apr 20

Frameworks Are Institutional Memory

Ken W. Algerverified - Sep 17

The Trust Gap: Why Your Product Fails Even When the Math is Right

Karol Modelski - Jul 16

Split-Brain: Analyst-Grade Reasoning Without Raw Transactions on the Server

Pocket Portfolio - Apr 8

Why I Changed My Mind About Exceptions

Steven Stuart - Nov 26, 2025
chevron_left
3.1k Points • 88 Badges
Boca Raton, FL • stevenstuartm.com
26Posts
12Comments
21Connections
Software architect with 15 years of experience bringing clarity to complex systems, from web and des... Show more

Commenters (This Week)

1 comment
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!