The invisible structures, hidden relationships, and emergent rules that shape every application
By Derek Mwale
Software has two architectures.
The first is the one we deliberately construct. It consists of modules, functions, databases, APIs, interfaces, queues, services, and the diagrams we draw before writing a single line of code. We discuss it during planning meetings, document it in repositories, and use it to explain how a system is supposed to work.
The second architecture is harder to see.
It emerges gradually from the decisions we make, the assumptions we leave undocumented, the relationships that develop between components, and the behaviors that appear after thousands of individual operations interact. It exists in the spaces between our abstractions. It lives inside the expectations of our developers, the habits of our users, and the invisible agreements that allow the entire system to function.
I call this the latent architecture of software.
It is the structure that exists beneath the structure we intended to build.
As developers, we spend considerable time designing the visible architecture. We decide where business logic belongs, how data moves between services, and which responsibilities each component should own. Yet some of the most influential properties of a system are not represented in its original design.
They develop through use.
A function becomes indispensable because several unrelated components quietly depend on its behavior. A database column acquires meaning beyond its original purpose. An API response becomes a contract that nobody formally documented. A small configuration setting determines the behavior of an entire deployment.
Eventually, the system contains rules that nobody explicitly designed but everybody must respect.
Understanding this hidden layer changes how we approach software engineering. It encourages us to look beyond the files in a repository and examine the relationships, expectations, and accumulated decisions that make a system behave the way it does.
1. Every Codebase Contains More Than Its Source Code
Consider a simple application that manages customer orders.
Its visible architecture might look straightforward:
- A frontend displays products and collects orders.
- An API receives requests.
- A service validates business rules.
- A database stores customers, products, and transactions.
- A payment integration processes charges.
A diagram containing these components appears to explain the system.
But the diagram leaves out several important questions.
Does the frontend assume that every order contains at least one item? Does the payment service expect amounts in a particular currency? Does the database contain historical records that older application versions still need? Does the order service depend on a specific sequence of database operations?
These assumptions may never appear in the architecture diagram.
Nevertheless, they influence whether the application works.
Imagine that the frontend sends prices as decimal numbers, while the payment integration expects integer values representing the smallest currency unit. Somewhere between these components, a conversion must happen correctly.
If the conversion is handled inconsistently, the application may contain a defect even though every individual component appears reasonable.
The hidden architecture includes the agreement about how money is represented.
The same principle applies to dates, identifiers, permissions, error responses, transaction boundaries, and countless other details.
A system is not merely a collection of components. It is a collection of components whose internal expectations must remain compatible.
The source code reveals what the system does. Its latent architecture helps explain why those particular behaviors are necessary, how they became interconnected, and what might break when one of them changes.
2. Assumptions Are Architectural Components Without Names
In software, an assumption is often an invisible dependency.
Suppose a function receives a user identifier and retrieves a profile:
def get_profile(user_id):
return User.objects.get(id=user_id)
The implementation is small. Its behavior appears obvious.
However, the surrounding application may depend on several unstated conditions.
The identifier must refer to an existing user. The calling code must handle the possibility that no user exists. The database must enforce the expected uniqueness rules. The caller must not assume that retrieving a profile automatically authorizes access to it.
Each condition influences the correctness of the operation.
Some assumptions are enforced by the programming language. Others are enforced by database constraints, validation rules, tests, or runtime checks. Many exist only in the developer's understanding.
That last category is particularly interesting.
An assumption that exists only in someone's understanding cannot be inspected directly by a compiler. It cannot automatically trigger a warning when violated. It may remain invisible until another developer changes the surrounding code.
This is how a simple implementation can acquire a surprisingly large conceptual footprint.
The function itself might occupy five lines, but the rules governing its safe use may span several modules.
We can think of an assumption as an architectural component that has not yet been given a name.
Once identified, it can be made explicit through an interface contract, a validation rule, a test, a type definition, or documentation.
For example:
def get_profile(user_id):
if user_id is None:
raise ValueError("user_id is required")
return User.objects.get(id=user_id)
This change makes one assumption explicit, although it does not address every possible concern. Authorization, missing records, and other failure conditions still require deliberate handling.
The broader lesson is not that every assumption belongs inside a function. It is that important assumptions should be visible somewhere in the system.
An undocumented assumption is a rule the system may depend on without knowing how to explain.
3. Dependencies Grow in Places We Do Not Expect
When developers discuss dependencies, they often think about package managers.
A project depends on a web framework, an authentication library, a database driver, and several external services. These dependencies can be listed, versioned, audited, and updated.
But software contains another kind of dependency: behavioral dependency.
Imagine that an application has a utility function called format_user().
Initially, the function exists to format a user's name for display. Over time, developers discover that it is convenient and begin using it in email notifications, audit logs, API responses, and report generation.
Eventually, several components depend on the exact output of this function.
Perhaps it joins the first and last names with a space. Perhaps it removes empty values. Perhaps it converts missing names into an empty string.
None of these behaviors may have been intended as permanent guarantees.
Yet changing the function could affect every component that adopted its output.
The dependency is not necessarily recorded in a package manifest. It is embedded in the behavior of the application.
This is one way latent architecture develops: convenience gradually becomes convention, and convention gradually becomes a constraint.
The more widely a behavior is reused, the more carefully its meaning must be managed.
This does not mean that reuse is inherently dangerous. Reuse is one of the most powerful tools in software engineering. It reduces duplication and creates opportunities for consistent behavior.
The important distinction is between deliberate reuse and accidental coupling.
Deliberate reuse occurs when developers understand and maintain a shared contract.
Accidental coupling occurs when components depend on details that were never intended to become part of their interface.
A useful engineering question is therefore not simply, "How many places use this function?"
It is also, "Which aspects of this function have those places come to depend on?"
The answer reveals more about the system's actual architecture than a list of imports alone.
4. Data Carries Architectural History
A database is often treated as a storage layer: a place where information waits until the application needs it.
In a mature application, however, data is also a record of architectural decisions.
Consider a table containing customer information.
Its original schema might include a name, email address, and registration date. Later, the application adds account verification, subscription status, notification preferences, and customer categories.
Eventually, some fields become redundant. Others are retained for compatibility. A field that originally represented a temporary state becomes essential to reporting. An identifier generated by an older service becomes the reference used by several newer services.
The database begins to preserve the history of the application.
Even when the code changes, the data may continue to reflect decisions made years earlier.
This creates a subtle distinction between the architecture developers see in the current repository and the architecture required to interpret existing records correctly.
Suppose an application changes how it represents order status.
The old version uses pending, completed, and cancelled. The new version introduces additional states and changes some transitions.
Updating the application code is only part of the task. Existing records must remain interpretable. Reports must continue to make sense. Background jobs must understand the new values. Integrations must not misinterpret states they were never designed to receive.
The latent architecture includes these historical obligations.
This is why database migrations require more than modifying a schema. They require thinking about the meaning of data across time.
A database is not just a snapshot of the present system. It is often a bridge between multiple generations of the system.
Developers who recognize this are more likely to design backward-compatible migrations, staged deployments, and explicit data transformations.
They understand that changing the structure of information can change the behavior of every component that relies on it.
5. Architecture Emerges From Repeated Decisions
Not every architectural property originates in a formal design document.
Some emerge from repeated local decisions.
A developer adds a helper function to avoid duplication. Another developer follows the same pattern. A third introduces a slightly different variation. After several months, the codebase contains a recognizable style, even though nobody deliberately established it.
This is an example of an emergent structure.
In complex systems, repeated local interactions can produce global patterns. Software is no exception.
Imagine a team that consistently places validation logic inside API controllers. At first, this is simply a convenient implementation choice.
As the application grows, new endpoints follow the same pattern. Eventually, controllers become responsible for request parsing, authorization checks, business validation, database access, and response formatting.
The architecture has changed.
No single decision necessarily caused the transformation. Each decision appeared reasonable within its immediate context.
Together, however, they produced a system in which controllers own far more responsibility than originally intended.
The latent architecture is the cumulative result.
This phenomenon is worth examining because software projects rarely remain static long enough for their initial architectural plans to describe every future detail.
Requirements change. Developers change. Frameworks evolve. Features accumulate. Deadlines influence implementation choices.
The resulting system is partly designed and partly discovered.
This does not imply that emergent architecture is always undesirable. Some of the best architectural patterns emerge from observing real usage and learning which abstractions are genuinely useful.
The challenge is distinguishing beneficial emergence from accidental complexity.
A useful practice is to periodically examine repeated patterns and ask whether they still serve the system.
If several modules independently implement the same business rule, perhaps the rule belongs in a shared domain component.
If every endpoint requires a slightly different workaround for the same limitation, perhaps the underlying interface needs redesigning.
If developers repeatedly struggle to determine where a particular responsibility belongs, perhaps the architecture does not express that responsibility clearly enough.
Patterns are signals. They reveal what the system is becoming.
6. The Architecture Between Components Matters Most
A component can be internally well designed while participating in a poorly designed system.
This is one of the most important observations in software engineering.
Consider two services, each with a clear responsibility. One manages orders; the other manages inventory.
Individually, both services might have clean interfaces and well-tested logic.
But what happens when an order is placed?
The order service might create a transaction, request inventory reservation, call a payment provider, and finally confirm the order.
What happens if payment succeeds but the confirmation request fails?
What happens if the inventory reservation expires while the payment request is still processing?
What happens if a client retries the request after a network timeout?
These questions exist between the services.
They concern ordering, consistency, retries, idempotency, timeouts, and failure recovery.
The architecture connecting the components determines whether the complete workflow remains correct when individual operations do not proceed as expected.
This is where seemingly simple systems reveal their deeper complexity.
A service diagram may show two boxes connected by an arrow. The actual relationship may require a carefully designed protocol for coordinating state across distributed operations.
The arrow represents much more than communication. It represents expectations about timing, success, failure, and responsibility.
As a system grows, these relationships can become more influential than the components themselves.
A useful way to investigate an unfamiliar application is therefore to trace a complete operation from beginning to end.
Follow a user action through the interface, request handler, business logic, database, external services, and eventual response.
At every boundary, ask four questions:
- What information crosses this boundary?
- What assumptions does each side make?
- What happens if the operation fails?
- Which component is responsible for restoring a valid state?
These questions reveal architectural details that are easy to miss when examining files individually.
They also help distinguish a collection of functioning components from a system that functions reliably as a whole.
7. Tests Reveal the Architecture Developers Believe In
Tests are commonly used to verify correctness.
They also provide clues about the architecture a team considers important.
A unit test might demonstrate that a function calculates a discount correctly. An integration test might verify that an order is persisted after a successful request. An end-to-end test might confirm that a customer can complete a purchase.
Together, these tests describe important behavioral expectations.
But tests can reveal something else: the boundaries developers believe should remain stable.
If a module can be tested independently, its dependencies are probably controlled well enough to isolate its behavior.
If testing a small function requires constructing an entire application environment, that may indicate tightly coupled responsibilities.
If changing a database implementation breaks dozens of unrelated tests, the application may expose storage details across more boundaries than necessary.
These observations are not automatic verdicts. Some systems genuinely require broad integration tests, and not every complex test indicates a design problem.
Nevertheless, the structure of a test suite offers a useful perspective on the structure of the application.
Tests also preserve knowledge that might otherwise disappear when developers leave a project.
A test documenting how duplicate payment requests should behave is more than a verification mechanism. It records a rule that future implementations must respect.
Similarly, a test for expired sessions, missing permissions, or inconsistent inventory quantities captures expectations that might not be obvious from the implementation.
In this sense, tests act as executable architectural documentation.
They make selected parts of the latent architecture explicit.
A mature testing strategy does not attempt to freeze every implementation detail. Instead, it protects the behaviors and boundaries that matter while allowing internal implementation to evolve.
That distinction is crucial.
The purpose of a test is not to prevent change. It is to help developers change the system without accidentally changing what the system means.
8. Observability Makes Invisible Structure Easier to See
Some aspects of architecture cannot be fully understood by reading source code.
They become visible only when the system is running.
A request may pass through several services before reaching its destination. A background job may wait in a queue. A database query may become slower as records accumulate. An external dependency may respond inconsistently during periods of high demand.
The code describes the possible execution paths. Observability helps reveal the paths that actually occur.
Logs show events. Metrics reveal patterns over time. Distributed traces connect operations across service boundaries.
Together, these signals make hidden relationships easier to investigate.
Suppose users report that an application occasionally takes several seconds to load an order history page.
The visible interface offers little explanation. The frontend may be functioning correctly, and the database may appear healthy when tested independently.
A distributed trace might reveal that the endpoint calls four services sequentially, each waiting for a response before proceeding.
The issue is not necessarily the speed of any individual service. It may be the architecture of the interaction.
Alternatively, the trace could reveal repeated database queries, unnecessary network calls, or an external request that dominates the total response time.
Observability transforms vague symptoms into evidence about actual system behavior.
It allows developers to compare intended architecture with operational reality.
This is particularly valuable because software behavior is influenced by factors that static diagrams cannot fully capture: workload distribution, network conditions, concurrency, data volume, and the timing of failures.
A well-designed system should therefore make important behavior observable.
Meaningful logs, request identifiers, useful metrics, and appropriately sampled traces are not merely operational conveniences. They are instruments for understanding the architecture that emerges during execution.
Without them, developers may be forced to infer the system's internal behavior from incomplete symptoms.
With them, hidden interactions become measurable.
9. Refactoring Is an Exercise in Architectural Discovery
Refactoring is often described as improving the internal structure of code without changing its externally observable behavior.
That definition is useful, but it leaves out an important part of the experience.
Refactoring frequently reveals what the existing system actually depends on.
A developer begins extracting a service from a controller and discovers that several unrelated features rely on a particular side effect. Another developer attempts to replace a database library and discovers that application code depends on a specific exception format. A team introduces a cleaner interface and discovers that external clients have come to rely on undocumented response fields.
Each discovery exposes part of the latent architecture.
The difficulty is that dependencies are not always visible where the dependency originates.
A function may write to the database, update a cache, publish an event, and trigger a notification. A caller that expects only a return value may unknowingly rely on several of these side effects.
Changing the function's implementation can therefore change behavior far beyond its apparent responsibility.
This is why safe refactoring begins with investigation.
Before moving code, identify its callers. Before changing a shared interface, examine its consumers. Before removing a field, determine whether historical records or external integrations depend on it.
Then establish tests around the behavior that must remain stable.
The objective is not to eliminate every hidden relationship. Some relationships are necessary. A payment service must interact with a payment provider, and an order service must communicate with inventory.
The objective is to make those relationships intentional, understandable, and manageable.
Good refactoring does not merely rearrange files.
It reduces the distance between the architecture developers believe they have and the architecture their software actually implements.
10. The Developer as an Archaeologist
There is an interesting similarity between understanding a mature codebase and investigating an archaeological site.
An archaeologist does not assume that every object can be understood in isolation. The position of an artifact, its relationship to nearby structures, and the layers in which it appears all contribute to its meaning.
Software requires a similar approach.
A function is an artifact. A database schema is a historical layer. A configuration file preserves a decision. An API contract connects one part of the system to another.
The task is to reconstruct the reasoning that connects these pieces.
Why does this service contain a particular validation rule?
Why does this table preserve a field that appears unnecessary?
Why does one endpoint use a different error format from the others?
Why does a background job run before another job that appears unrelated?
Sometimes the explanation is simple. Sometimes it reflects an old requirement. Sometimes it is a deliberate workaround for a limitation in an external system.
And sometimes nobody remembers.
The absence of an explanation does not automatically mean that the design is wrong. It means that further investigation may be necessary before changing it safely.
This perspective encourages a particular kind of engineering discipline: curiosity before intervention.
Instead of immediately rewriting unfamiliar code, trace its behavior. Examine its callers. Review its tests. Inspect its data. Look at the logs. Search for the same assumption in other parts of the application.
Build a model of the system before attempting to improve it.
This is especially valuable in older applications, where the original architecture may no longer match the architecture required to keep the system operating.
The codebase is not simply a problem to solve. It is a history to understand.
11. Making the Latent Architecture Explicit
If every sufficiently complex application develops hidden structures, can developers do anything to manage them?
Yes.
The goal is not to eliminate every implicit relationship. That would be impossible. The goal is to identify the relationships most likely to affect correctness, maintainability, security, and change.
Several practices help.
Document important contracts. Record what modules and services expect from one another, including data formats, error behavior, authorization requirements, and compatibility guarantees.
Make invariants enforceable. If an order cannot exist without a customer, enforce that rule where appropriate through application validation and database constraints. Do not rely solely on a comment or an undocumented convention.
Reduce accidental coupling. Use explicit interfaces and deliberate boundaries so that components depend on stable behavior rather than incidental implementation details.
Treat data migrations as behavioral changes. Consider how older records, application versions, reports, and integrations will behave before changing the meaning or structure of stored information.
Test failure paths. Verify what happens when requests are retried, services become unavailable, records are missing, or operations succeed only partially.
Observe the running system. Collect enough operational evidence to understand how components interact under real workloads.
Review recurring patterns. When developers repeatedly encounter the same difficulty, investigate whether it points to a missing abstraction, an unclear contract, or a responsibility placed in the wrong component.
These practices help turn invisible assumptions into explicit engineering decisions.
They also make the system easier to explain to someone who did not participate in its original development.
That is an important measure of architectural quality: not whether every detail is documented, but whether the important rules can be discovered, understood, and changed safely.
Conclusion: The Architecture Beneath the Diagram
Software architecture is often presented as something developers design before implementation.
In reality, architecture continues to develop throughout the life of a system.
Every new feature introduces relationships. Every integration establishes expectations. Every migration carries decisions forward. Every repeated workaround influences future implementation choices.
Over time, these interactions create a structure that may be considerably more complicated than the original design.
That structure is the latent architecture of software.
It explains why changing a small function can affect an unrelated feature. It explains why an apparently redundant database field may be essential. It explains why two individually reliable services can produce an unreliable workflow when their interaction is poorly designed.
Most importantly, it explains why reading source code is necessary but not always sufficient for understanding a system.
The deeper task is to understand the relationships that give the code its meaning.
As developers, we should learn to examine not only what each component does, but also what it assumes, what it influences, and what other components have come to expect from it.
We should look for the contracts that were never written, the dependencies that grew through convenience, and the historical decisions that remain embedded in current behavior.
This is not an argument for making every application more complicated. Quite the opposite.
Recognizing latent architecture helps us distinguish essential complexity from accidental complexity. It helps us decide where stronger boundaries are needed, which assumptions should be enforced, and which abstractions genuinely improve the system.
The best architectures are not necessarily the ones with the most diagrams, services, or layers.
They are the ones whose important relationships are understandable, whose rules remain consistent, and whose components can evolve without creating unnecessary surprises.
Every codebase has a visible structure.
The real engineering challenge is learning to see the structure underneath it.
Because software is never just the code we write. It is also the network of expectations that makes that code work.
Derek Mwale — Software in My Mind. Rock in My Soul.
