Documenting your infrastructure
Infrastructure that is not documented is infrastructure that only one person understands, and only on the days that person remembers. A small estate can get by on informal knowledge for a while, but as soon as something breaks at an inconvenient time, or a second person needs to help, the gaps become expensive. Documentation is the work of making the estate understandable to someone who is not you, at two in the morning, under pressure.
Inventory as the foundation
An inventory is more than a list of hostnames. For each system, record what it is, what it does, who owns it, how to reach it, what depends on it, and what it depends on. Include network devices, hypervisor hosts, virtual machines, backup appliances, and services that run on shared infrastructure. A machine that is not in the inventory is a machine that will be missed during a recovery.
Keep the inventory in a format that is easy to update and easy to search. A spreadsheet works for a small estate. A structured text file or a simple database works better as the estate grows. What does not work is a document only one person has, stored in a format that requires a specific application to read. A Markdown table kept next to your other notes gives you search with grep, a diff whenever something changes, and no dependency to install.
Runbooks for the things that break
A runbook is a short, step-by-step description of how to perform a specific task or recover from a specific failure. The best runbooks are written by the person who knows how to do the task, at a time when nothing is broken, and they are tested by someone who has never done it before.
Write runbooks for the things that break most often and the things that matter most. A runbook for restarting a critical service, restoring a backup, or recovering from a failed replication should exist before the failure happens. Keep each one numbered and explicit, and short enough to follow in one sitting. Assume the reader is tired and stressed, and does not remember the details.
Change log as institutional memory
A change log records what changed, when, why, and who did it. It is different from version control. Version control tracks changes to files. A change log tracks changes to the infrastructure. A new firewall rule, a DNS record update, a patch applied to a production server, a backup configuration changed, all belong in the change log.
Keep the change log close to the work. If it requires a separate process to update, it will not be updated. A simple text file in a shared location, or a structured entry in the same system that holds the inventory, is usually enough. The goal is to be able to answer what changed in the last week, which is often the first question in an investigation.
Diagrams that help
Diagrams are useful when they show relationships that are hard to see in a list. A network diagram that shows which hosts sit in which VLAN and where traffic flows between segments is worth maintaining. A diagram that tries to show everything becomes unreadable.
One point is easy to get wrong on a diagram. A VLAN membership label says where a host is attached, not what that host is allowed to reach. Traffic between segments is filtered by the firewall or router rules that enforce the policy, so the diagram should record where that filtering happens. NIST SP 800-125B treats network segmentation and firewall-based traffic control as separate concerns in virtual networks (NIST SP 800-125B).
Keep diagrams simple and focused. One diagram per question beats one diagram that tries to answer everything. A diagram for network topology, a diagram for backup flow, a diagram for identity and access. Update them when the infrastructure changes, or they become misleading.
Version control for documentation
If the documentation lives in text files, put it in version control. The history of changes, the ability to see what was different last month, and the ability to collaborate on updates are all useful. Even a small estate benefits from treating documentation as code, with the same review and commit discipline. A commit message written in the words of the change, followed by a line in the change log pointing at that commit, gives you both the file history and the operational history.
Review cadence
Documentation that is never reviewed becomes wrong. Schedule a review at a regular cadence. Quarterly is reasonable for a small estate. After every significant change, review the affected documentation. After every incident, check whether the documentation helped or hindered the response, and update it accordingly.
Checklist
- Maintain an inventory that records what each system is, what it does, and what it depends on.
- Write runbooks for the most common failures and the most critical recoveries.
- Keep a change log close to the work and update it as part of the change.
- Draw simple, focused diagrams that answer specific questions.
- Put documentation in version control if it lives in text files.
- Review documentation quarterly and after every significant change or incident.
- Treat documentation as living work, not a one-time task.