Homelab Infrastructure as Code

Infrastructure-as-Code repository used to build, configure and operate my Proxmox-based homelab.

The project is primarily a learning and experimentation environment for DevOps, infrastructure automation and systems administration.

It uses Terraform for infrastructure provisioning and Ansible for host and guest configuration. Infisical is the current operator's normal secret source, not a universal prerequisite in the target design. Some current wrappers still require it; see actual inputs and gaps.

The approved reconstruction direction adds K3s for suitable workloads, Flux as the initial GitOps candidate and TrueNAS centralized application storage. K3s is experimental and installed; Flux/CSI remain planned. Rebuild useful functionality and preserve technical data; do not clone every historical guest. The physical target is one Proxmox K3s server VM each on pve-k8s-01, pve-k8s-02 and pve-core, with mini-PC workload preference and core fallback. A single-host lab is not physical HA. Review the canonical architecture before implementation.

Start with the documentation index, architecture and roadmap. Read AGENTS.md and the assigned task before implementation. Follow the documentation maintenance contract. For loss of the controller or control plane, start with the recovery contract and minimum kit specification. It records prerequisites and missing evidence, not a completed recovery process. The Recovery Kit preparation provides the capture plan and synthetic structural checks (make recovery-check); no kit has been exported or recovered. The headless controller profile is validated in CI but has not been deployed; it is now an optional test adapter for portable Mac/Linux recovery, not a required bootstrap dependency.

The Stage A profile now provides a separate three-node headless root, OS baseline and gated Make interfaces. VMs303-305 are deployed; the Debian baseline and zero-change second convergence are live verified. Three 4-GiB VMs use pve-lab while workstations stay off; K3s is installed. The Task110 K3s bootstrap now provides pinned, serial installation and private lab-token/snapshot interfaces. The minimum development baseline is live verified and operator accepted after targeted system-pod repair; advanced lifecycle drills remain open. Flux/demo now has reviewed manifests, scoped RBAC and make flux-check / make flux-install interfaces. Code/CI pass; live bootstrap is blocked on controller SSH reachability. No Flux installation, CSI or application migration is claimed yet.

The Tailscale router pair provides a separate repository-only LXC/Ansible workflow (make tailscale-check/plan/apply/configure). Allocations, deployment and failover are not yet verified; pve-infra stays intact.

For a new environment, start with Starter prerequisites and input contracts, not the original operator's credentials. Proxmox/access/network and suitable storage must already exist. Infisical/private/Starter/Recovery paths must reuse one definition. Stage A now supports explicit private/Infisical inputs and your public SSH key; follow the Starter example. Other adapters and full kit generation remain planned, not operational claims. Recovery is a supporting capability, not a prerequisite for disposable learning. Priority pve-infra evacuation assessment is parallel work, including an interim core path; no live inventory/cutover yet.

Goals

• Treat infrastructure configuration as code
• Make VM deployments reproducible
• Minimize manual configuration
• Separate infrastructure provisioning from OS configuration
• Keep credentials and secrets outside of Git
• Build reusable automation instead of one-off scripts
• Experiment with production-style DevOps workflows in a homelab environment

Tech Stack

• Proxmox VE — virtualization platform
• Terraform — VM and infrastructure provisioning
• Ansible — host and guest configuration
• Infisical — secrets management
• Prometheus — metrics collection
• Grafana — monitoring and visualization
• Docker Compose — service deployment where appropriate
• Git / Forgejo — source control and CI/CD experimentation
• Make — repeatable local workflows

Architecture

The following diagram describes the existing operator workflow, not the new Kubernetes target. The single authoritative target and service reconstruction matrix are in architecture; implementation order is in roadmap. K3s lab installation does not imply service migration.

flowchart TD
    Admin[Admin Workstation]

    Secrets[Infisical]

    Terraform[Terraform]
    Ansible[Ansible]

    Proxmox[Proxmox VE Cluster]

    Workstations[Workstation VMs]
    Services[Infrastructure VMs / Containers]

    Monitoring[Prometheus + Grafana]

    Admin --> Terraform
    Admin --> Ansible

    Secrets --> Terraform
    Secrets --> Ansible

    Terraform --> Proxmox
    Ansible --> Proxmox

    Proxmox --> Workstations
    Proxmox --> Services

    Monitoring --> Proxmox
    Monitoring --> Workstations
    Monitoring --> Services

Terraform

Terraform manages infrastructure resources such as:

• Proxmox virtual machines
• CPU and memory allocation
• Virtual disks
• Network interfaces
• VM tags
• References to PCI/USB mappings owned by Ansible

Current workstation roots adopt existing VM hardware; they do not reconstruct installed operating systems (clone sources are currently null). Base-image creation and complete recovery remain separate work.

Infrastructure-specific secrets are injected at runtime through Infisical rather than stored in Terraform configuration.

Operator workflow (plan queries live infrastructure; apply changes it and requires explicit approval of the reviewed plan):

make workstations-plan
make workstations-apply WORKSTATIONS_HEADLESS_APPROVED=yes
make workstations-verify

Ansible

Ansible handles configuration that belongs to the operating system or Proxmox host rather than to the VM resource itself.

Examples include:

• Guarded retirement of workstation VFIO and physical resource mappings
• Host and guest OS configuration
• Monitoring deployment
• Node Exporter deployment
• Host auditing

The roles aim for idempotence; syntax checks alone do not prove repeatable convergence on a fresh host. Consult the relevant qualification/runbook first.

Headless Workstations

pve-lab is a network-managed Proxmox host. Windows and Linux workstations use standard virtual VGA/noVNC and virtual input, not a physical GPU, USB peripherals or a KVM. Existing OS disks, EFI/TPM and network identities are preserved. The retired passthrough exclusivity hook is removed; this does not imply enough RAM to run every workstation concurrently. Size concurrent use explicitly.

See headless conversion and maintenance for approval gates, state reconciliation and remote versus physical validation.

Repository Structure

.
├── ansible/
│   ├── inventories/
│   ├── playbooks/
│   └── roles/
│
├── terraform/
│   ├── examples/
│   └── stacks/
│       ├── deb13-monitoring/
│       ├── pve-lab-workstations/
│       ├── pve-compute-forgejo-runner/           # historical
│       ├── pve-compute-forgejo-runner-migration/ # active CT301
│       ├── pve-core-garage/
│       ├── pve-core-tfstate/
│       ├── pve-lab-controller/                 # undeployed, optional test host
│       └── pve-lab-k3s/                        # deployed Stage A VMs, local state
│
├── services/
│   ├── monitoring/
│   └── ci-images/
│
├── docs/                  # architecture, roadmap, runbooks, dated audits
├── tasks/                 # bounded work and acceptance evidence
├── scripts/
├── tests/
├── .forgejo/workflows/
├── AGENTS.md
├── keys/
├── Makefile
└── ansible.cfg

terraform/stacks/pve-lab-workstations

Manages existing workstation identities and virtual-only display/input.

terraform/stacks/deb13-monitoring

Provisions infrastructure used by the monitoring stack.

ansible/roles/pve_workstation_host

Retires workstation-owned passthrough mappings, hooks and VFIO boot overrides with explicit maintenance approval. Does not reboot or change CPU affinity.

services/monitoring

Contains the Prometheus/Grafana monitoring configuration.

Guest Configuration

Guest configuration is discovered from Proxmox VM tags. A guest must have the guest tag, and capability tags such as workstation, dev, gaming, or nvidia determine which roles are selected. The guest_base role always runs; capabilities do not imply one another.

The dynamic inventory reads Proxmox API credentials from runtime environment variables: PROXMOX_URL, PROXMOX_USER, PROXMOX_TOKEN_ID, and PROXMOX_TOKEN_SECRET. Guest SSH settings are supplied through GUEST_LINUX_USER, GUEST_WINDOWS_USER, and optionally GUEST_SSH_PRIVATE_KEY_FILE. These values must come from Infisical or another secure runtime injection mechanism, never from Git.

Windows guests currently use OpenSSH with PowerShell as the shell. This keeps the Linux and Windows connection model consistent with the repository's existing SSH-based controller conventions. A live prerequisite remains: each Windows guest must have OpenSSH Server, PowerShell remoting over SSH, and the chosen key-based account configured before Ansible can connect.

Workflows (only guests-check is offline; inventory queries Proxmox and guests-apply performs live changes requiring approval):

make guests-check
make guests-inventory
make guests-apply LIMIT=dev
make guests-apply LIMIT=w11-game

guests-check validates the playbook without contacting guests. guests-inventory queries Proxmox and shows discovered groups. guests-apply requires an explicit LIMIT; it does not run against every guest by default.

Historical NVIDIA tags in existing VM profiles are not hardware assignments. Workstations now use virtual console drivers; no NVIDIA driver provisioning or physical passthrough is part of their deployment.

Controller Prerequisites

Use Git, Make, Terraform matching the controlled workflow version, and Python compatible with requirements-controller.txt (Ansible 14 requires Python >=3.12). make setup-controller CONTROLLER_PYTHON=/path/to/python3 recreates disposable .venv from the Python and collection dependency files. CI pins its own Python; local setup otherwise uses system python3. Downloads require network access. Live workflows additionally need authorized SSH/API trust and runtime credentials; see the relevant runbook rather than copying private inputs into Git.

Forgejo Validation

CI currently performs state-independent validation only. Production Terraform state is operator-local; plan/apply CI is blocked pending the reviewed state migration and CI/CD safety gates.

The repository includes a validation-only Forgejo Actions workflow at .forgejo/workflows/validate.yml. It targets main only and is gated by CI_QUALITY_APPROVED; run139 passed on CT301/Linux AMD64 at 01ccefb. Outstanding runner trust findings still block formal security closure. Label homelab-iac selects the existing image for controller setup, nine-root backend-disabled validation, Ansible/Compose checks, tests and secret scanning. See CI quality boundaries and local validation.

The workflow does not request Proxmox or Infisical credentials or invoke plans, apply, destroy or convergence. That does not make the shared runner safe for untrusted code: CT301 uses one daemon and three connections with a global socket allowlist. No pull-request job is authorized on that runner. The manual publish-ci-base.yml workflow handles later image releases; first publication uses a one-time controller-driven SSH procedure. The workflow requires package-write Actions secrets and explicit socket access. See docs/forgejo-runner.md for the intended runner separation and one-time bootstrap checks.

Secrets

Never commit credentials to this public repository.

Current normal workflows use Infisical where implemented. Explicit private inputs are the intended reusable alternative, with no automatic fallback; input contracts distinguish actual support from gaps.

Files containing local configuration, Terraform state and environment-specific values are excluded from Git.

Example configuration files are provided where appropriate:

terraform.tfvars.example
.infisical.example.json

The tracked .infisical.json contains only non-secret project-link metadata. Machine Identity credentials, access tokens, and exported secrets remain runtime-only.

Current Status

This repository is actively evolving as I expand the homelab and learn more about infrastructure automation.

Capability	Evidence level and limits
Selected Proxmox resources, guest/host roles and monitoring	Implemented; dated deployment evidence in the documentation index, not complete homelab reconstruction
Quality CI	LIVE VERIFIED in run139; runner trust preflight remains separate and unresolved
Terraform state	Seven stack states remain local, including Stage A; no migration; single-writer model; recovery inventory follow-up before capture; future backend decided by requirements
Independent recovery	Contract accepted; preparation/tests implemented; private payloads, custody qualification and recovery tests pending
K3s / Flux / TrueNAS integration	Stage A installed, etcd verified, system-pod permissions still block110; Flux/CSI planned, no data migration; Task100 accepted
Reusable headless module / protected CD	Stage A module implemented; protected CD remains gated; existing workstation state unchanged

Task files own current progress. Dated audits remain historical evidence and must not be rewritten to imply later implementation or live verification.

Why This Repository Exists

My background is primarily in software development, but building this homelab led me deeper into infrastructure, Linux, networking and DevOps.

Rather than configuring systems manually, I use the lab to practice building infrastructure that is reproducible, version-controlled and automated.

The goal is not simply to self-host services, but to understand the tooling and engineering practices used to operate infrastructure reliably.