Nixie - bare metal provisioning engine for NixOS

Warning

Basic features are functional and you can try them out, but some parts are still missing and there are known issues.

Nixie enables zero-touch deployment of NixOS to multiple machines over the network, turning blank hard drives into fully configured systems in minutes.

Nixie is a simpler, stateless alternative to Foreman, MAAS, OpenStack Ironic, Tinkerbell, etc., focusing on NixOS deployments on bare metal (though it can also be used in virtualized environments).

Features

Currently, only x86_64-linux is supported.

• Simple, declarative JSON configuration
• Build a custom NixOS installer from a flake
• Built-in PXE server based on Pixiecore to serve netboot components from the custom installer
• Install NixOS from a flake using nixos-anywhere
• Remote power-on with Wake-on-LAN
• Host status check with IP discovery
• Stateless and ephemeral ¹
• Fast, under 2 minutes to install NixOS from empty hard drives ²
• Support IPv6 single-stack

Usage

Example command to boot a custom NixOS installer and install the corresponding NixOS configuration from ./examples/flake.nix on multiple bare-metal machines based on the MAC addresses defined in ./examples/hosts.json.

# Running as root for privileged ports
sudo nixie \
    --installer ./examples#nixosConfigurations.installer \
    --flake ./examples \
    --hosts ./examples/hosts.json \
    --ssh-key ~/.ssh/id_ed25519

TODO add a demo video/asciinema.

Please see the full example in ./examples.

How it works

TODO refine the diagram after implementation.

sequenceDiagram
    participant Nix
    participant Nixie
    participant Machines@{ "type" : "collections" }

    Nixie->>Nixie: Load hosts.json

    loop For each machine
    Nixie->>Machines: Try checking status
    Nixie->>Nixie: Skip if already installed
    end

    Nixie->>Nix: Build installer components<br/>(kernel, initrd, squashfs)
    Nixie->>Nixie: Start server components in goroutines<br/>(DHCP/TFTP/HTTP/API)

    loop For each machine
        Nixie->>Machines: Broadcast Wake-on-LAN magic packet

        activate Machines

        Note over Machines: Power on and start PXE boot

        Machines->>Nixie: UEFI firmware broadcast DHCP request
        Nixie->>Machines: DHCP provide IP (via Proxy DHCP) and next server info
        Machines->>Nixie: Request kernel
        Nixie->>Machines: TFTP send kernel
        Machines->>Nixie: Request initrd
        Nixie->>Machines: TFTP send initrd

        Note over Machines: Boot into NixOS installer
        Note over Machines: SystemD starts nixie-agent service
        Machines->>Nixie: nixie-agent phone home to request install with MAC address
        Nixie->>Nixie: Find flake based on MAC address and get client IP from API request
        Nixie->>Nix: Build NixOS configuration
        Nixie->>Machines: nixos-anywhere format disks via SSH based on disko configuration
        Nixie->>Machines: nixos-anywhere install system closure via SSH
        Nixie->>Machines: nixos-anywhere trigger reboot

        Note over Machines: Reboot after installation completed

        Nixie->>Machines: nixos-anywhere confirms machine rebooted
        deactivate Machines

        activate Machines

        Nixie->>Machines: Check host status
    end

    Note over Nixie: Return when all machines are installed

Acknowledgements

This project is a rewrite of my previous ephemeral PXE server implementation in my khuedoan/homelab repository, which used a stateless PXE server in Docker with Ansible templating and targeted generic Linux distributions. It has been rebuilt from the ground up for NixOS, offering a more integrated and robust experience in this project. All credits from the previous implementation are also referenced here, as similar knowledge and concepts apply.

• Ephemeral PXE server ¹ inspired by Minimal First Machine in the DC
• NixOS netboot with pixiecore
• The Pixiecore library
• Custom agent for the installation process inspired by OpenStack ironic-python-agent and Tinkerbell Worker
• Wireshark's WakeOnLAN wiki page
• AMD's Magic Packet Technology white paper

---

1. No pre-configured PXE server is required to install other machines - you only need to run Nixie on your laptop or workstation. This solves the "first machine in the data center" problem: in bare-metal setups with persistent PXE servers, you would otherwise need to automate the installation of the PXE server itself. ↩︎

2. Using the provided examples with a warm cache on a 1 Gbps network. Factors that affect the speed include whether the configuration has been cached in the Nix store, the size of the NixOS configuration, the network speed, etc. ↩︎