Nodes
A node is a server that Rookery applies without a desktop, such as one that hosts Incus and Docker machines. Creating one, taking in a server that exists, and what Rookery leaves to you.
A node is a server that Rookery sets up and keeps up to date, without a desktop. Its plugin features install what the server is for, such as Incus or a Docker engine for Rookery's machines. Rookery then has a provisioner for each, and the provisioner records which node it is on.
| Part | On a node |
|---|---|
| System | Ubuntu 24.04 or later on x86_64, its time zone and locale, and Ubuntu's security updates without reboots |
| SSH | Keys only. The SSH user takes Rookery's key, and root keeps a login with keys when Rookery logs in as root or root had one before |
| Controller | rookeryd and box, with the actions status, update, reboot and secrets. Sessions run as the SSH user |
| Firewall | ufw: incoming traffic is refused except SSH, the tailnet and what the features open |
| Tailnet | Tailscale, joined as a workstation joins (Tailnet) |
| Features | Only those the profile lists, such as incus/server and docker/engine |
| Not on a node | A desktop, the agent and operator accounts, VNC, T3, Cockpit, and the agents' features |
A node is a host type between managed and workstation. It is applied and rolled out like a workstation. It has no cloud seed or USB installer, because it is a server you install with Ubuntu first.
Create one
Install Ubuntu on the server from your provider's panel, or with the rescue-install recipe when that installer does not work. Then add it by its address as a node. This example uses the made-up name big and the address 192.0.2.20:
rookery host add big 192.0.2.20 --user root --auth ssh --type node \
--feature docker/engine --feature incus/server
rookery apply big--feature names the features it gets, which go into the profile's ansible_vars as rookery_plugin_features. rookery host set big feature.incus/server=off changes them later. rookery host create big --type node asks for the address, the login and the features at a terminal, with no desktop or accounts. The screens show the type node on the Install tab and the features of a node on the Features tab.
The first apply installs Rookery's key for the SSH user. A node added with --auth password is logged in to with Rookery's key afterwards, and Rookery forgets the password. --auth ssh stays as it is.
An ssh or managed host becomes a node with rookery host set big type=node. A node is not applied over a workstation that Rookery set up: the role stops before it changes anything.
| Profile field | On a node |
|---|---|
type | node |
access | Where Rookery reaches it and as whom. The SSH user is the account sessions run as |
target | cloud, or container for a node without systemd. laptop is refused |
ansible_vars | rookery_plugin_features, the settings of the features, and the node's own settings below |
| The agent's and the operator's fields | Refused, as a node has neither account. rookery host set HOST type=node takes them out |
| Setting | Default | Meaning |
|---|---|---|
rookery_ssh_from | any | Who reaches SSH from outside the tailnet: any, or one address or network. The address an apply comes from always does. A server taken in keeps SSH as limited as its firewall had it |
rookery_harden_ssh | true | Keys only for SSH. It is skipped while the SSH user has no key |
rookery_manage_firewall | true | Set the node's firewall |
rookery_install_tailscale | true | Install Tailscale and join the tailnet with a key Rookery makes (Joining the tailnet) |
rookery_timezone | Europe/Berlin | The node's time zone |
Features for a node
| Feature | What it sets up | Provisioners Rookery registers |
|---|---|---|
incus/server | Incus, the bridge incusbr0 at 10.77.0.1/24, a storage pool in a directory, and the API on port 8444 that trusts Rookery's client certificate | big-incus-vm and big-incus-container |
docker/engine | The Docker engine, and Sysbox when asked | big-docker-container, which reaches the engine over SSH |
The guides of the Incus and Docker plugins list their settings. A feature says which host types it is for. Rookery refuses to apply a node that lists a feature for workstations, and the reverse.
Provisioners the node offers
At the end of an apply the role records what each feature offers in /etc/rookery/applied.json, and rookeryd reports it. Rookery then registers a provisioner for each:
| What | Where it comes from |
|---|---|
| The name | NODE-PLUGIN-TYPE, such as big-incus-vm. rookery provisioner rename gives it another, and it stays the node's |
| The node | The host it is on, on the provisioner's page (node over the API), and kept when the node is renamed |
| Settings | Where Rookery reaches the node, such as server=https://192.0.2.20:8444 |
| Secrets | Public values only, such as the Incus server's certificate, kept as the provisioner's own incus_server_cert |
Rookery makes its Incus client certificate and key once, as the secrets incus_client_cert and incus_client_key. Only the certificate goes to the node, which trusts it. The key stays in Rookery's store.
Reach what runs on a node
The endpoints of a resource on a node's provisioner are dialled by the node's controller, not by the server (Reach an endpoint). So resources.connect reaches a container on incusbr0 or docker0, and a port published on the node's 127.0.0.1, also when the server has no route to them. Each request to the provisioner has the node's name in node, and the Docker and Incus plugins then report addresses as the node sees them.
| Address of the endpoint | The node's controller |
|---|---|
127.0.0.1 | Connects to that port of its loopback |
On a network of one of the node's interfaces, such as 10.77.0.57 | Connects to it |
| Any other, a link-local or a metadata address | Refuses it |
The node's firewall allows outgoing connections. The controller's connections to its bridges pass.
Rookery takes from a node's report only what is the node's own, as the node is a machine it manages:
| Part of the report | Taken when |
|---|---|
| A feature | The node's profile names it |
| A provisioner | The feature's manifest lists it in offers |
| A setting | It is one of the provisioner's own arguments. A URL in it has to point at the node's address, or Rookery leaves the provisioner out |
| A secret | It is one the provisioner declares |
The node's provisioners are those Rookery registered for it or made its own. A node's report never changes or removes any other provisioner.
An apply with the same features doesn't change them. A feature taken out of the profile takes its provisioners with it at the next apply. While one of them still holds machines or deployments, Rookery refuses to apply the node without the feature, and it refuses to remove the node. Remove those resources first, or keep the feature.
Take in a server that exists
A server that has Incus or Docker installed already is added and applied as a node like a new one. The features take what is there:
-
Incus with a storage pool keeps its pools, networks, profiles and trusted clients. Only the API address, when it has none, and Rookery's certificate are added.
-
An installed Docker engine stays, with its containers and volumes, also one from Docker's own packages.
-
A firewall rule made by hand that lets SSH in from some addresses only stays, and the node does not open SSH to every address. The address the apply comes from is let in too. Delete the rule by hand to open SSH wider.
-
Root keeps a login with keys it had, such as the one the Rookery server in its container uses. The node never opens root to passwords.
-
A provisioner registered by hand for the node's server becomes the node's instead of a second one, with its settings as they were. That happens only when Rookery itself can tell: the server it names is on the node's address, the same name or a name that Rookery's server finds at the same address.
-
Rookery leaves a provisioner as it is when only the node says it's the same, and also one of another node. The node's word may be the provisioner's certificate, which is public. It may also be the provisioner's server among the aliases. The apply says so in a line and doesn't register a second one. If you know it reaches the node, make it the node's and apply again:
rookery provisioner run incus-vm adopt node=big rookery apply bigThis is also the way for the provisioner of the local Docker engine of a Rookery server in a container on the node.
Apply the node from the Rookery server that holds the provisioners registered for this machine, such as the server of the recipe. Those provisioners are that server's. A Rookery elsewhere doesn't see them, and registers new ones beside them.
Then check the node and its provisioners:
rookery run big status
rookery provisioner listThe Rookery server on a node
The rookery-server recipe installs Rookery's server in a container named rookery. When a node has that container, it offers the action Upgrade the Rookery server (upgrade-server). It runs the server from the image of the version of the Rookery that runs the action, or of the version it is given. The node's controller does it, and a job follows it:
-
The new container takes over what the running one has:
Taken over Such as Name and data rookery, and the folders or volumes it mountsPorts and networks --publish 127.0.0.1:7777:7777Restart policy --restart unless-stoppedGroups and extra hosts --group-add,--add-host incus-host:host-gatewayUser and environment The variables the image does not set A Docker socket mounted into it stays.
-
Logging, such as the
max-sizeandmax-fileadaemon.jsongives every container, is carried over too. -
An option it cannot carry over, such as
--privilegedor devices, stops it before anything changes. Upgrade such a container by hand. -
The old container is stopped and kept until the new one keeps running. If the new one stops, the old one runs again.
-
It refuses to go to an older version than the one running.
When the server upgrades its own container, it restarts during the job. The output stays in the node's session.
Run the server from this Rookery
With source=here, the node builds the server's image from the Rookery that runs the action, so a build of your own runs there without a release:
rookery run big upgrade-server source=here| Step | What happens |
|---|---|
| Check | Rookery asks the node for its CPU and checks that it is a static Linux program for it |
| Send | It sends its build context with the request: the Dockerfile CI builds the published image from, requirements.txt and the program itself |
| Build | The node's controller runs docker build -t rookery:VERSION - with the context on standard input. The node reads it as a stream and does not write it to disk |
| Swap | The container runs from rookery:VERSION, as for a release |
| Report | Rookery follows the node's session. When the server replaced is the one it talks to, it waits for the server to answer again and reads how the session ended |
The checks refuse a Rookery that cannot run in the image:
| Rookery | Use instead |
|---|---|
| A release for Linux | It qualifies |
| Rookery for macOS or Windows | Rookery for Linux of the same release |
| A build of your own with the system's C library | A static build: cargo build --release --target x86_64-unknown-linux-musl -p rookery |
| A build for another CPU | The build for the node's CPU |
The tag rookery:VERSION is local to the node, so a self-built image never passes for a published one. Its labels give its version, the source self-built and the repository of the published images. When an image of that tag already has the same program, the build is skipped. Once the container runs, the helper removes the unused self-built images, except the two newest.
The action needs the admin role. Its version is that of the Rookery sent.
Back to a release
source=release, the default, pulls the published image again. A server that runs a self-built image goes back to ghcr.io/aiviarium/rookery, the repository its image names:
rookery run big upgrade-server version=0.7.0It still refuses an older version than the one running.
The action needs the machine to be a node. Update a server that is not a node by running the rookery-server recipe again with rookery_server_update=true.
What Rookery does not do
- It never makes a storage pool on a disk, which would erase it. Create such a pool with
incus storage createand name it inrookery_incus_pool. - Taking a feature out does not remove Incus, Docker or their guests from the node. Only the provisioner goes.
- Docker opens the ports its containers publish by itself, past
ufw. Publish a port on127.0.0.1or on the tailnet address to keep it private. - It does not take a workstation to a node, or a node back to
managed.
Last updated on