Kolla-Ansible is an official OpenStack project and a production-recommended deployment method.
Ubuntu 24.04 · 3 Controllers + 2 Compute + 1 Bastion · External Ceph · Swift via Ceph RadosGW · Masakari
Nodes: ct0, ct1, ct2 (control + network + monitoring + storage-client), comp0, comp1 (compute), base (bastion/deployment host). DNS domain: kdinesh.com. Values below (IPs, FQDNs) are this environment’s own — replace them with yours if you’re following along.
Core services this deploys: Keystone (identity), Glance (images), Nova (compute), Neutron (networking), Cinder (block storage), Horizon (dashboard), Heat (orchestration), Swift via RGW (object storage), Masakari (instance HA).
Interfaces
- ens33 — management/API network, all nodes including bastion. Network 10.10.40.0/24, gateway 10.10.40.1, DNS 172.16.16.100, search domain kdinesh.com.
- ens35 — Neutron external/floating-IP provider network (physnet1), no IP, all 5 control/compute nodes.
Kolla VIPs (unused IPs on 10.10.40.0/24): internal 10.10.40.10, external 10.10.40.11. Both sit on the same subnet in this environment; for production, official guidance is to keep the internal/management VIP on an isolated network not reachable from outside, and put the external VIP on its own public-facing interface/subnet — see the Kolla-Ansible production architecture guide.
External Ceph in this environment is reachable at 172.16.16.130 — that’s my env’s cluster, a single reachable endpoint for a lab/test build. For production, point this at a proper multi-node Ceph cluster (3+ mons, multiple OSD hosts), not a single node.
For production, use NIC bonding (active-backup/LACP) on ens33/ens35 to avoid a single point of failure at the NIC level — this environment runs single NIC per role, no bonding.
1. IP addressing & DNS
Assign static IPs on ens33 via netplan on all 6 nodes (ct0–ct2, comp0–comp1, base); ens35 stays unconfigured, used directly by Neutron. On Ubuntu 24.04 cloud images, disable cloud-init’s network management first (/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg → network: {config: disabled}, then remove /etc/netplan/50-cloud-init.yaml), or DHCP reasserts itself on reboot.
Host | FQDN | ens33 IP |
|---|---|---|
base | base.kdinesh.com | 10.10.40.3 |
ct0 | ct0.kdinesh.com | 10.10.40.5 |
ct1 | ct1.kdinesh.com | 10.10.40.6 |
ct2 | ct2.kdinesh.com | 10.10.40.7 |
comp0 | comp0.kdinesh.com | 10.10.40.8 |
comp1 | comp1.kdinesh.com | 10.10.40.9 |
DNS — add these A records to your zone (served from 172.16.16.100):
Record | IP | Purpose |
|---|---|---|
base.kdinesh.com | 10.10.40.3 | bastion/deployment host |
ct0.kdinesh.com | 10.10.40.5 | controller |
ct1.kdinesh.com | 10.10.40.6 | controller |
ct2.kdinesh.com | 10.10.40.7 | controller |
comp0.kdinesh.com | 10.10.40.8 | compute |
comp1.kdinesh.com | 10.10.40.9 | compute |
internal.kdinesh.com | 10.10.40.10 | kolla_internal_fqdn |
openstack.kdinesh.com | 10.10.40.11 | kolla_external_fqdn |
Confirm resolution from any node: nslookup ct0.kdinesh.com, nslookup internal.kdinesh.com, nslookup openstack.kdinesh.com.
2. Base host prep (all 6 nodes, including bastion)
$ sudo apt update && sudo apt -y upgrade
$ sudo apt -y install chrony # configure NTP
$ sudo systemctl enable –now chrony
From the bastion (base, 10.10.40.3 — this is the deployment host everything below runs from):
$ ssh-keygen -t ed25519
$ ssh-copy-id <user>@ct0
$ ssh-copy-id <user>@ct1
$ ssh-copy-id <user>@ct2
$ ssh-copy-id <user>@comp0
$ ssh-copy-id <user>@comp1
3. Install Kolla-Ansible (on the bastion)
$ sudo apt install -y git python3-dev libffi-dev gcc libssl-dev libdbus-glib-1-dev python3-venv
$ python3 -m venv /opt/kolla-venv
$ source /opt/kolla-venv/bin/activate
$ pip install -U pip
$ pip install git+https://opendev.org/openstack/kolla-ansible@stable/2026.1
$ sudo mkdir -p /etc/kolla
$ sudo chown $USER:$USER /etc/kolla
$ cp -r /opt/kolla-venv/share/kolla-ansible/etc_examples/kolla/* /etc/kolla
$ cp /opt/kolla-venv/share/kolla-ansible/ansible/inventory/multinode ./
$ kolla-ansible install-deps
4. Inventory — ./multinode
You can refer to the link below for the sample inventory file used in my environment.
5. /etc/kolla/globals.yml
You can download the sample globals.yml file used in my environment from the link below –
# — core —
kolla_base_distro: “ubuntu”
openstack_release: “2026.1”
network_interface: “ens33”
neutron_external_interface: “ens35”
# Needed on the compute nodes too, not just controllers — without this,
# Kolla never builds br-ex/bridge-mappings on the compute nodes and
# provider-network port binding fails there (PortBindingFailed in nova-compute)
enable_neutron_provider_networks: “yes”
kolla_internal_vip_address: “10.10.40.10” # replace: your internal VIP — keep on an isolated/internal-only network for prod
# External VIP: same subnet as internal here (lab setup) — for prod, use a separate public-facing network
kolla_external_vip_interface: “ens33”
kolla_external_vip_address: “10.10.40.11” # replace: your external VIP
kolla_enable_tls_external: “yes”
kolla_external_fqdn: “openstack.kdinesh.com” # replace: your external FQDN
kolla_internal_fqdn: “internal.kdinesh.com” # replace: your internal FQDN
enable_cinder: “yes”
enable_heat: “yes”
enable_horizon: “yes”
# — external Ceph: Glance/Cinder/Nova backend —
enable_ceph: “no” # Ceph is external, Kolla does not deploy it
glance_backend_ceph: “yes”
cinder_backend_ceph: “yes”
nova_backend_ceph: “yes”
enable_valkey: “yes” # required: Cinder needs a coordination/lock backend when a Ceph (RBD) backend is enabled
cinder_cluster_name: “cinderha” # required: ct0/ct1/ct2 are all in [storage], so cinder-volume runs on all 3
# — Swift via Ceph RadosGW (external) —
enable_ceph_rgw: true
ceph_rgw_keystone_user: “ceph_rgw”
ceph_rgw_external_fqdn: “<public-fqdn-for-rgw>” # replace: your RGW external FQDN
ceph_rgw_internal_fqdn: “<internal-fqdn-for-rgw>” # replace: your RGW internal FQDN
ceph_rgw_port: 8080
ceph_rgw_swift_compatibility: true
ceph_rgw_swift_account_in_url: false # must match your RGW’s own rgw_swift_account_in_url
ceph_rgw_hosts:
– host: ceph-rgw
ip: 172.16.16.130 # replace: your Ceph RGW node IP
port: 8080
# — Masakari (VM/instance HA) —
enable_masakari: “yes”
enable_masakari_instancemonitor: “yes”
enable_masakari_hostmonitor: “yes”
# enable_hacluster switches on automatically when enable_masakari_hostmonitor is “yes”
6. Passwords
$ kolla-genpwd
Note ceph_rgw_keystone_password in the generated /etc/kolla/passwords.yml — you’ll need this exact value on the Ceph side in Step 7 so RGW’s Keystone auth matches what Kolla creates.
7. External Ceph setup
Ceph is already deployed — I’m just configuring the required pools and permissions here. If you want to test this, you can refer to my old blog on deploying a single-node Ceph cluster.
Note: a single-node Ceph cluster is not recommended for production.
7.1 Pools and client permissions (on the Ceph cluster)
$ ceph osd pool create images
$ ceph osd pool create volumes
$ ceph osd pool create backups
$ ceph osd pool create vms
$ rbd pool init images
$ rbd pool init volumes
$ rbd pool init backups
$ rbd pool init vms
$ ceph auth get-or-create client.glance \
mon ‘profile rbd’ \
osd ‘profile rbd pool=images’ \
mgr ‘profile rbd pool=images’
$ ceph auth get-or-create client.cinder \
mon ‘profile rbd’ \
osd ‘profile rbd pool=volumes, profile rbd pool=vms, profile rbd-read-only pool=images’ \
mgr ‘profile rbd pool=volumes, profile rbd pool=vms’
$ ceph auth get-or-create client.cinder-backup \
mon ‘profile rbd’ \
osd ‘profile rbd pool=backups’ \
mgr ‘profile rbd pool=backups’
Nova ephemeral disks reuse client.cinder — no separate client.nova key needed.
7.2 aes256k cephx key fix
Newer Ceph builds can issue keys in the aes256k format, which Kolla’s client libraries on Ubuntu 24.04 don’t handle — auth looks fine, writes fail silently . Check and enable the legacy cipher if it’s not already allowed, then rotate keys:
$ ceph config get mon auth_allowed_ciphers
$ ceph config set mon auth_allowed_ciphers aes,aes256k
$ umask 077
$ ceph auth rotate –key-type=aes client.glance > /root/ceph.client.glance.keyring
$ ceph auth rotate –key-type=aes client.cinder > /root/ceph.client.cinder.keyring
$ ceph auth rotate –key-type=aes client.cinder-backup > /root/ceph.client.cinder-backup.keyring
$ ceph auth rotate –key-type=aes client.rgw.ceph-rgw > /root/ceph.client.rgw.ceph-rgw.keyring
Use these rotated keyrings, not the originals from 7.1. Make sure there’s no extra blank/whitespace lines in any keyring or ceph.conf file you copy over — Kolla’s INI parser breaks on stray whitespace.
7.3 RGW — Keystone integration for Swift
Kolla creates the swift service/endpoint and ceph_rgw Keystone user automatically during deploy (Step 8), based on what you set in Steps 5/6 — point RGW at that same Keystone with matching credentials:
$ ceph config set client.rgw.ceph-rgw rgw_keystone_api_version 3
$ ceph config set client.rgw.ceph-rgw rgw_keystone_url “http://internal.kdinesh.com:5000” # replace: your internal Keystone URL
$ ceph config set client.rgw.ceph-rgw rgw_keystone_admin_user ceph_rgw
$ ceph config set client.rgw.ceph-rgw rgw_keystone_admin_password “<ceph_rgw_keystone_password from passwords.yml>” # replace
$ ceph config set client.rgw.ceph-rgw rgw_keystone_admin_domain default
$ ceph config set client.rgw.ceph-rgw rgw_keystone_admin_project service
$ ceph config set client.rgw.ceph-rgw rgw_keystone_accepted_roles “member,admin”
$ ceph config set client.rgw.ceph-rgw rgw_keystone_token_cache_size 500
$ ceph config set client.rgw.ceph-rgw rgw_keystone_implicit_tenants true
$ ceph config set client.rgw.ceph-rgw rgw_s3_auth_use_keystone true
$ ceph orch restart rgw.ceph-rgw
The ceph_rgw Keystone user needs the admin and service roles — Kolla assigns these when it creates the service user.
7.4 ceph.conf and keyrings
Ceph already has /etc/ceph/ceph.conf on the admin/mon node — copy it as-is:
$ scp <ceph-admin-node>:/etc/ceph/ceph.conf /etc/kolla/config/glance/ceph.conf
cp /etc/kolla/config/glance/ceph.conf /etc/kolla/config/cinder/ceph.conf
cp /etc/kolla/config/glance/ceph.conf /etc/kolla/config/nova/ceph.conf
Then the rotated keyrings from 7.2:
copy key rings from ceph to bastion/deployment node
$ cp ceph.client.glance.keyring /etc/kolla/config/glance/
$ cp ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-volume/
$ cp ceph.client.cinder.keyring /etc/kolla/config/nova/
$ cp ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-backup/
$ cp ceph.client.cinder-backup.keyring /etc/kolla/config/cinder/cinder-backup/
8. Deploy
$ kolla-ansible bootstrap-servers -i ./multinode
$ kolla-ansible prechecks -i ./multinode
$ kolla-ansible pull -i ./multinode # pulls images from the public registry on all 5 nodes
$ kolla-ansible deploy -i ./multinode
$ kolla-ansible validate-config -i ./multinode
No local registry — images pull directly from the public Kolla registry. Make sure all 5 nodes have outbound internet on TCP/443.
9. Post-deploy / CLI
$ pip install python-openstackclient -c https://releases.openstack.org/constraints/upper/2026.1
$ kolla-ansible post-deploy -i ./multinode
$ mkdir -p ~/.config/openstack
$ cp /etc/kolla/clouds.yaml ~/.config/openstack/clouds.yaml
10. Verification
$ openstack endpoint list
$ openstack compute service list
$ openstack network agent list
$ openstack volume service list
$ openstack container list # exercises Swift API via Ceph RadosGW / Keystone
You can log in to the UI and check the health status and all functionalities.
You can deploy a test VM and check that it works.
To test control-plane HA, shut down the node currently holding the VIP and check whether the VIP moves to a new node and the OpenStack UI and other services keep working.
You can also test instance HA: deploy a test VM, check which host it’s running on, then shut down that host and see whether the VM gets recreated on another host. By default, Masakari waits 3 minutes before recreating the VM on another host — you can update this wait time in globals.yml to reduce it.
Next, I’ll post how to enable the Octavia LB service.
Great work! 👍 Very useful and nicely explained.