Infrastructure as code with pyinfra¶

Veit Schiele

05 May 2025

6–7 minutes

Nowadays, there are many tools for automating, provisioning or configuring servers: Puppet, Chef, Salt, Ansible etc. They all suffer from ‘over-abstraction’ – each has its own syntax and debugging processes. This is a far cry from manual provisioning via SSH, where feedback is immediate. These tools seem to make deployment and configuration processes fantastically easy – until they start to get in the way. This is the reason pyinfra was created.

pyinfra tries to combine the good parts of pyinfra versucht, die guten Teile von Ansible and Fabric. However, unlike Ansible, it does not use YAML as the declaration language but Python. So with pyinfra, dealing with ‘if this, then that, else that’ has become much easier: since Ansible is purely declarative, such checks are difficult to implement; pyinfra, on the other hand, can be used imperatively and makes if-else queries a breeze.

In addition, pyinfra, like tentakel, can also execute one-off commands throughout the infrastructure.

Advantages¶

Declarative operations: pyinfra is executed in two phases, starting with a ‘dry run’ that identifies which pyinfra commands do not lead to any changes and can therefore be skipped.
Direct debugging: pyinfra immediately returns the output of the failed commands without having to go through an abstraction layer. This means that errors can be found much more quickly.
Erweiterbarkeit: In pyinfra, both the inventory and the operations are written in Python. pyinfra can therefore be easily extended with any Python packages.
Performance: pyinfra is up to ten times faster than Ansible, see also Performance.
Agentless: In contrast to Ansible, which requires Python, pyinfra requires nothing more than a POSIX shell.
Tool integration: The inventory can also be taken from Terraform, for example, and executed in Docker containers.

Examples¶

Client-side assets

Projects often need to precompile assets shortly before deployment, which are then uploaded to the remote host, for example in config.py:

config.py¶

from pyinfra import local, logger


logger.info("Run yarn install & build")
local.shell("yarn install")
local.shell("yarn run build")

Distribute data across multiple environments

For an application that is to be deployed in two environments, a good layout would be

├── deploy.py
├── group_data
│   ├── all.py
│   ├── production.py
│   └── staging.py
└── inventories
    ├── production.py
    └── staging.py

group_data/all.py contains all the information that both environments share:

group_data/all.py¶

env = "dev"
git_repo = "https://ce.cusy.io/cusy_app"

The differences are then described in group_data/staging.py and group_data/production.py:

group_data/production.py¶

env = "production"
git_branch = "main"

group_data/staging.py¶

env = "staging"
git_branch = "feature/42"

RDynamic inventories and data

pyinfra configures your application in Python. This can be used to generate files for the deployment of inventory and data. In the following example, we get the list of target hosts from an internal inventory API in the inventory.py file:

inventory.py¶

import httpx


def get_servers():
    db = []
    app = []

    servers = httpx.get("inventory.cusy.io/api/v1/app_servers").json()

    for server in servers:
        if server["group"] == "db":
            db.append(server["hostname"])

        elif server["group"] == "app":
            app.append(server["hostname"])

    return db, app


db_servers, app_servers = get_servers()

Now you can access this inventory from group_data/all.py:

group_data/all.py¶

from pyinfra import inventory

primary_db_server = inventory.db_servers[0].name

You can find more examples in the pyinfra-examples repository.

Connectors¶

Connectors enable the integration of pyinfra with other tools. Essentially, they can do the following two things:

The @ssh and @local connectors implement how commands are executed, whereas @terraform and @vagrant create inventory hosts and data. Finally, the @docker connector can do both.

You can find more connectors in the Connectors Index. And if you can’t find a suitable connector, you can also write your own: Writing Connectors.

Deployment pipelines¶

You can easily integrate pyinfra into deployment pipelines, for example by first establishing an ssh connection in a GitLab CI/CD pipeline and then connecting to pyinfra:

Configure your server to accept connections with the private key, for example you should include the public key in the file ~/.ssh/authorized_keys.
Now save the private key in a file variable https://ce.cusy.io/ci/variables/CUSTOM_ENVIRONMENT_VARIABLE_OF_TYPE_FILE.

Then define a deploy job:

.gitlab-ci.yml¶

deploy:
  image: ghcr.io/astral-sh/uv:debian
  variables:
    SERVER: map.cusy.io
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  needs:
    - test
  script:
    - echo "Set up ssh"
    - mkdir -p ~/.ssh
    - echo "Check if SSH_PRIVATE_KEY is set and available as file"
    - if [ -f $SSH_PRIVATE_KEY ]; then echo "[SUCCESS] SSH_PRIVATE_KEY is set and exists"; else exit -1; fi
    - echo "Ensure minimal permissions on ssh key"
    - chmod 400 $SSH_PRIVATE_KEY
    - echo "Add remote server fingerprint to known_hots"
    - ssh-keyscan -t ed25519 -H $SERVER >> ~/.ssh/known_hosts
    - echo "Test the connection to the server using ssh"
    - ssh -i $SSH_PRIVATE_KEY cusy-map@$SERVER echo "[SUCCESS]"
    - echo "Run the deploy script using pyinfra"
    - uvx pyinfra -y -vvv @ssh/$SERVER --data ssh_key=$SSH_PRIVATE_KEY --data ssh_user=cusy-map ./ci/deploy.nightly.py

Help and support¶

If you would like to learn more about pyinfra in a workshop or if you need help with your pyinfra project, you are welcome to contact us:

Veit Schiele

Mail

Phone