Disclosure: PyInfra core contributor here.
We just shipped 3.8.0.
PyInfra is an agentless infrastructure automation tool. Same job description as Ansible, Salt, Chef. SSH into hosts, describe desired state, it diffs and converges. No agent, no central server, no daemon.
The difference: your "playbook" is just Python. Not Python cosplaying as YAML. Not Jinja smuggled inside YAML inside a Helm chart inside a Kustomize overlay. Actual Python:
from pyinfra.operations import apt, files, server
apt.packages(packages=["nginx"], update=True)
files.template(src="nginx.conf.j2", dest="/etc/nginx/nginx.conf")
server.service(service="nginx", running=True, enabled=True)
About YAML. Wonderful format. For about eleven minutes. Then someone needs an `if`, and you have `{% if %}` inside a string inside a list inside a map. Then someone types `no` as a country code for Norway and it ships to prod as `False`. Then someone indents with a tab and the parser dies without saying where. Congratulations, you reinvented a programming language. Badly. The honest move is to admit you wanted code, then write code.
PyInfra skips the eleven good minutes and goes straight to code.
Release notes in the link. Happy to answer questions.
Infrastructure as Code, not infrastructure as YAML.
> The honest move is to admit you wanted code, then write code
This war will never end ... because there are genuine tradeoffs on both sides. YAML being a bad data description format isn't actually central to the question of whether you describe infra as data or as code. You can use JSON if you want. Data is static, 100% predicatable. Code is non-deterministic right up to the halting problem. If your infra should look different on wednesday to thursday, well it can do that! Some people like it, some people think it's the definition of hell.
Terraform makes an interesting tradeoff to try and have the best of both worlds but ultimately still falls on the same issue ... I've not seen one project yet of any complexity that didn't use workarounds to implement optional components (let's just pretend there's a list of them and it has 1 or zero elements in it!).
Ultimately I agree with your philosophy but maybe not your language. IMHO You really want a language that is built from the ground up around static typing and immutable constructs for this. Get as close to that predictable determinism as possible. But then, if the whole world knows python, I guess python it is.
> Infrastructure as Code, not infrastructure as YAML.
Right on.
It's amazing to me that we've spent decades with programming languages and environments which can accurately guess what you're about to type next, which have enormous expressiveness while maintaining cogency, which are intuitive and well understood by humans, which have endless libraries and an infinity of ways of connecting with the world.
And what do we use to configure the most sophisticated infrastructure to run such code? Yet another mark-up language!
Many domains are better served by a more limited programming language, so you can analyze a program and/or make guarantees about it.
Real regexes (actually regular…) are infinitely better than Python code matching the same string (if they are sufficient) - you can compute their intersection, union, complement; check if they can match anything at all (and generate an example automaticallly).
For software builds, Bazel and others use Starlark, which is a restricted Python subset, so builds can be guaranteed finite and can be reasoned about.
Ansible may or may not offer any benefits in return for the limits (I am not an ansible guru), but in general, most tasks do not need a Turing complete configuration/specification language - and it is then better to NOT have Turing completeness.
The "you don't want a full programming language" trope I see repeated a lot but I think far more people end up wishing for a Turing complete language than wishing it _wasn't_ Turing complete.
They do, until a configuration endless loop brings down their production system.
This is not really different than C vs Rust, or even Perl regular expressions (unbounded execution time) vs real regular expression. With great powers comes great abilities to shoot yourself in the foot.
The power/guarantee balance is delicate, and you can’t hold the stick at both ends. People will always complain.
The number of times I've seen a configuration endless loop bring down anything are so few compared to the time wasted on DSLs and having to bend over backwards to do things a first-class programming language can do simply. Same with PCRE I've seen that maybe.. once.
The environment around the language can put in limits (on time, number of operations, etc.)
Convex does this well, replacing SQL (somewhat yaml-like sucky old declarative language) with JS/TS but in a well-locked-down environment with limits to ensure one mutation or query doesn’t take down the whole DB.
> It's amazing to me that we've spent decades with programming languages and environments which can accurately guess what you're about to type next, which have enormous expressiveness
You've almost guessed the problem. Too much expressiveness is a bad thing. This is a problem I encounter a lot more often then I'd be happy to. It's very often is much easier to build something more generic than what the user actually needs, and then testing it becomes a nightmare.
To make this more concrete, here's a case I'm working on right now. Our company provides customers with a tool to manage large amounts of compute resources (in HPC domain). It's possible to run the product on-prem, or in different clouds, or a combination of both. Typically, the management component comes with a PXE boot and unfolds from there. A customer wanted integration with a particular cloud provider that doesn't support this management style, nor can it provide a spare disk to be used for management, nor any other way our management component was prepared to boot.
The solution was to use netboot that would pre-partition the disk and use the first N partitions to store the management component as well as the boot, ESP / bios_grub partition etc. It had to be incorporated into the existing solution that encompasses partitioning and mounting all the resources available to a VM, including managing RAIDs, LVM, DM and so on.
The developers implemented it as a GPT partition name with a pre-defined value that would instruct our code to ignore the partitions found prior to the "special" partition and allow the user to carry on as usual, pretending that the first fraction of the disk simply didn't exist (used by netboot + the management component).
This solved the immediate problem for the user who wanted this ability, but created thousands of problems for QA: what happens if there's a RAID that uses the "hidden" partitions? What happens if the user accidentally creates second /boot partition? What happens if the user wants whole-disk encryption? And so on. It would've been so much better if these questions didn't exist in the first place, than to try to answer them, given the "simple" solution the developers came up with.
If you programmed for just a year, I'm sure you've been in this situation at least a few times already. This is exceedingly common.
* * *
There's an enormous value to being able to restrict the possible ways a program can run. Most GUI projects? -- They don't need infinite loops! It just makes programs unnecessarily hard to verify. But it's "easy" to have a single loop language element that can be made infinite if necessary. Configuration languages exclude whole classes of errors simply by making them impossible to express.
However, I have to agree that, specifically, YAML is a piss-poor configuration language. It has way too many problems that overshadow the benefits it offers. We, collectively, decided to use it because everyone else decided to use it, making it popular... and languages are "natural monopolies". So, one could certainly do better ditching YAML, if they can afford to go unpopular. But ditching the idea of a configuration language is throwing the baby out with the bathwater.
The problem is that it is actually not just Python, branched with “normal if statements”: https://docs.pyinfra.com/en/3.x/deploy-process.html#checking...
You can write `if CHECK: do something`. There's nothing preventing that.
I've been down this path, implemented my own version of PyInfra many times over the years. I've used Ansible and my own implementations in anger. The _if param is far far far from the worst offender and it's a natural addition, especially when you are laying out a bunch of unrelated checks into something that looks more like a table.
This! Been trying to find the best (least worst) solution to this since 2015 when I started pyinfra. Done ast parsing/hacking, done weird context managers instead, tried rewriting statements to context managers. _if is the latest, and I think least worst, option right now.
Basically a flaw of the entire model where you write code as if executing a single host which is then executed on many in parallel, forcing the two step diff and deploy that causes this.
Funny thing is since v3 this behavior (diff then execute) is even desired with the yes prompt like terraform.
I don't understand why not use function chaining, Javascript style. Make the order of execution explicit through scope.
Thank you for this. I've implemented my own version of this a couple times over the previous 25 years. This is how my code always looked.
I've used Salt, CFEngine, Chef, Puppet, Make, Bash, and many hand-rolled iterations of this approach. I finally threw in the towel and forced myself to come to terms with Ansible and it's quirks because I needed the wider community support.
Now with AI tooling, I'm not so convinced the community modules moat is an actual moat. I'm going to very seriously consider porting all my Ansible code to this and see how it feels. I anticipate I'll be much happier after the change.
Do you have any plans to integrate with/build on other communities modules? i.e. even if it's not perfect, being able to call Ansible or Salt modules from PyInfra would be one way to fill the gap.
As a heads-up, your comments here were flagged. I think some people must have thought your (current) writing style rather LLM-ish.
It obviously was LLM assisted, but I think collectively we will have to get over our distaste for text that has some LLM’isms in spots as long as it isn’t obviously completely outsourced to a bot, unless we just want to shut down message boards completely.
English is not my first language, so I lean on an LLM to clean up the frenchisms, but the ideas are mine :-)
I would very much rather read frenchisms than LLMisms. I regard translated idioms from other languages as an actual positive, they lend the text additional charm and interest because they give me a fresh new look at how a thought can be expressed. LLMisms just sound like someone tossed a bunch of marketing material into a blender.
I'd be curious how unintelligible the original post was, probably a lot more intelligible than you might think assuming it wouldn't be in literal French? It's kind of sad you (the royal you -- is that a englishism?) feel the need to clean up the frenchisms Just like you meet people in Mali wish American English accents because they learn it from movies, we're all going to end up writing some smoothed over glob of LLMnglish.
I've used Pyinfra for a few years, great software.
Frenchisms are great! “No A, No B, No C. Just D” in every comment “assisted” by AI isn’t that great :(
What was the obvious part in it? I still can't tell.
A sentence. Another sentence. Not a subordinate. Just single verb sentences.
But tbh I didn't see it much on OP's comment.
Ah shit...
I should try this myself, but I also rolled my own https://github.com/mattbillenstein/salty
In the spirit of Saltstack with full python throughout including Mako templating. It has a very simple set of operators mostly around idempotent file management and shell commands to do things like restart services.
This enables very fast deploys - small changes on a small number of machines in < 10 seconds.
Yeah, but I have Claude Code or Codex do this Ansible stuff and they do just fine with all this and then there's a gazillion of examples that they can lean on and once the patterns are established, it's pretty smooth. Opus 4.5 was when the big inflection was I was heavy into automation all summer. It was Opus 4.0. It was like pulling teeth. And then when 4.5 came out, it was just beautiful.
"I only use tools that my LLM knows how to use" is not the flex that you think it is
Yeah, you’re gonna eat your words when you do something that’s not “install this package” and “create this user”.
For anything dynamic and sufficiently complicated, ansible is horrible. Pyinfra is much better.