Introducing Propagator: multi-everything deployment made easy
This post was written by Shlomi Noach.
Outbrain is happy to release its own Propagator as open source. The propagator is a schema & data deployment tool which makes it easy to deploy, review, audit & fix deployments to your database servers.
What does multi-everything mean? It is:
- Multi-server: push your schema & data changes to multiple instances in parallel
- Multi-role: different servers have different schemas
- Multi-environment: recognizes the differences between development, QA, build & production servers
- Multi-technology: supports MySQL, Hive (Cassandra on the TODO list)
- Multi-user: allows users authenticated and audited access
- Multi-planetary: TODO
With dozens of database servers in our company (and these are master database servers), from development machines to testing machines, through build machines to production servers, and with a growing team of over 70 engineers, we faced the growing problem of controlling our database schema evolution. Controlling creation of tables, columns, keys, foreign keys; controlling the creation of data that must be consistent across all servers became an infeasible task. Some changes were lost; some servers forgot along the way, and inconsistencies blocked our development & deployments again and again.
The are some good schema deployment tools around. Alas, they do not answer for the case of a large group of engineers working in continuous deployment. Existing tools require a linear execution path, or do not support partial deployments; or do not recognize the difference between environments; or require a carefully laid out XML input; or do not support user feedback…
Not all deployments are the same. Not all servers are the same. You don’t ALTER a big table on production. One may be using different schema names on different servers. One may have multiple schemas on a single server with identical structure. One may wish to only deploy to some development servers, possibly to a test server, but not to all, and yet be able to pick up on where you left a few days later on to complete your deployment. Some deployments fail, and for various reasons (e.g. John created that table manually on this particular test server, so obviously you can’t CREATE it again), and one wants to be able to skip it, or mark it as “OK”, or put some comments, or hint that you need assistance from a DBA. One wants to be able to quickly add new servers to your deployment group.
Above all, one wants every single deployment to be fully auditable. One wants to know exactly who did what and when. If their deployment failed, one wants to know that. One wants to know why it failed. One wants to be able to pick up and try it again after the DBA found the problem. One wants to be able to review yesterday’s deployments, and be able to contact Jane and say “hey, I see you hit a problem here; I know what the problem is; you should do this and that, then please try to deploy again”.
There’s so much more, but let’s leave that for the manual available.
This is how Propagator came to be. For the past three months, it has been in production use at our company. Our engineers used it to deploy hundreds of schema and data changes and provided with tons of feedback in the form of bug reports, feature requests and general hints on easy of use and expected behavior.
We are supporters and advocates of open source. We heavily rely on open source solutions and do our best to contribute back. The propagator is available on GitHub and is released under the Apache 2.0 license.
Alas, we won’t allow you a glimpse into our own internals, but why not enjoy some screenshots based on the PHPUnit tests?