Self-healing servers with event-driven automation
Restart a stopped service in seconds, warn before a disk fills, configure new machines the moment they join. How event-driven automation works with Salt's beacons and reactors, compared with Event-Driven Ansible and cron-style tools.
"Self-healing" usually means something modest and valuable: when a known thing breaks, fix it before a person has to. A service stops, so start it. A disk passes 90 percent, so warn someone while there is still time. A new machine appears, so give it its configuration. The question is how fast, and how safely.
Three ways to react#
| Approach | How it notices | How fast |
|---|---|---|
| Scheduled runs (cron, Puppet-style agent runs, nightly playbooks) | It doesn't; it re-applies config on a timer | Minutes to hours |
| Event-Driven Ansible | Rulebooks listen to outside sources (monitoring alerts, webhooks, queues), then run playbooks that connect into the machine | Seconds after the alert, plus the connection |
| Salt beacons and reactors | The agent on the machine watches it and reports events over its open connection; the master reacts | Seconds |
Event-Driven Ansible is a real, supported product from Red Hat and a good fit if your monitoring already produces the events. Salt's model differs in one useful way: the machine is the sensor. You do not need a monitoring system to notice a stopped service first, and you do not need an inbound connection to fix it.
Doing it on Salt without writing YAML#
Salt's beacons and reactors can do a lot, and they are fiddly to set up: beacon configuration in pillar, reactor files in Jinja, and the security question of trusting data that a minion sent. Gardience turns the common cases into event rules you set in the console, through the API, or with Terraform:
- Service rules: if a watched service stops on Windows or Linux machines, restart it on that machine within seconds.
- Disk rules: if a disk passes a fill level (for example
C:at 90 percent), record it in the audit log and send a notification, at most once an hour per machine and disk. - Join rules: when a machine joins the fleet for the first time, apply a saved state to it at once. Pairs with first-boot enrollment for machines built by Terraform or cloud templates.
Every firing is recorded in the tamper-evident audit log. Events are checked before anything acts on them: an event is trusted only if it comes from the machine it names, names only plain values, and refers to a service an admin chose to watch.
Keep it safe#
Automatic fixes are changes, so the same rules should apply as for people:
- Narrow scope. Restarting a named service is safe to automate; "run any script the event mentions" is not.
- Locks win. A machine under maintenance should not be "healed" back.
- Rate limits. A flapping service or a full disk should not flood the log or the pager.
- Record everything. If a fix fires a thousand times a day, you want to see it and fix the cause.
Questions#
What is the difference between Event-Driven Ansible and Salt reactors?#
Event-Driven Ansible listens to outside event sources and then runs playbooks that connect into machines. Salt beacons run on the machine itself and report events over the agent's existing connection, so the master can react without monitoring in between or an inbound connection.
Can Windows services be restarted automatically?#
Yes. Service rules work on Windows and on Linux machines that use systemd.
Will a self-healing rule fight my maintenance work?#
Not in Gardience: locked machines refuse automatic restarts, and join rules skip them.
Try Gardience on your own machines
One supported console for Windows and Linux on open-source Salt: patching, a scored STIG/CIS-aligned baseline, drift repair, event-driven automation, two-person approval and a tamper-evident audit log. Flat yearly pricing, not per node.
Free 90-day trial (25 nodes)See it work Compare