Host agent
Probes check your services from the outside. The agent checks the machine itself and shows what only the inside reveals: a full disk, a dead process, or a host that is up but no longer doing its job. It measures and reports, and it executes nothing, so you can run it on production hosts without reviewing what it might be told to do.
What it measures
Section titled “What it measures”Every 15 seconds, on the host:
| Signal | Detail |
|---|---|
| CPU | Total utilization, 0 to 100 |
| Load average | 1, 5 and 15 minutes (Unix only) |
| Memory and swap | Used, total, percentage |
| Disk | Every mounted filesystem, used and total and percentage |
| Uptime | Seconds since boot |
| Services | For each watched process: running, process count, CPU, resident memory |
The table is the complete list: the agent collects these signals about the host and nothing else.
Install
Section titled “Install”Create an enrollment token in the app under Host agents, then New agent. It hands you a ready-made command:
PERSTAT_TOKEN=psag_… sh -c "$(curl -fsSL https://agent.perstat.io/install.sh)"The environment assignment keeps the token out of ordinary process arguments and ps output. A
literal command pasted into an interactive shell can still land in shell history. Inject the
token from your secret manager or through hidden input, and unset the variable afterwards.
The agent runs on Linux (x86_64 and aarch64, static musl builds without runtime dependencies) and on macOS as a universal binary. A Windows agent is planned.
The installer detects OS and architecture, downloads the matching binary, and checks SHA-256. With an Ed25519-capable OpenSSL 3, it also verifies the release signature against the key in the script. On systems without that capability, such as common macOS LibreSSL builds and older distributions, it warns and verifies through TLS transport plus SHA-256. Installer and key come from the same origin, so the first install trusts that origin. Every later self-update verifies against a key compiled into the agent and is fail-closed, as described below.
On Linux the installer sets up a systemd unit, on macOS a LaunchDaemon. Set
PERSTAT_NO_SERVICE=1 to get the binary without a service.
Re-running is safe. If the host is already enrolled, the installer switches to upgrade mode: it refreshes the binary and the service, keeps the existing identity, and needs no token. Running the same line from your configuration management on every host, repeatedly, is the intended use. For more than a handful of machines, see agents on a fleet.
Enrollment tokens
Section titled “Enrollment tokens”| Property | Default | Range |
|---|---|---|
| Lifetime | 24 hours | 1 to 720 hours |
| Single use | yes | can be made reusable |
The token is shown once and stored as a hash. Single use suits one machine, and a reusable token suits a fleet roll-out. A token stays valid until it expires, so choose a short lifetime.
The token itself is opaque. Which organization and project it belongs to is resolved on the server, never parsed by the agent.
Monitors you can build on it
Section titled “Monitors you can build on it”An agent reports, and a monitor decides what is worth an incident. Create monitors of type
agent and bind them to the agent. One monitor watches one signal:
metric |
Fields | Fires when |
|---|---|---|
availability |
grace_seconds (default 180, range 60 to 3600) |
The host stopped reporting for longer than the grace |
cpu |
threshold (default 90) |
CPU above the threshold |
mem |
threshold (default 90) |
Memory above the threshold |
disk |
threshold (default 90) |
The fullest filesystem above the threshold |
service |
service_name, grace_seconds (default 120, range 60 to 3600) |
The process has been absent for longer than the grace |
Two settings apply to all of them. breach_severity is down (opens an incident, pages
on-call, shows on status pages) or degraded (an in-app notice). notify_push toggles the push
notification.
One disk monitor covers every mount point, because the threshold applies to the fullest filesystem. It watches filesystems filling up: a log file that adds 700 MB to a 25 GB disk moves the figure by three points, so a 90 percent threshold stays quiet. To alert on a single growing file or directory, use a heartbeat monitor.
Perstat evaluates agent monitors about every 30 seconds. They report directly, without check regions or quorum, so a single breaching evaluation opens the incident once the grace has passed.
Watching services
Section titled “Watching services”service_name is a process name, not a systemd unit. The agent scans the process table and
matches three ways: the exact process name, the exact basename of the executable (which is what
makes postgres and postmaster both work), and the Linux 15-character name truncation.
Matching is case sensitive on Linux.
Good to know before you configure it: a service that starts as a script shows up in the
process table as python3, sh, or node, so two such services share a name. Compiled daemons
and anything that sets its own process name are identified precisely.
To see the real names, either run ps -eo comm= | sort -u on the host, or switch on the
process inventory for that agent. It is opt-in, off by default, and transmits process and
executable names only, deduplicated and capped, never command lines, paths, users, or PIDs. With
it on, the monitor form suggests names instead of asking you to type them.
The agent scans the process table only when at least one service monitor exists for it.
Network and firewall
Section titled “Network and firewall”Outbound only. The agent never opens a listening socket. There is no inbound rule to write and no port to expose, and the systemd unit restricts address families and system calls to match.
Two destinations over HTTPS on port 443:
| Host | Purpose |
|---|---|
api.perstat.io |
Enrollment, measurements, configuration polling |
agent.perstat.io |
Installer, binaries, self-update downloads |
Both need to resolve. On a machine that runs an authoritative nameserver rather than a resolver, make sure a usable resolver is configured.
Timing
Section titled “Timing”Measurements every 15 seconds, sent in a batch every 60 seconds, configuration polled about every 5 minutes. The first measurement is sent immediately after enrollment, so a new host appears within seconds.
When a push fails, measurements queue in memory (about six hours’ worth) and are sent when the connection returns, so a short outage leaves your history complete. The queue lives in memory and starts fresh when the agent restarts.
One consequence of the 5-minute configuration poll: after you add a service monitor, it can take a few minutes before the agent is watching that process. Until real data arrives, the monitor stays pending rather than firing, so a newly created monitor never raises a false alarm.
Self-update
Section titled “Self-update”The agent keeps itself current. Releases are ed25519-signed, the signing key stays offline, and the agent verifies the signature against a key compiled into its own binary before swapping anything. It is fail-closed: an unverifiable release is not installed, and a lower version never qualifies. The swap happens inside the agent’s own directory and needs no root helper.
Set DATARGO_AGENT_NO_SELFUPDATE=1 in the service environment if you would rather drive
versions yourself.
Removing an agent
Section titled “Removing an agent”Remove the identity with the trash icon on the agent. This revokes its token, archives the monitors attached to that agent, and resolves their open incidents. Archived monitors free their plan slots while their historical records remain subject to the applicable retention rules.
Remove the software on the host:
curl -fsSL https://agent.perstat.io/uninstall.sh | sudo shIt stops and removes the service, deletes the binary, state, and service user, and deregisters the host. If the machine is already gone, removing the identity in the cockpit is sufficient.
When an agent goes quiet
Section titled “When an agent goes quiet”Worth understanding precisely, because it decides what you see.
The availability monitor fires: silence past its grace period opens an incident. This is the monitor that tells you a host is gone, and it is the reason to give every agent one.
The other monitors pause their evaluation once the last report is older than 180 seconds. They write no checks, so they neither open nor resolve anything, which is why a dead host produces one incident rather than one per metric. Those tiles keep displaying their last reading, usually green, so read the availability monitor when a host is silent.
To tell a dead host from a healthy one that cannot reach Perstat, pair the availability monitor with an external check against the same machine. The agent’s signal is the arrival of its report, so both cases look alike from the agent’s side.
Data minimalism
Section titled “Data minimalism”The agent never collects command lines, environment variables, or usernames. Process inventory is opt-in and limited to names. The agent measures the host and reports, and its whole scope fits in the table under What it measures.
It also executes nothing. There is no remote command channel, no script hook, and no way for anyone at Perstat to run something on your machine.
Good to know
Section titled “Good to know”- Service monitors check processes. The check answers “is a process of this name alive”. A failed unit with a lingering process, a wedged process, or restart flapping inside the grace window therefore reads as healthy. For critical services, add an external check or a heartbeat that tests what the service delivers.
- Custom measurements run through heartbeats. To alert on a number only the host knows, use
a heartbeat monitor: each one has a healthy endpoint (
/ping/<token>) and a failure endpoint (/ping/<token>/fail), and a timer on the host decides which to call. The failure endpoint breaches at the next evaluation rather than waiting out the period. That covers what a percentage threshold cannot, such as a count that dropped or a directory that grew. - One monitor per signal. Watching four processes on a host means four monitors.
- Agents on a fleet: roll the agent out to many machines at once.
- Rolling out host agents: the app, the REST API, and the timings.