Credentials

MetricHarbor stores the name of a credential and the shape of it. It never stores the secret, and there is no field in the API in which to put one — offer it a password and it refuses the request.

The secret lives on your agent, encrypted, on your machine. A full dump of MetricHarbor's database is not a route into your infrastructure.

This is a real security property and it costs you something: you declare a credential in the web interface and then supply it on the host. The three mechanisms below exist to make that bearable.

The kinds#

The split is the point: everything in the middle column is configuration, so MetricHarbor knows it and can show it to you. Everything in the right-hand column is a secret, so it only ever exists on your host.

1. Declare it, then run one command#

In Credentials → Declare credential, pick a kind and give it a name you will be willing to retype — ssh-monitor, postgres-monitor. The detail page then shows the exact command, filled in:

runuser -u metricharbor -- metricharbor-agent cred add postgres-monitor --type database_password

Running these commands#

runuser is used here rather than sudo deliberately. It comes from util-linux, which is essential on every distribution these packages target; sudo is not installed by default on a minimal Debian, and some hardened builds remove it on purpose. If you have sudo and prefer it, sudo -u metricharbor … does the same thing.

Not su - metricharbor either way: that account has a nologin shell on purpose, so su refuses it with "This account is currently not available."

Running these as plain root works too — the agent gives the files it writes to whoever owns the credential store, so the service can still read them. Preferring the service account is tidier, not required.

You are prompted for the password. A secret is never a command-line argument. /proc/<pid>/cmdline is world-readable and shell history is a file on disk, so there is no --password= form of this command, by design. Use the prompt, --password-file, a creds.d drop file, or cred import.

Not stdin. Piping a secret in is refused rather than accepted quietly — reading a line unechoed is impossible when stdin is a pipe, and the alternative is echoing the secret:

$ echo 'hunter2' | metricharbor-agent cred add db --type database_password
error: a secret cannot be typed when stdin is not a terminal; use --<field>-file,
       the creds.d drop file, or cred import

2. Let the agent generate it — SSH only#

SSH keys are the one kind where the secret does not exist yet, and nobody should be moving private keys around:

runuser -u metricharbor -- metricharbor-agent cred new-ssh ssh-monitor --username monitor

The keypair is created on that host. The private half never exists anywhere else — not in MetricHarbor, not in a browser, not in a paste buffer. The command prints the public key to add to ~monitor/.ssh/authorized_keys on the hosts you want monitored.

--restrict emits a command= forced-command prefix, limiting the key to the collector's own commands. Worth using: it means a stolen key from that host cannot be used for a shell.

3. A file drop, for configuration management#

/etc/metricharbor/creds.d/postgres-monitor.cred    # mode 0600

Dropped in plaintext by Ansible or Puppet, then encrypted and shredded by the agent on its next start, so a fleet rollout needs no second step. Use this if you already manage secrets with ansible-vault or similar and do not want to shell out to a CLI.

Checking it worked#

metricharbor-agent cred list      # names only; it will not print a secret
metricharbor-agent check          # config parses, credentials resolve, targets answer

In the web interface, each credential shows one of three states:

• held — one or more agents report holding a secret under that name.
• not on any agent — no agent has it. Any target that needs it is collecting nothing. This is the single worst failure in the design, because it produces silence rather than an error, so the page shows it in red with the command to fix it.
• unknown — no agent has reported in yet, so what they hold is not known. Not the same as broken; telling you to go fix a file on a host that has simply not checked in yet would waste your time.

Moving credentials between hosts#

The encryption key is bound to the machine, so copying creds.d/ to a replacement host will not decrypt. That is deliberate — a stolen disk image is not a stolen credential — and it is why export exists:

runuser -u metricharbor -- metricharbor-agent cred export --out creds.mhbackup
# Passphrase: ********   (yours, not the machine key — this bundle is portable)

runuser -u metricharbor -- metricharbor-agent cred import creds.mhbackup    # on the new host

The same bundle is how one credential reaches a fleet. A database password often serves twenty hosts: enter it once, export, and distribute the bundle with whatever you already use for secrets. Backup, host migration and fleet rollout are the same two commands, and the secret still never touches MetricHarbor.

What is protected, and what is not#

Be clear about this, because it is easy to over-read.

The at-rest encryption protects backups, disk images and stray copies. It does not protect against root on that host: if an attacker is root, they own the credentials regardless of what any software does.

What you get is a smaller blast radius in both directions. If one agent host is compromised, the attacker gets the credentials on that host — which is why per-agent credentials and new-ssh --restrict are worth the effort — and nothing on your other agents, and nothing in MetricHarbor beyond that one agent's ingest token. If MetricHarbor is compromised, an attacker gets no credential of yours at all.