Observability security

How Guidely limits what it can do with your Grafana, Prometheus, Tempo and Loki, and what it keeps.

What Guidely may request

Every request to your backend is checked against a fixed allowlist of read-only query endpoints before it is sent: by Guidely for every connection, and again by the relay for relay connections (the relay carries an identical copy, so it refuses anything else even if asked). Only GET is allowed; paths with .., //, \, %-encoding, whitespace, ? or # are refused, and the query parameters together may be at most 16,000 characters.

KindAllowed paths
Prometheus/api/v1/query, /api/v1/query_range, /api/v1/series, /api/v1/labels, /api/v1/label/<name>/values, /api/v1/status/buildinfo
Loki/loki/api/v1/query, /loki/api/v1/query_range, /loki/api/v1/labels, /loki/api/v1/label/<name>/values, /loki/api/v1/series, /loki/api/v1/status/buildinfo
Tempo/api/traces/<trace id>, /api/v2/traces/<trace id>, /api/search, /api/search/tags, /api/v2/search/tags, /api/v2/search/tag/<name>/values, /api/echo, /api/status/buildinfo
Grafana/api/health, /api/datasources, /api/frontend/settings, and /api/datasources/proxy/uid/<uid>/ followed by one of the Prometheus, Loki or Tempo paths above

<name> is letters, digits, _ and .; a trace id is up to 32 hex characters; a data source UID is up to 64 letters, digits, _ and -. Everything else (admin APIs, writes, deletes, rules, dashboards, alerting, users, other Grafana APIs and any other data source type) is refused before a request is made.

With Grafana, also give the service account only the Viewer role (see Connect Grafana), so Grafana itself refuses anything else.

Direct connections

  • Public addresses only. The URL must be public: private (RFC 1918), loopback, link-local, carrier-grade NAT, multicast and other reserved IPv4 and IPv6 ranges are refused, as are host names without a dot and names ending in .local, .localhost, .internal, .lan, .home or .corp. Any port is allowed. The check runs when the connection is saved and again on every request, against the addresses the name resolves to at that moment, so a name can't be repointed at a private address later. Redirects are not followed.
  • Credentials (bearer tokens, passwords, header values) are encrypted at rest (AES-256-GCM), never returned by the API or shown again (only their last characters), left out of the audit log, and sent only to the connection's own URL.
  • Each request has 10 seconds, and answers over 5 MB are refused.

Relay connections

  • The relay connects out to Guidely over HTTPS and accepts no connections. Its token (gly_rly_…) is shown once; Guidely stores only its SHA-256 hash. Deleting the relay revokes the token at once.
  • Upstream credentials are set in the relay's own environment and never sent to Guidely, which knows only each target's name and kind.
  • The relay applies the allowlist above itself, with a 10-second timeout, a 5 MB answer limit, no redirects and no compression. It writes nothing to disk and logs no query strings or answers.
  • A job can only name a target the relay was configured with; an unknown name is answered with an error.
  • Jobs and answers pass through Guidely's queue (Redis, or memory on a single server) for at most a minute and are then gone.

What is stored

Only the connection settings: names, URLs, data source UIDs, relay names and the target names and kinds each relay reports, the per-project environment selectors and panel queries, and (for direct connections) the encrypted credentials. Query answers (metrics, log lines, traces) are passed straight to the page or AI assistant that asked and are not written to the database, logs, caches, analytics or backups.

Creating, changing and deleting connections and relays, and changing a project's setup, are recorded in the audit log.

Who can see it

  • Viewing panels needs the View backend traces, logs and metrics permission (observability.view). Every role has it by default; admins can revoke it per member.
  • Adding connections and relays needs a workspace admin.
  • Setting up a project (which connection answers what, environment selectors, panels) needs the Developer settings permission.
  • AI assistants (MCP) act as their user and go through the same permission checks; read-only connections can use the observability tools.

HIPAA workspaces

Logs and traces can contain PHI. In a workspace with HIPAA mode on, every look at backend logs or traces (an error event's logs, a trace, a trace search, an AI assistant's query) is recorded in the audit log with the project and what was looked at (ids, never content). Nothing changes about storage: answers still aren't stored.

AI

AI issue analysis can read the backend trace and log lines of the error it explains, and an AI assistant reads whatever its tools return. Both follow the rules of the rest of Guidely's AI features: what is read goes to the model with the rest of the analysis or conversation and isn't stored by Guidely; in HIPAA workspaces, Guidely's own AI key is used only under a business associate agreement with the model provider, a project's own key otherwise, and there is no AI analysis when neither is available. See Security: Scout and AI.

Limits

  • 300 live backend queries a minute per workspace; past that, panels say to try again in a minute.
  • A panel covers at most 7 days.
  • 30 connection tests a minute per workspace.
  • A relay may make 1,200 requests a minute to Guidely (polls and answers together), far more than it needs.