fix: drop docker logs older than 6d before sending to loki

grafana cloud loki rejects entries older than 7 days with HTTP 400.
loki.source.docker has no "tail since" option, so on first start it
replays logs from each container's start time — long-running
containers (coolify-proxy, traffmonetizer) produced weeks of backlog
that loki refused to ingest.

insert a loki.process drop stage (older_than=144h) between the docker
source and loki.write.gc so stale entries are filtered before egress.
journal source already bounded by max_age=12h — no filter needed there.
This commit is contained in:
2026-04-24 14:20:50 +07:00
parent 4cfeec3c8c
commit a4aa225a70
+12 -1
View File
@@ -114,10 +114,21 @@ configs:
}
}
// Grafana Cloud Loki rejects entries older than 7 days (HTTP 400).
// Long-running containers replay weeks of backlog on first start — drop
// anything older than 6 days to stay safely under the cutoff.
loki.process "drop_old" {
forward_to = [loki.write.gc.receiver]
stage.drop {
older_than = "144h"
drop_counter_reason = "entry_too_old"
}
}
loki.source.docker "containers" {
host = "unix:///var/run/docker.sock"
targets = discovery.relabel.container_logs.output
forward_to = [loki.write.gc.receiver]
forward_to = [loki.process.drop_old.receiver]
}
loki.relabel "journal" {