Table of Contents

General Resolutions

General resolutions are a category of general tips/tricks or observations and how to resolve them ranging from server service tips to deployment hacks. The goal is to provide a findable category of problems which have been encountered before and can be resolved


DNS won't resolve

If on a server when executing the ```dig``` command you receive

scotty@heimdall:~$ dig

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>>
;; global options: +cmd
;; connection timed out; no servers could be reached

Check the systemd-resolved.service as if this isn't running the DNS will not be resolved.

sudo systemctl status systemd-resolved.service

   Loaded: loaded (/lib/systemd/system/systemd-resolved.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-19 00:28:49 PDT; 1h 28min ago
     Docs: man:systemd-resolved.service(8)
           https://www.freedesktop.org/wiki/Software/systemd/resolved
           https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
           https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
 Main PID: 31511 (systemd-resolve)
   Status: "Processing requests..."
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/systemd-resolved.service
           └─31511 /lib/systemd/systemd-resolved

Prometheus Logs Missing

As of 2.x Prometheus no longer writes log files to the expected /var/log/prometheus directory. All data is now written to stdout and must be accessed via the journal command

For example to access the prometheus logs we use

  sudo journalctl -u prometheus -f

To access an exporter stdout for troubleshooting you would use

 sudo journalctl -u blackbox_exporter -f