Alert on a stopped RouterOS container via ntfy and Telegram

Steps#

1. Put the sink settings into the envlist. Remove old keys, then add one key per setting. Leave out any key you don't use: the script skips ntfy when OBSERVER_NTFY_URL is empty and Telegram when either Telegram key is empty.

   ssh <ROUTER> '/container/envs/remove [find list=observer key~"OBSERVER_(NTFY|TELEGRAM)"]; /container/envs/add list=observer key=OBSERVER_NTFY_URL value="<NTFY_URL>"; /container/envs/add list=observer key=OBSERVER_NTFY_TOKEN value="<NTFY_TOKEN>"; /container/envs/add list=observer key=OBSERVER_TELEGRAM_TOKEN value="<TG_TOKEN>"; /container/envs/add list=observer key=OBSERVER_TELEGRAM_CHAT value="<CHAT_ID>"'
   

   A line typed by hand is saved in your shell history with the tokens in it. My installer builds the same command from a gitignored env file instead. Check the keys exist without printing their values (I set three: no ntfy token):

   $ ssh <ROUTER> ':put [:len [/container/envs/find list=observer key~"OBSERVER_(NTFY|TELEGRAM)"]]'
   3
   

   Expected: the number of keys you added.

2. Save the watchdog as ups-watchdog.rsc. It holds no secrets. ups-notify sends the global $upsMsg to every configured sink; ups-watchdog builds the list of stopped or missing containers and calls ups-notify when that list changes. The file, unchanged:

   # Router-side watchdog: alerts when the observer or mosquitto container stops (the observer can't
   # report its own death). No secrets in this file: ups-notify reads the ntfy/Telegram settings from
   # envlist "observer" (OBSERVER_NTFY_URL, OBSERVER_NTFY_TOKEN, OBSERVER_TELEGRAM_TOKEN, OBSERVER_TELEGRAM_CHAT).
   #   ups-notify    sends the global $upsMsg to every configured sink
   #   ups-watchdog  every 1 min (scheduler); one message per change in which containers are down;
   #                 skipped during the first 3 min after boot
   # Usage: /import file-name=usb1-part1/ups-watchdog.rsc
   # Test:  :global upsMsg "router: test"; /system/script/run ups-notify
   # Rollback: /system/scheduler/remove [find name=ups-watchdog]; /system/script/remove [find name~"^ups-(watchdog|notify)\$"]
   /system/scheduler/remove [find name=ups-watchdog]
   /system/script/remove [find name~"^ups-(watchdog|notify)\$"]
   
   /system/script/add name=ups-notify policy=read,write,test source={
   :global upsMsg
   :local env do={
     :local id [/container/envs/find list=observer key=$1]
     :if ([:len $id]=0) do={:return ""}
     :return [/container/envs/get $id value]
   }
   :local ntfy [$env "OBSERVER_NTFY_URL"]
   :local ntfyTok [$env "OBSERVER_NTFY_TOKEN"]
   :local tg [$env "OBSERVER_TELEGRAM_TOKEN"]
   :local chat [$env "OBSERVER_TELEGRAM_CHAT"]
   :if ($ntfy != "") do={
     :local hdr "Title: ups-watchdog,Priority: 4"
     :if ($ntfyTok != "") do={:set hdr ($hdr . ",Authorization: Bearer " . $ntfyTok)}
     :do {/tool/fetch url=$ntfy http-method=post http-data=$upsMsg http-header-field=$hdr output=none} on-error={:log warning "ups-notify: ntfy failed"}
   }
   :if ($tg != "" && $chat != "") do={
     :local body ("{\"chat_id\":\"" . $chat . "\",\"text\":\"" . $upsMsg . "\"}")
     :do {/tool/fetch url=("https://api.telegram.org/bot" . $tg . "/sendMessage") http-method=post http-header-field="Content-Type: application/json" http-data=$body output=none} on-error={:log warning "ups-notify: telegram failed"}
   }
   :log warning ("ups-notify: " . $upsMsg)
   }
   
   /system/script/add name=ups-watchdog policy=read,write,test source={
   :global upsWdState
   :global upsMsg
   :if ([:typeof $upsWdState]="nothing") do={:set upsWdState ""}
   :if ([/system/resource/get uptime] >= 3m) do={
   :local down ""
   :foreach n in={"observer";"mosquitto"} do={
     :local id [/container/find name=$n]
     :if ([:len $id]=0) do={:set down ($down . $n . " missing; ")} else={
       :if ([/container/get $id running]!=true) do={:set down ($down . $n . " not running; ")}
     }
   }
   :if ($down != $upsWdState) do={
     :set upsMsg "router: observer and mosquitto running again"
     :if ($down != "") do={:set upsMsg ("router: " . $down)}
     /system/script/run ups-notify
     :set upsWdState $down
   }
   }
   }
   
   /system/scheduler/add name=ups-watchdog interval=1m start-time=startup policy=read,write,test on-event="/system/script/run ups-watchdog"
   :put "ups-watchdog installed"
   

   To adapt: the envlist name (line 16), the container names (line 42) and the "running again" text (line 49). The first two lines after the header remove an earlier install, so re-importing the file replaces it.

3. Copy the file to the router disk. My disk is usb1-part1:

   scp ups-watchdog.rsc <ROUTER>:/usb1-part1/
   

4. Dry-run the import and read the last line.

   $ ssh <ROUTER> '/import file-name=usb1-part1/ups-watchdog.rsc dry-run verbose=yes'
   …
   #line 57
   /system/scheduler/add name=ups-watchdog interval=1m start-time=startup policy=read,write,test on-event="/system/script/run ups-watchdog"
   #line 58
   :put "ups-watchdog installed"
   No syntax errors found in the import file
   

   Expected: the last line is No syntax errors found in the import file. In a script, test for that line. Do not grep for syntax error: the success line contains it (Failure modes, row 1).

5. Import.

   $ ssh <ROUTER> '/import file-name=usb1-part1/ups-watchdog.rsc'
   ups-watchdog installed
   Script file loaded and executed successfully
   

   Expected: both lines. The router logs each script add with the full script source (system,info new script added by ssh-cmd:…), which is another reason to keep secrets out of the source.

6. Check the scheduler entry. Right after my import:

   $ ssh <ROUTER> '/system/scheduler/print detail where name=ups-watchdog'
   0  name="ups-watchdog" start-time=startup interval=1m
      on-event=/system/script/run ups-watchdog owner="admin"
      policy=read,write,test run-count=0 next-run=2026-10-16 14:32:05
   

   Expected: interval=1m, start-time=startup and a next-run within the next minute.

7. Send a test message through the same path a real alert takes.

   $ ssh <ROUTER> ':global upsMsg "router: ups-watchdog installed (test)"; /system/script/run ups-notify; :delay 5s; /log/print where message~"ups-notify"'
   …
    2026-10-16 14:31:08 script,warning ups-notify: router: ups-watchdog installed (test)
   

   Expected: the script,warning ups-notify: line with your text, and the message on each sink. On mine the log also had 2026-10-16 14:31:07 fetch,info Download from api.telegram.org FINISHED one second earlier. The message arrived on ntfy, and in the Telegram chat as (bot name replaced):

   [10/16/26 2:31 PM] <BOT>: router: ups-watchdog installed (test)
   

   No log line from ups-notify other than the final one means no fetch failed. A failed fetch logs ups-notify: ntfy failed or ups-notify: telegram failed (lines 27 and 31).

8. Stop a watched container and start it again. [UNVERIFIED: down path not triggered yet] Stopping a container stops its service (my observer is the NUT server for the LAN hosts), so do it while on grid power. Keep the container stopped for at least 70 s: the scheduler checks once a minute, and a shorter stop can fall between two checks and go unreported. How restart-policy=always treats a manual stop was not checked.

   ssh <ROUTER> '/container/stop [find name=observer]'
   

   Expected, within two minutes (one scheduler run after the container has stopped): a message router: observer not running; on both sinks, built by lines 45 and 50, and a log line ups-notify: router: observer not running; . Then:

   ssh <ROUTER> '/container/start [find name=observer]'
   

   Expected, within two minutes after the container is running: router: observer and mosquitto running again (line 49). I have not run this step; the expected messages come from reading the script, not from a run.