# Alert on a stopped RouterOS container via ntfy and Telegram > A RouterOS 7 scheduler script that pushes one ntfy and Telegram message per change when named containers stop, with every secret read from a container envlist. Published 2026-11-02. HTML version: https://guides.tsvinc.dev/routeros-container-watchdog-ntfy-telegram/ This installs a RouterOS 7 scheduler script that checks named containers once a minute and sends one message to ntfy and Telegram each time the set of stopped containers changes. The endpoints and tokens are read from a container envlist at run time, so the script file holds no secrets. > **Scope** — Verified on: MikroTik hAP ax^3, RouterOS 7.24.4 (stable), watching containers `observer` and `mosquitto` (2026-10-16 install and test message; 2026-10-17 read-only check). **The down path (a container actually stopping) has not been triggered yet.** > **Needs:** RouterOS 7 with the container package and at least one container; SSH and SCP to the router (I used `admin`, group `full`; a narrower user needs at least `ssh` and `ftp` on top of `read,write,test`, untested); a disk on the router for the `.rsc` file; an ntfy topic URL and/or a Telegram bot token plus chat id. > **Time:** 15 min. **Risk:** low. The script only reads container state and sends HTTP requests. Adding the sink settings to an envlist that a running container uses puts them into that container's environment on its next re-create. Rollback removes two scripts, one scheduler entry and the envlist keys. ## Before you start Placeholders used below: | Placeholder | Meaning | |---|---| | `` | `user@address` of the router; add your own SSH options (key, `-o ControlPath=none`) | | `` | full ntfy topic URL, `https://ntfy.sh/` | | `` | optional ntfy access token (`tk_…`); leave the key out if the topic needs none | | `` | Telegram bot token from BotFather | | `` | Telegram chat id the bot writes to | - **RouterOS version.** Same command and router as the field notes, output cut to the relevant lines: ```console $ ssh '/system/resource/print' … version: 7.24.4 (stable) … architecture-name: arm64 board-name: hAP ax^3 … ``` - **The containers to watch exist and run.** The script below watches `observer` and `mosquitto`; change line 42 of the `.rsc` for your names. ```console $ ssh '/container/print proplist=name,running where name~"^(observer|mosquitto)\$"' Flags: R - RUNNING Columns: NAME, RUNNING # NAME RUNNING ;;; mosquitto 2 R mosquitto yes ;;; observer 1 R observer yes ``` - **Envlist name.** The script reads envlist `observer` (line 16 of the `.rsc`), which my observer container already uses for the same ntfy and Telegram settings. Use any list name; if no container uses it, the keys only serve the script. - **Secret values must not contain** a double quote, `$`, backslash, `;` or space. They are pasted into a RouterOS command inside double quotes, and those characters break the quoting. My installer refuses such values; check yours by hand. - **Backup.** Export the config before step 1 and copy the file off the router (my disk is `usb1-part1`; substitute yours here and in steps 3–5): ```bash ssh '/export file=usb1-part1/backups/pre-watchdog' scp :/usb1-part1/backups/pre-watchdog.rsc . ``` **The export is a secret file.** On 7.24.4 a plain `/export` (no `show-sensitive`) still writes every `/container envs` entry with its value: my export from 2026-10-16 14:32 holds seven, tokens and passwords included. After step 1 your ntfy and Telegram secrets are in every export. Keep exports out of git and off shared disks. ## Steps 1. **Put the sink settings into the envlist.** Remove old keys, then add one key per setting. Leave out any key you don't use: the script skips ntfy when `OBSERVER_NTFY_URL` is empty and Telegram when either Telegram key is empty. ```bash ssh '/container/envs/remove [find list=observer key~"OBSERVER_(NTFY|TELEGRAM)"]; /container/envs/add list=observer key=OBSERVER_NTFY_URL value=""; /container/envs/add list=observer key=OBSERVER_NTFY_TOKEN value=""; /container/envs/add list=observer key=OBSERVER_TELEGRAM_TOKEN value=""; /container/envs/add list=observer key=OBSERVER_TELEGRAM_CHAT value=""' ``` A line typed by hand is saved in your shell history with the tokens in it. My installer builds the same command from a gitignored env file instead. Check the keys exist without printing their values (I set three: no ntfy token): ```console $ ssh ':put [:len [/container/envs/find list=observer key~"OBSERVER_(NTFY|TELEGRAM)"]]' 3 ``` Expected: the number of keys you added. 2. **Save the watchdog as `ups-watchdog.rsc`.** It holds no secrets. `ups-notify` sends the global `$upsMsg` to every configured sink; `ups-watchdog` builds the list of stopped or missing containers and calls `ups-notify` when that list changes. The file, unchanged: ```text # Router-side watchdog: alerts when the observer or mosquitto container stops (the observer can't # report its own death). No secrets in this file: ups-notify reads the ntfy/Telegram settings from # envlist "observer" (OBSERVER_NTFY_URL, OBSERVER_NTFY_TOKEN, OBSERVER_TELEGRAM_TOKEN, OBSERVER_TELEGRAM_CHAT). # ups-notify sends the global $upsMsg to every configured sink # ups-watchdog every 1 min (scheduler); one message per change in which containers are down; # skipped during the first 3 min after boot # Usage: /import file-name=usb1-part1/ups-watchdog.rsc # Test: :global upsMsg "router: test"; /system/script/run ups-notify # Rollback: /system/scheduler/remove [find name=ups-watchdog]; /system/script/remove [find name~"^ups-(watchdog|notify)\$"] /system/scheduler/remove [find name=ups-watchdog] /system/script/remove [find name~"^ups-(watchdog|notify)\$"] /system/script/add name=ups-notify policy=read,write,test source={ :global upsMsg :local env do={ :local id [/container/envs/find list=observer key=$1] :if ([:len $id]=0) do={:return ""} :return [/container/envs/get $id value] } :local ntfy [$env "OBSERVER_NTFY_URL"] :local ntfyTok [$env "OBSERVER_NTFY_TOKEN"] :local tg [$env "OBSERVER_TELEGRAM_TOKEN"] :local chat [$env "OBSERVER_TELEGRAM_CHAT"] :if ($ntfy != "") do={ :local hdr "Title: ups-watchdog,Priority: 4" :if ($ntfyTok != "") do={:set hdr ($hdr . ",Authorization: Bearer " . $ntfyTok)} :do {/tool/fetch url=$ntfy http-method=post http-data=$upsMsg http-header-field=$hdr output=none} on-error={:log warning "ups-notify: ntfy failed"} } :if ($tg != "" && $chat != "") do={ :local body ("{\"chat_id\":\"" . $chat . "\",\"text\":\"" . $upsMsg . "\"}") :do {/tool/fetch url=("https://api.telegram.org/bot" . $tg . "/sendMessage") http-method=post http-header-field="Content-Type: application/json" http-data=$body output=none} on-error={:log warning "ups-notify: telegram failed"} } :log warning ("ups-notify: " . $upsMsg) } /system/script/add name=ups-watchdog policy=read,write,test source={ :global upsWdState :global upsMsg :if ([:typeof $upsWdState]="nothing") do={:set upsWdState ""} :if ([/system/resource/get uptime] >= 3m) do={ :local down "" :foreach n in={"observer";"mosquitto"} do={ :local id [/container/find name=$n] :if ([:len $id]=0) do={:set down ($down . $n . " missing; ")} else={ :if ([/container/get $id running]!=true) do={:set down ($down . $n . " not running; ")} } } :if ($down != $upsWdState) do={ :set upsMsg "router: observer and mosquitto running again" :if ($down != "") do={:set upsMsg ("router: " . $down)} /system/script/run ups-notify :set upsWdState $down } } } /system/scheduler/add name=ups-watchdog interval=1m start-time=startup policy=read,write,test on-event="/system/script/run ups-watchdog" :put "ups-watchdog installed" ``` To adapt: the envlist name (line 16), the container names (line 42) and the "running again" text (line 49). The first two lines after the header remove an earlier install, so re-importing the file replaces it. 3. **Copy the file to the router disk.** My disk is `usb1-part1`: ```bash scp ups-watchdog.rsc :/usb1-part1/ ``` 4. **Dry-run the import and read the last line.** ```console $ ssh '/import file-name=usb1-part1/ups-watchdog.rsc dry-run verbose=yes' … #line 57 /system/scheduler/add name=ups-watchdog interval=1m start-time=startup policy=read,write,test on-event="/system/script/run ups-watchdog" #line 58 :put "ups-watchdog installed" No syntax errors found in the import file ``` Expected: the last line is `No syntax errors found in the import file`. In a script, test for that line. Do not grep for `syntax error`: the success line contains it (Failure modes, row 1). 5. **Import.** ```console $ ssh '/import file-name=usb1-part1/ups-watchdog.rsc' ups-watchdog installed Script file loaded and executed successfully ``` Expected: both lines. The router logs each `script add` with the full script source (`system,info new script added by ssh-cmd:…`), which is another reason to keep secrets out of the source. 6. **Check the scheduler entry.** Right after my import: ```console $ ssh '/system/scheduler/print detail where name=ups-watchdog' 0 name="ups-watchdog" start-time=startup interval=1m on-event=/system/script/run ups-watchdog owner="admin" policy=read,write,test run-count=0 next-run=2026-10-16 14:32:05 ``` Expected: `interval=1m`, `start-time=startup` and a `next-run` within the next minute. 7. **Send a test message through the same path a real alert takes.** ```console $ ssh ':global upsMsg "router: ups-watchdog installed (test)"; /system/script/run ups-notify; :delay 5s; /log/print where message~"ups-notify"' … 2026-10-16 14:31:08 script,warning ups-notify: router: ups-watchdog installed (test) ``` Expected: the `script,warning ups-notify:` line with your text, and the message on each sink. On mine the log also had `2026-10-16 14:31:07 fetch,info Download from api.telegram.org FINISHED` one second earlier. The message arrived on ntfy, and in the Telegram chat as (bot name replaced): ```text [10/16/26 2:31 PM] : router: ups-watchdog installed (test) ``` No log line from `ups-notify` other than the final one means no fetch failed. A failed fetch logs `ups-notify: ntfy failed` or `ups-notify: telegram failed` (lines 27 and 31). 8. **Stop a watched container and start it again.** `[UNVERIFIED: down path not triggered yet]` Stopping a container stops its service (my `observer` is the NUT server for the LAN hosts), so do it while on grid power. Keep the container stopped for at least 70 s: the scheduler checks once a minute, and a shorter stop can fall between two checks and go unreported. How `restart-policy=always` treats a manual stop was not checked. ```bash ssh '/container/stop [find name=observer]' ``` Expected, within two minutes (one scheduler run after the container has stopped): a message `router: observer not running; ` on both sinks, built by lines 45 and 50, and a log line `ups-notify: router: observer not running; `. Then: ```bash ssh '/container/start [find name=observer]' ``` Expected, within two minutes after the container is running: `router: observer and mosquitto running again` (line 49). I have not run this step; the expected messages come from reading the script, not from a run. ## Verify The scheduler runs once a minute and the watchdog's state is empty (no watched container down). From my router, about ten hours after the install: ```console $ ssh '/system/scheduler/print detail where name=ups-watchdog' 0 name="ups-watchdog" start-time=startup interval=1m on-event=/system/script/run ups-watchdog owner="admin" policy=read,write,test run-count=594 next-run=2026-10-17 00:26:05 $ ssh '/system/script/environment/print' Columns: NAME, VALUE # NAME VALUE 0 upsMsg router: ups-watchdog installed (test) 1 upsWdState ``` Expected: `run-count` grows by one per minute (594 runs from the first at 14:32:05 to 00:25:05), and `upsWdState` is empty while every watched container runs. When one is down it holds the same text as the last alert, for example `observer not running; `. Read the state with `environment/print`: `:global upsWdState` typed in a new SSH session prints nothing (field notes, 2026-10-17). ## Undo / rollback `[UNVERIFIED: rollback not exercised]` From the `.rsc` header and my installer: ```bash ssh '/system/scheduler/remove [find name=ups-watchdog]; /system/script/remove [find name~"^ups-(watchdog|notify)\$"]' ssh '/container/envs/remove [find list=observer key~"OBSERVER_(NTFY|TELEGRAM)"]' ``` The second line also removes the alert settings from any container that uses the same envlist, from its next re-create on. Skip it if a container sends its own alerts from these keys. The globals `upsMsg` and `upsWdState` stay in `/system/script/environment` until a reboot. ## Failure modes | Symptom (exact message) | Cause | Fix | |---|---|---| | `No syntax errors found in the import file`, followed by my installer's `ABORT: dry-run found errors` | The installer's error check was a case-insensitive grep for patterns including `syntax error`, which matches the success line | Treat only `No syntax errors found` as success (step 4) | | ntfy shows the test, Telegram seems to have nothing, while the router log has `fetch,info Download from api.telegram.org FINISHED` | Not a send failure: the message was in the bot chat when I opened it; I had only looked at notifications | Open the chat itself before debugging the token or chat id | ## Field notes - **2026-10-17 00:26 router time (21:26 UTC), hAP ax^3, RouterOS 7.24.4 (read-only checks):** both containers running; scheduler at `run-count=594`; `upsWdState` empty in `/system/script/environment`; the only `ups-notify` warning in the log is the test from the install, so no watched container stayed stopped through a scheduler run in those ten hours. The `observer` container was re-created twice in that window (14:32:46–14:32:54 and 23:55:45–23:55:52 router time) and no alert went out: both gaps fell between two runs, which tick at :05 each minute. `:global upsWdState; :put [:typeof $upsWdState]` from a new SSH session printed `nothing`, while `environment/print` shows the scheduler's empty string: the script sets it to empty on its first run only, and to the new state on each change. - **2026-10-16 14:26–14:31 router time (UTC+3), hAP ax^3, RouterOS 7.24.4 (install):** my installer first stopped at the dry run with `No syntax errors found in the import file` on screen followed by its own `ABORT`; its error grep matched the success line. I changed the check to require the success line and re-ran it. At 14:31:06 the router logged both scripts and the scheduler as added, at 14:31:07 the Telegram fetch `FINISHED`, at 14:31:08 the `ups-notify` warning with the test text. The test arrived on ntfy. I thought the Telegram message was missing until I opened the bot chat: it was there, stamped 2:31 PM, with the observer's own startup alert a minute later. The stop/start test (step 8) was not run. ## Notes - **No retry.** `ups-notify` makes one fetch per sink. A failed fetch is logged, and `ups-watchdog` stores the new state anyway (line 52), so an alert sent while the uplink is down is lost; the next message goes out only at the next change. The alerts my observer sends itself retry until delivered; why both exist: [The alert about the blackout needs the internet too](https://tsvinc.dev/posts/ups-alerts-survive-the-outage/). - **Fixed messages only.** The Telegram body is built by string concatenation (line 30): a `"` or `\` in `$upsMsg` breaks the JSON. `/tool/fetch` takes headers as a comma-separated list (`http-header-field="h1:fff,h2:yyy"`), so a comma in the ntfy title or token would split the header. - **Short stops go unreported.** The check runs once a minute; a stop and restart between two runs (a re-create took under 10 s here) sends nothing. Twice observed on 2026-10-16. - **Policy.** `read,write,test` is what I deployed for the scripts and the scheduler. I have not tested a narrower policy, and my SSH user was `admin` (`full`). - **After a reboot** the scheduler starts 3 s after startup (MikroTik's scheduler docs), the globals start empty, and the watchdog does nothing until uptime reaches 3 minutes. A watched container that is not running by then is reported as down. - **ntfy `Priority: 4`** is "high" in ntfy's table (long vibration burst, pop-over notification). - Re-creating a container with new envlist values: [Update a RouterOS 7 container from a new image tar](/routeros-container-update-from-tar/). Other RouterOS container traps: [Six gotchas deep in RouterOS containers](https://tsvinc.dev/posts/routeros-container-gotchas/). Message wording: [Designing alerts you'll actually understand at 3am](https://tsvinc.dev/posts/alerts-you-understand-at-3am/). ## Sources - [MikroTik: Fetch](https://help.mikrotik.com/docs/spaces/ROS/pages/8978514/Fetch) — `http-method`, `http-data` (64 KB limit), `http-header-field` list format, `output=none`. - [MikroTik: Container](https://help.mikrotik.com/docs/spaces/ROS/pages/84901929/Container) — `/container/envs/add list= key= value=` and `envlist`. - [MikroTik: Scripting](https://help.mikrotik.com/docs/spaces/ROS/pages/47579229/Scripting) — global variables in the environment, `do {…} on-error={…}`. - [MikroTik: Scheduler](https://help.mikrotik.com/docs/spaces/ROS/pages/40992881/Scheduler) — `start-time=startup` runs 3 s after boot; a reboot resets `run-count`. - [ntfy: Sending messages](https://docs.ntfy.sh/publish/) — POST body as the message, `Title` and `Priority` headers, priority levels, `Authorization: Bearer tk_…`. - [Telegram Bot API](https://core.telegram.org/bots/api) — `sendMessage` with `chat_id` and `text` (1–4096 characters), JSON request bodies.