txmesh
Sign in
Docs

How txmesh works

A hosted bot service for MeshCore, and the wire contract that lets you build your own.

The pieces Identity and namespace Uplink: what the gateway publishes Downlink: the send bridge Hop chains What a backend has to do A backend you can build in an evening What our setup looks like Channels and keys Connection Privacy decisions we made, which you get to make too Versioning

txmesh gives a MeshCore mesh bots: !commands that answer over the air, scheduled announcements, automated NWS weather and river-flood alerts, and route traces. It runs as a free hosted service. The radio side is an openHop repeater running the openhop-txmesh-plugin (MIT), and everything between that plugin and our backend crosses one MQTT broker as documented JSON. This paper is that contract, plus the design decisions behind our backend, so that anyone can point the same plugin at their own broker and write their own engine.

The backend itself is not open source. The plugin is, and the plugin is the only part that touches a radio. Everything below is what you need to replace us.

1. The pieces

 your repeater host                        ours (or yours)
 ┌──────────────────┐   companion TCP    ┌────────┐  MQTT/WSS 443  ┌───────────┐     ┌───────────┐
 │ openHop repeater ├───────────────────►│ plugin ├───────────────►│ connector ├────►│ mosquitto │
 └──────────────────┘                    └────────┘                └───────────┘     └─────┬─────┘
                                                                        meshcore/#  ▲      │
                                                                                    │      ▼
                                                                    send/<channel>  │ ┌───────────┐
                                                                                    └─┤ collector │ archive · bot engine · alert pollers
                                                                                      └─────┬─────┘
                                                                                            ▼
                                                                                      ┌──────────┐     ┌───────────┐
                                                                                      │ postgres ├────►│ api + web │
                                                                                      └──────────┘     └───────────┘
PieceWhat it doesOurs or yours
openHop repeaterThe LoRa node. Keeps repeating for everyone.Yours
PluginAdds a companion identity to the repeater (a second node on the same board), mirrors what that node hears to MQTT, and transmits text it receives from MQTT.Open source, MIT
Brokermosquitto. Every login is confined to its own namespace.Ours (or yours)
ConnectorA WebSocket front in front of the broker that answers WS-level pings on the radio's behalf. See §10.Ours (or yours)
CollectorSubscribes to everything, archives it, runs the bot engine and the alert pollers.Ours (this is what you would replace)
API + webThe map, roster, topology, chat archive, route pages, admin.Ours

The topic schema below originated in our own earlier observer firmware, a MeshCore fork that is no longer offered and was never part of the MeshCore project. The plugin publishes the same topics, so a backend written against that firmware keeps working, but there is no upstream specification to defer to: this document is the reference.

2. Identity and namespace

Every gateway publishes under a prefix:

meshcore/<username>/<node_name>

username is the MQTT login (obs-alice on our broker), node_name is the companion identity's name. The plugin sanitises both: #, +, / and space become -. The plugin also knows a fleet prefix, meshcore/<username>/all, which exists only so that one message can be sent through every gateway a user runs.

On our broker each login holds an ACL for meshcore/<username>/# and nothing else. That is the whole security model for attribution: a gateway cannot publish under another user's prefix, so the topic is the proof of who reported something. A backend that runs its own broker should keep that property. A shared "observer" role with meshcore/# lets any tenant forge another's traffic.

The plugin supports two other layouts (base_topic verbatim, or meshcore/<IATA>/<node> via iata), which exist for compatibility with the community meshcore-mqtt-broker family. Under those there is no fleet topic.

The 8-hex-character node id used in topics below (<pk8>) is the first four bytes of the node's public key, lowercase.

5. Hop chains

MeshCore frames carry the path they took as a list of hop hashes: the leading 1, 2 or 3 bytes of each relaying repeater's public key, in travel order. The header's path-length byte encodes both: hash_size = (path_len >> 6) + 1, hop_count = path_len & 0x3F.

The plugin publishes the chain as hops (hex, full width, so a 2-byte mesh gives 4 hex characters per hop) and hops_n (hop count). A direct hearing is hops_n: 0 and no hops. A backend splits the chain by dividing the hex length by hops_n; consecutive pairs are the RF adjacencies that build a topology graph.

5.1 Adverts

The chain is read straight from the frame. This is exact.

5.2 Messages

The companion protocol hands the plugin the decrypted text with only the path length, not the path. To recover the chain the plugin caches recent raw text frames (64 frames, 30 s) and matches one structurally: same payload type, same path-length byte, a ciphertext length consistent with the text, and the frame's lead byte equal to the channel's 1-byte hash (sha256(key)[0]) or the sender's prefix byte for DMs. The oldest match wins. This is a heuristic, and the plugin's README says so. Treat hops on msg/* as best effort and hops_n as reliable.

5.3 Resolving hashes to nodes

A hash is a prefix, so it collides. At one byte every hop collides with something; at three bytes it still collides once the roster is large enough. Our resolver treats a chain as a Viterbi problem: for each hop, the candidate set is every known node whose key starts with that hash; the chain is anchored at whichever ends have known positions (the requester's advertised position, the gateway's); and the path chosen minimises total geographic length, with a penalty for stepping between candidates whose known meshes are disjoint. That last part matters more than it sounds: with no anchors, geography alone picks the tightest cluster, which can be on another mesh entirely.

The plugin sets the repeater's path-hash width from path_hash_bytes (default 2). Ask your community to run 2-byte hashes; 1-byte chains are barely resolvable.

6. What a backend has to do

The minimum that behaves correctly on a real mesh, in the order the problems will find you:

  1. Subscribe meshcore/# and parse by kind. Split the topic on /, find the first segment in {status, telemetry, sensors, contact, heard, msg, advert, packets, send}; the segment before it is the node, everything before that is the prefix, anything after is the sub-key (<pk8>, channel, dm). Ignore send on the way in. This grammar tolerates all three prefix layouts.
  2. Treat retained replays as history, not events. On every reconnect the broker replays every retained message under meshcore/#: each gateway's whole heard/ and contact/ table and the status of every gateway that ever existed, including the offline last-will of one unplugged months ago. If a replay stamps "last seen" or increments an edge count, every restart makes a dead node live and every link look fresh. Use the MQTT retain flag on delivery: update content, never timestamps or counts.
  3. Split the sender out of channel text. "Name: body" is the on-air form. A message with no ": " has no sender. A group-text sender is a claim, not a signature: the 2-byte MAC is a filter, and any channel member can put any name in front of a colon. Archive it, never authorise on it.
  4. Fire once per message, not once per gateway. Several of your gateways will hear the same !command, seconds apart, over different paths. We claim a cooldown window per (bot, scope) in one atomic database statement before rendering; scope * gives one reply for the mesh. For location-aware bots we claim per site then per path, so each place that heard the request over a distinct chain answers from its own radio and co-located radios share one voice.
  5. Reply through the gateway that heard it. Publish to <that gateway's prefix>/send/<channel>. That puts the answer back on the air where the question came from. Fleet topics (<user>/all/send/) are for announcements.
  6. Never answer on Public, and never react to your own gateway's transmissions (sender == node). A bot that echoes on the shared channel is a nuisance for a whole region; one that answers itself is a loop.
  7. A reply that renders empty transmits nothing and releases its claim. Otherwise the gateway with nothing to say (the one that heard the request direct, for a path bot) wins the claim and silences the ones with an answer.
  8. Keep an audit row for every send. The plugin gives you nothing back.

Everything else (templates, helpers, scheduling, alert pollers, a map) is product.

7. A backend you can build in an evening

Nothing in §6 needs a cluster. The smallest honest version is one always-on box (a Raspberry Pi is plenty), a broker, and one Python process with SQLite. No cloud, no container, no web page yet.

7.1 Parts

PartChoiceWhy
Brokermosquitto, on the same boxThe plugin only needs MQTT 3.1.1 with a username. Password file plus ACL file is enough for a home mesh; dynamic security is for many tenants.
Engineone Python process, paho-mqttSubscribes meshcore/#, archives, answers. Single process means the "fire once" claim can be a local table.
StorageSQLiteOne file. Every payload archived as-is is already a queryable history.
Radioyour existing openHop repeater with the pluginNothing else to buy.

7.2 Broker

/etc/mosquitto/conf.d/mesh.conf:

listener 1883
allow_anonymous false
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl
mosquitto_passwd -c /etc/mosquitto/passwd obs-home
mosquitto_passwd    /etc/mosquitto/passwd engine

/etc/mosquitto/acl, one block per gateway login, one for the engine:

user obs-home
topic readwrite meshcore/obs-home/#

user engine
topic readwrite meshcore/#

That keeps the property from §2: a gateway can only write under its own prefix. On a LAN plain 1883 is fine; the plugin's config for it is "host": "<box ip>", "port": 1883, "transport": "tcp", "tls_enabled": false. If the repeater reaches the box over the internet, put TLS on 8883 with a real certificate and leave tls_enabled true. Do not use mosquitto's own WebSocket listener without reading §10 first.

7.3 The engine

The whole thing, minus the bots you will write. It archives every payload, ignores retained replays as events, splits the sender out of channel text, refuses Public and its own echoes, claims a cooldown per bot before answering, and replies through the gateway that heard the request.

#!/usr/bin/env python3
"""Minimal MeshCore bot engine for the openhop-txmesh-plugin topic schema."""
import json, sqlite3, time
import paho.mqtt.client as mqtt

BROKER, USER, PASS = "127.0.0.1", "engine", "change-me"
KINDS = {"status", "telemetry", "sensors", "contact", "heard", "msg", "advert", "packets", "send"}
PROTECTED = {"public"}            # never answer here
COOLDOWN_S = 30                   # per bot, mesh-wide

db = sqlite3.connect("mesh.db", isolation_level=None)   # autocommit
db.executescript("""
CREATE TABLE IF NOT EXISTS archive (ts INTEGER, topic TEXT, retained INTEGER, payload TEXT);
CREATE TABLE IF NOT EXISTS fires   (bot TEXT PRIMARY KEY, until INTEGER);
CREATE TABLE IF NOT EXISTS heard   (node TEXT, pk8 TEXT, name TEXT, hops TEXT, hops_n INTEGER,
                                    updated INTEGER, PRIMARY KEY (node, pk8));
""")

# --- bots: trigger prefix -> function(sender, args, ctx) -> reply text or "" ----------
def bot_ping(sender, args, ctx):
    return f"pong {sender}, heard by {ctx['node']} at {ctx.get('snr', '?')} dB"

def bot_seen(sender, args, ctx):
    row = db.execute("SELECT name, updated FROM heard WHERE name = ? COLLATE NOCASE"
                     " ORDER BY updated DESC LIMIT 1", (args,)).fetchone()
    if not row:
        return f"never heard {args}"
    if row[1] is None:
        return f"{row[0]} is known, but only from a replay; no live hearing yet"
    return f"{row[0]} last heard {int(time.time() - row[1]) // 60} min ago"

BOTS = {"!ping": bot_ping, "!seen": bot_seen}

# --- topic grammar: meshcore/<...prefix...>/<node>/<kind>[/<sub>] -----------------
def parse(topic):
    parts = topic.split("/")
    if len(parts) < 3 or parts[0] != "meshcore":
        return None
    ki = next((i for i, x in enumerate(parts) if x in KINDS), -1)
    if ki < 1:
        return None
    return {"node": parts[ki - 1], "kind": parts[ki], "sub": parts[ki + 1:],
            "prefix": "/".join(parts[:ki])}

def claim(bot):
    """One reply per cooldown window, however many gateways heard the trigger."""
    now = int(time.time())
    cur = db.execute("INSERT INTO fires (bot, until) VALUES (?, ?)"
                     " ON CONFLICT(bot) DO UPDATE SET until = excluded.until"
                     " WHERE fires.until <= ?", (bot, now + COOLDOWN_S, now))
    return cur.rowcount == 1

def on_message(client, _userdata, m):
    t = parse(m.topic)
    if not t or t["kind"] == "send":
        return
    payload = m.payload.decode("utf-8", "replace").strip()
    db.execute("INSERT INTO archive VALUES (?,?,?,?)", (int(time.time()), m.topic, int(m.retain), payload))
    if t["kind"] == "status" or not payload:
        return
    data = json.loads(payload)
    if t["kind"] == "heard" and t["sub"]:
        # content may update on a retained replay; the timestamp only on live traffic
        stamp = None if m.retain else int(time.time())
        db.execute("INSERT INTO heard VALUES (?,?,?,?,?,?) ON CONFLICT(node, pk8) DO UPDATE SET"
                   " name = excluded.name, hops = excluded.hops, hops_n = excluded.hops_n,"
                   " updated = COALESCE(excluded.updated, heard.updated)",
                   (t["node"], t["sub"][0], data.get("name"), data.get("hops"),
                    data.get("hops_n"), stamp))
        return
    if t["kind"] == "msg" and t["sub"] == ["channel"] and not m.retain:
        channel, text = data.get("channel", ""), data.get("text", "")
        if channel.lstrip("#").lower() in PROTECTED or ": " not in text:
            return
        sender, body = text.split(": ", 1)
        if sender == t["node"]:
            return                                     # our own transmission
        for trig, fn in BOTS.items():
            if body.lower().startswith(trig) and claim(trig):
                out = fn(sender, body[len(trig):].strip(), {**data, **t})
                if out.strip():
                    topic = f"{t['prefix']}/send/{channel.lstrip('#')}"
                    client.publish(topic, out.encode()[:160], qos=0, retain=False)
                    db.execute("INSERT INTO archive VALUES (?,?,?,?)",
                               (int(time.time()), topic, 0, out))   # the audit row
                else:
                    db.execute("DELETE FROM fires WHERE bot = ?", (trig,))   # give the claim back

c = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
c.username_pw_set(USER, PASS)
c.on_connect = lambda cl, *_: cl.subscribe("meshcore/#", qos=0)
c.on_message = on_message
c.connect(BROKER, 1883, keepalive=45)
c.loop_forever()

Run it, send !ping on #bot from any companion app, and the repeater answers. The archive table is everything the gateway ever said, with the retain flag, so the map and the roster can be built later from data you already have.

Two deliberate simplifications. The claim table is mesh-wide per bot, which is the * scope from §6; per-site and per-path fan-out is a second key on that table. And a follow-up frame is not shown; if a bot needs two, publish the second on a timer at least 5 s later.

7.4 What to add next, in the order it pays off

  1. Announcements. A cron entry that publishes text to meshcore/<user>/all/send/<channel> is a scheduled announcement. No engine change.
  2. A weather poller. One HTTP GET a minute to the NWS alerts API for your zone, dedupe on alert id in SQLite, publish new ones to the fleet send topic. Cut the text to 160 bytes yourself; the plugin will otherwise do it mid-word.
  3. A roster. contact/<pk8> rows are it: name, type, position, last advert. Upsert them like heard above.
  4. Topology. Split each hops chain into consecutive pairs (§5) and count them per gateway. That is the edge list a map draws.
  5. A path bot. The requester's chain arrives on the !path message itself (hops, hops_n); resolve each hash against the roster prefixes and pick the geographically shortest assignment (§5.3). Reply with the resolved names, or a link to a page you render from the archive.
  6. More than one gateway. Nothing above changes as long as one process does the claiming. When you want a second process, move fires to a database both can reach and keep the claim a single atomic statement.

8. What our setup looks like

For contrast, and so nobody thinks the home build is a toy version of something exotic. Ours is the same three roles, made restartable, running on Kubernetes and spread across more than one region so that losing a site does not take the bots off the air:

None of that changes the wire contract. A gateway pointed at our broker or at the box in §7 publishes the same bytes.

9. Channels and keys

The plugin pushes its channels list into the repeater on every companion connect: [{idx, name, secret?}], slots 1 to 39 (slot 0 is Public and would be overwritten).

The plugin never sees cipher text as text: the repeater decrypts, the plugin publishes the result. A backend receives plaintext for every channel the gateway has joined.

10. Connection

The plugin connects with paho-mqtt, MQTT 3.1.1, clean session, client id observer_<node>_<epoch>, keepalive 45 s, TLS with system roots, over WebSockets on 443 by default or plain TCP. It owns its own reconnect ladder (10, 30, 60, 120, 300 s, then a breaker that probes every 30 min) and disables the library's.

Do not point the plugin at mosquitto's WebSocket listener directly. mosquitto's ws layer sends a WebSocket-level PING every 300 s and drops the TCP connection 10 s later if no PONG comes back. It is a bare FIN, no CLOSE frame, and neither side logs a reason. The MQTT keepalive sits above the ws framing and does not help, so the radio looks healthy right up to the cut. Every connection died at about 310 s, every bot reply published in the down window was silently lost, and the archive looked fine throughout because the reconnect landed a few seconds later.

Our collector endpoint, the one Bot setup shows, is a connector in front of the broker that answers those PINGs on the client's behalf. If you run your own broker, either front it the same way, or use the plain MQTT-over-TLS listener (transport: tcp, port 8883) and skip WebSockets.

11. Privacy decisions we made, which you get to make too

12. Versioning

The plugin's payloads evolve additively: new keys may appear, existing keys keep their meaning, optional keys are omitted rather than nulled. A backend should ignore unknown keys and tolerate absent ones. This document describes plugin 0.1.3; the version is in openhop-plugin.json in the plugin repo, and each release is tagged.

Known gaps in 0.1.3, so nobody wastes an afternoon: advert.hash never emitted, telemetry.relayed always 0, telemetry.dropped never emitted, no DM send, no send acknowledgement.

txmesh is community-run and free. The plugin is MIT-licensed at github.com/zfouts/openhop-txmesh-plugin.

txmesh · Community-run MeshCore bot service · Texas · [email protected]