Skip to content
Docs

Quickstart

Run the local demo and open your first message timeline. About 10 minutes, no AWS account.

Based on 2 repo files at 54ea61b

docs/quickstart.md, demo/README.md

On this page

Run Trailbeam on your laptop against an SQS emulator, then open one message's timeline. No AWS account or credentials are used.

Before you start#

You need:

  • git
  • Docker with Compose v2: Docker Desktop on macOS or Windows, or Docker Engine with the compose plugin on Linux
  • About 2 GB of free disk for the images (about 1.5 GB once built)
  • Access to nick-lennox/trailbeam on GitHub (the repository is private for now) and the GitHub CLI, logged in with gh auth login

Check the tools:

shell
git --version
docker compose version   # v2.x or later
docker info              # look for a "Server Version:" line

docker info prints a Server: heading even when the daemon is down, so look for Server Version:. If it isn't there, start Docker first.

1Step 1: Get the code#

shell
gh repo clone nick-lennox/trailbeam
cd trailbeam

A plain git clone https://github.com/nick-lennox/trailbeam fails with could not read Username unless git already has credentials for an account with access.

2Step 2: Start the demo#

shell
docker compose -f demo/compose.yaml up -d --build --wait

The first run downloads the base images and builds four images, which takes 1 to 5 minutes. The command returns when every container is running. Run docker compose -f demo/compose.yaml ps and you should see 7 containers, all Up:

Service What it does
sqs ElasticMQ, an SQS emulator, with queue orders and its dead-letter queue orders-dlq
producer Python with the boto3 hooks. Sends one message a second to orders
consumer-python Four consumer threads on orders, and one thread that receives and deletes from orders-dlq
consumer-node Node.js with @trailbeam/sdk. Four concurrent consumers on orders
deleter A separate Python process that deletes messages the consumers hand to it, by receipt handle
collector The OpenTelemetry Collector with collector/trailbeam-collector.yaml
trailbeam Trailbeam and web screen on 127.0.0.1:4318. Its status reads (healthy)

orders has a 10 s visibility timeout and a redrive policy to orders-dlq with a maxReceiveCount of 3.

3Step 3: Open a timeline#

Wait about a minute, then open http://127.0.0.1:4318/ and click any message ID.

The timeline shows when the message was sent, each delivery and its consumer, and the delete. A message that a slow consumer held too long looks like this:

Timeline sample
  1. 12:00:01.204Sentproducer
  2. 12:00:01.230Receivedconsumer-python · receive count 1
  3. inferredVisibility timeout expiredno delete within the 10 s timeout
  4. 12:00:11.412Receivedconsumer-node · receive count 2
  5. 12:00:11.540Deleteconsumer-node · accepted
  6. 12:00:14.251Deleteconsumer-python · failed, 13 s after its receive

Each recorded step shows an interval, from just before its SQS request was sent to when the response came back. SQS acted somewhere inside it.

A step is drawn above another only when an observed fact proves it came first, and the page names that fact under the step:

Source The page says
clock A's interval ended before B's started
process_order In one process, A's call returned before B's call was issued
receive_count Two receives on one queue, and SQS gave A the lower receive count
sqs_timestamps The send and the first receive, and SQS's SentTimestamp is earlier than its ApproximateFirstReceiveTimestamp

Two markings to know:

  • Dashed border with an inferred badge: Trailbeam worked this step out; nothing recorded it. It shows a time window and a reason instead of a time. See Recorded and inferred steps.
  • Dashed box with a concurrent badge, "concurrent, order unknown": no fact orders these steps. Inside the box each step is a bar on one time scale, sorted by request start, which is not necessarily the order SQS acted in. Expect it around a send and its first receive: an idle consumer was already waiting in a long poll, and ElasticMQ stamps both SQS timestamps in the same millisecond. See Reading a timeline.

A receive from a Lambda wrapper shows only "no later than" one time, because the wrapper sees the event after Lambda received the message.

4Step 4: Explore the demo scenarios#

The demo produces each of these on a schedule:

Look for What happened Guide
Deliveries 2 (1 ok), first and last seen about equal The consumer set the visibility timeout to 0 on delivery 1, the timeout expired, and delivery 2 was deleted Visibility timeouts
Deliveries 2 (1 ok), last seen about 13 s after first seen A slow consumer ran past the 10 s visibility timeout. Delivery 2 went to another consumer, which deleted it. The first consumer's late delete then failed with ReceiptHandleIsInvalid, because ElasticMQ rejects a stale handle Visibility timeouts
Deliveries 4 (1 ok), after about a minute A poison message: three receives on orders, a move to orders-dlq reported by SQS, then a receive and a delete there Dead-letter queue
/duplicates Messages delivered more than once Duplicate processing
/stuck Messages with no final delete after 60 s (the demo sets TRAILBEAM_STUCK_AFTER_S to 60) Stuck messages

Every message page shows the notice for the published default receipt-hash key. That's expected in the demo. To run it with your own key:

shell
TRAILBEAM_RECEIPT_HASH_KEY=$(openssl rand -hex 32) docker compose -f demo/compose.yaml up -d --build --wait

The same data as JSON#

shell
port=${TRAILBEAM_DEMO_PORT:-4318}
id=$(curl -s "http://127.0.0.1:$port/api/messages?limit=1" | python3 -c 'import json,sys; print(json.load(sys.stdin)["messages"][0]["message_id"])')
curl -s "http://127.0.0.1:$port/api/messages/$id" | python3 -m json.tool
curl -s "http://127.0.0.1:$port/api/health"   # "status": "ok", span counts, "max_receive_counts": {"orders": 3}

The fields are in the HTTP API.

5Step 5: Stop the demo#

shell
docker compose -f demo/compose.yaml down      # keeps the timelines, on the volume trailbeam-demo_trailbeam-data
docker compose -f demo/compose.yaml down -v   # deletes them too

Next steps#