by skunxicat

Realtime Monitoring with asciigraph

asciinema

Sometimes you just want to see what’s happening without opening another browser tab. I’ve been using variations of this SQS monitoring loop for a long time when I need to observe and experiment with queue behavior:

while :
   do aws sqs get-queue-attributes \
    --queue-url "$sqs" \
    --output json \
    --attribute-names All \
    | jq -r '.Attributes 
    | .ApproximateNumberOfMessages as $messages 
    | .ApproximateNumberOfMessagesNotVisible as $invisible
    | .ApproximateNumberOfMessagesDelayed as $delayed
    | .VisibilityTimeout as $timeout 
    | .DelaySeconds as $delay
    | $messages + " " + $invisible + " " + $delayed + " " + $timeout + " " + $delay
    ' | xargs -n5 bash -c 'echo -e "\r\033[1A\033[0KMessages: $0 InFlight: $1 Delayed: $2 VisibilityTimeout: $3 DelaySeconds: $4"'
    sleep 2
  done

It works - the line updates in place showing current values like this:

Messages: 23 InFlight: 5 Delayed: 0 VisibilityTimeout: 900 DelaySeconds: 0

But static numbers don’t tell the story as clearly as seeing patterns over time.

Adding Visual Context

The same data becomes much more useful when you can see patterns over time. Here’s how to pipe those SQS metrics into asciigraph:

Simple SQS Monitoring

#!/bin/bash

# Set your queue URL
SQS_QUEUE_URL="https://sqs.region.amazonaws.com/account/queue-name"

# Fetch SQS metrics and format for asciigraph
while true; do
    aws sqs get-queue-attributes \
        --queue-url "$SQS_QUEUE_URL" \
        --attribute-names All \
        --output json \
        --query 'Attributes' \
    | jq -r '{
        messages: .ApproximateNumberOfMessages,
        processing: .ApproximateNumberOfMessagesNotVisible,
        delayed: .ApproximateNumberOfMessagesDelayed
    } | [.messages, .processing, .delayed] | map(tonumber) | join(",")'
    
    sleep 2
done | asciigraph -r \
    -w 80 \
    -sn 3 \
    -sl "Queue Depth,Processing,Delayed" \
    -sc "red,green,blue"

That’s it. Run it and watch your queue behavior in real-time.

What You See

The graph shows three key SQS metrics:

  • Queue Depth (red) - Messages waiting to be processed
  • Processing (green) - Messages currently being processed (not visible)
  • Delayed (blue) - Messages scheduled for future delivery

When things are working well:

  • Queue depth spikes briefly then drops to zero
  • Processing messages appear as queue depth decreases
  • Delayed messages follow their schedule

Why This Works

Numbers in CloudWatch dashboards don’t show the story. Seeing the real-time flow helps you understand:

  • How quickly your consumers process messages
  • Whether your visibility timeout is appropriate
  • If delayed messages are behaving as expected
  • Queue behavior during traffic spikes

Dependencies

  • aws-cli configured with appropriate permissions
  • jq for JSON processing
  • asciigraph for terminal visualization
# Install asciigraph
go install github.com/guptarohit/asciigraph/cmd/asciigraph@latest

Conclusion

Sometimes the best debugging tools are the simplest ones. A few lines of bash and asciigraph can give you more insight than staring at CloudWatch dashboards.

Real-time terminal monitoring is perfect for:

  • Understanding queue behavior during experiments
  • Debugging message processing issues
  • Watching system behavior during load tests
  • Quick validation without opening browser tabs

Sometimes you just need to see what’s happening.


This monitoring approach is part of our cloudless philosophy - using simple, composable tools to understand complex systems.