Debugging Synergy Applications: Tracing with Percepio’s Tracealyzer

In the last post, we started to examine how we could use the ThreadX trace capabilities to generate trace data that can be viewed using Percepio’s Tracealyzer. Tracealyzer visualizes the events that occur while the software is executing which provides developers with the ability to peer into the silicon and see how their software is actually running. Tracealyzer could be described as the oscilloscope for software developers. In this post, we will examine several visualizations available in Tracealyzer that can help developers understand their application.

The first visualization that we started to look at in the last post is the primary Tracealyzer screen which displays statistical information in addition to tasks and user events that can all be filtered based on the developers’ interests:

Now this first view is extremely powerful, but it doesn’t necessarily tell the whole story on how the application code is behaving. As you might recall from Transitioning to ThreadX: Managing Thread Priorities, as the number of tasks in a system increasing, the theoretical maximum CPU load in order for all those tasks to meet their deadlines is approximately 70%. Tracealyzer includes a CPU Load graph that can be used to see whether the CPU load is balanced or whether there are potential issues with CPU load that could lead to response time delays. An example can be seen below:

Developers can even track communication flow through their application by using the communication flow graph. This feature allows developers to see where communication and signaling is occurring in their application in a very easy to read manner. For example, below the reader can see an application where serial information is received via a UART. Each byte is received, and a semaphore is used to signal that event which then queues the data to be processed by the Comm Task.

My absolute favorite view within Tracealyzer is the User Events view. Since every embedded application is different, there will be different data sets that will be interesting to different developers. For example, monitoring a particular sensors value or monitoring the stack and heap usage can be visualized by using the User Event view. Data visualization can help developers see patterns in the way their system behaves and when those patterns are broken, it’s possible that a bug is hiding in those areas or that a low occurrence event is happening in the system.

The visualizations that we have just explored are just the tip of the iceberg. There are several dozen different visualizations that developers can use to explore their applications behavior. In many cases, it is useful to use multiple visualizations together in order to fully understand how the application is behaving. Before we end today’s blog, I’d like to provide you with a few best practices to consider when using Tracealyzer:

  • Setup Tracealyzer at the start of a project, not when a problem or issue is encountered
  • Take a baseline trace with every repository commit. This will act like an application fingerprint that can be compared to if an issue is discovered in later code revisions
  • Use the snap-shot trace option when debugging a small section of code like a function or a single task
  • Use streaming trace to get the larger picture on application behavior during testing or determining application performance
  • Use the View Filter box to filter out events that are not under investigation in order to declutter the visualizations. Too much information can overload an engineer’s ability to spot issues

Until next time,

Live long and profit!

Professor_IoT

Hot Tip of the Week: The entire manual for Tracealyzer for ThreadX is available online. Developers interested in reading several RTOS-agnostic user stories can find several below:

  • Unexpected timing anomaly A customer had an issue with a periodic task not running as expected, sometimes it was delayed for unknown reasons.
  • Increased Response Time A customer used two instances of Tracealyzer to compare trace data before and after a problematic code change.

Read Stop Guessing, a white paper authored by Percepio founder and CEO Johan Kraft, that adds a lot more details about the capabilities of Tracealyzer and also covers RTOS debugging.