CDT and Parallella

Multicore Debugging for the Masses

Andreas Olofsson (Adapteva)

Marc Khouzam (Ericsson)

About us


  • Andreas Olofsson
    • Founder and CEO of Adapteva
    • Creator of Parallella

  • Marc Khouzam
    • CDT Committer, lead for Debug
    • Working on Multicore Debugging since the fall of 2010

Agenda

  • Parallella and Epiphany
  • Debugging Parallella with CDT
  • Possible next steps
  • Other news for CDT Debug
  • Q&A

An Epiphany Introduction

Epiphany Architecture

Massive Parallelism Has Arrived!

Massive Parallelism
  • How in the ^**&#@ are we doing to program AND DEBUG this??

The Parallella Computer

Parallella computer

Debugging for Massive Parallelism

Debugging issues

Debugging Parallella with CDT

  • Ericsson (CDT) and Adapteva (Parallella) collaboration
  • Goals:
    • Open-source multicore debug solution for Parallella
    • Open example for multicore debug in CDT
    • Easy access to multicore debugging for all
    • Collaboration and exposure to improve multicore debug

Previous Debug Solution for Epiphany

  • Single-core traditional hardware debug using GDB
  • Load program on a core and debug it
  • No global perspective
  • Needs one session per core to deal with entire system
  • Does not scale past 16 cores, and even then

Implemented Enhancements


  • Single session to debug the entire Epiphany
  • Connecting to an already running system

  • A debug thread is a single core
  • A debug process is a group of cores running the same binary
  • Traditional debugging for each core

  • Epiphany Multicore Visualizer

  • Support for Non-stop mode

Debugging the main chip

Debug of Parallella main chip
  • Remote Debug of ARM chip (Ubuntu) with GDB/gdbserver
  • Traditional debug operations for ARM program

Multicore Debugging the Epiphany

Debug of Epiphany chip
  • Remote Multicore Debug of Epiphany with e-GDB/e-server
  • Traditional debug operations for all Epiphany cores

Multicore Debugging Parallella

  • Debug using both sessions
Debug of Parallella main chip

Scaling issues

Debug view not scaling



  • Debug view with 64 cores
  • Unmanageable!
  • Need Visualization

Generic Multicore Visualizer

Generic Multicore Visualizer
  • Monitor entire system at a glance
  • Control program execution from Visualizer
  • Synchronized with Debug view

Epiphany Multicore Visualizer

  • Hardware layout
  • Cores and network
  • Process distribution
  • Load of cores
  • Load of network

Epiphany Multicore Visualizer Layout

Epiphany Visualizer Layout



  • Actual hardware layout
  • Core IDs
  • Color shows binary loaded
  • Selection of cores
  • Status bar info

Program Distribution and Load

Program Distribution



  • Layout with network
  • Load per core
  • Load per network link

64-core Layout

Network Load

64-core Load

Network Load

Possible next steps

Status reporting

  • Running state per core in Visualizer

Grouping

  • GDB IT sets support
  • Run Control operations on group
  • Breakpoints limited to group

User-selectable cores to debug

Selecting cores to debug
  • Attach to a group/process to debug it
  • Debug view only shows attached groups
  • Attaching directly from Visualizer

Debugging thousands of cores

  • 4096 cores

Thousands of cores mockup

Debugging thousands of cores

Possible solutions

  • Zooming
  • Use of scrollbars or mouse drag
  • Use of colors to pin-point an issue
  • Automatic issue detection

  • More study needed, more collaboration

Visualizer Zooming

Before zooming After zooming
  • Zoom directly
  • Pin & Clone
    • Multiple Visualizer views
    • One high-level view
    • One or more zoomed views

Visualizer Pin & Clone

Visualizing both chips
  • The entire Parallella: ARM and Epiphany chips

Other news for CDT Debug

Return Values

Display of Return Values
  • After a Step-Return
  • Top element of Variables view
  • Removed at next execution
  • Plan to generalize for Step-Over

Trace Control View

Tracing Experiment

  • Graphical view
  • Navigation Scrollbar
  • Circular Buffer
  • Disconnected Tracing
  • Optional automatic refresh
Examining Tracing Result

Dynamic-Printf

Dynamic Printf Type

  • No re-compiling
  • No re-deploying
  • Debugger can disconnect
Dynamic Printf Properties

Hardware and Temporary Breakpoints

Breakpoint type
  • Through Add Breakpoint or Breakpoint Properties

New Launch UI

New Launch UI

Standalone CDT Debugger

> run-gdbstandalone.sh [-b <build_log>] -e <binary> [arg1 ... argn]
Standalone Debugger

Resources

Themes

Use the below links to change themes
Default - Sky - Sky2 - Blood - Beige - Simple - Serif - Night - Moon - Solarized