The pan tracker bench setup, with the Jetson Orin Nano in its case, the gearmotor on its mount, the NUCLEO-H723ZG and jumper hub, the 3D-printed driver and CAN enclosures, a bench supply and a Saleae
In progresssince Oct 2026

Jetson + STM32 pan tracker

The goal: a camera on a motor that finds an AprilTag and keeps it centered (pan only). A Jetson Orin Nano will run the vision in ROS 2, and an STM32H723 will run the motor in a 1 kHz closed loop under FreeRTOS, shutting the motor off if the Jetson stops responding. The two already talk over CAN at 1 Mbps.

  • STM32H723
  • FreeRTOS
  • C
  • Jetson Orin Nano
  • ROS 2
  • CAN
  • Python
p99 control tick jitter
77 ns
CAN to the Jetson, 0 errors
1 Mbps
camera, 0 dropped frames
720p60
motor stop with braking
37 ms

Done

  • Jetson on JetPack 7.2.1 and ROS 2 Jazzy, IMX219 camera
  • STM32 bring-up at 520 MHz: FreeRTOS tasks, logging, UART CLI
  • CI: host tests and firmware build on every push
  • 1 kHz control tick, 77 ns p99 jitter
  • Board clock moved from the ST-LINK’s RC oscillator to its crystal (0.43% fast to +5 ppm)
  • CAN link to the Jetson over a USB-C cable, 10,000/10,000 round trips
  • Motor open loop: VNH5019 driver, encoder, current sense, fault latch, braking stop
  • 3D-printed enclosures for the motor driver and CAN transceivers
  • Nuc-hatch shield rev A designed in KiCad and ordered

Next

  • Nuc-hatch bring-up when the boards arrive
  • Firmware for the HOME, END+/-, AUX and e-stop inputs, then homing
  • Motor update in the 1 kHz control tick, then velocity and position loops
  • Safety: heartbeat timeout and watchdog
  • ROS 2 tracking pipeline
  • Measured latency and step response
PrototypeDec 2025

fretListener

Real-time guitar note and fret recognition from audio. Play a note and it shows the pitch, the string and the fret, live. The same pitch can be played in up to three places on the neck, so a note CNN finds the pitch first, then a small string-disambiguator CNN picks the position by its timbre. Everything is built from scratch: the recording tool, a self-recorded dataset, the training pipeline and two realtime GUI apps.

  • Python
  • PyTorch
  • torchaudio
  • librosa
  • CNN
  • Tkinter
hand-recorded clips, about 2 hours
12,192
note accuracy, random split
99.8%
note accuracy, guitar it never heard
94.7%
two-string disambiguation
98–100%
fretListener after hearing a D3: the note model is 98.8% sure of the pitch, the E/A/D disambiguator puts it on the low E string at fret 10, and a red dot marks that spot on the fretboard

Done

  • Recording tool: prompts for a position, records takes, slices them at each onset into labeled clips
  • Six datasets across Stratocasters and an Ibanez RG550, frets 0–12
  • Note CNN for 37 pitches, E2 to E5
  • String disambiguators for each neck region
  • Live fretboard listener and a scale and arpeggio practice trainer
  • Every training run logs its command, settings, curves and per-class accuracy

Next

  • A proper held-out test set, plus testing every model on a guitar it’s never heard
  • About 10× more clips per region for the 3-string disambiguators
  • Chords, bends and distortion (clean single notes only so far)