Incorporate audio as positional input

This one’s a little out there, but it comes from a player who got feedback from his coach that he’s short-hopping basically all his shots. I immediately started thinking how to track his progress towards taking more balls at their apex or falling.

Detecting a tight short-hop is too hard for the CV. I see it misclassify them as volleys all the time. But they have a distinct sound profile - that double-strike of the court then your paddle in close succession. I could track his progress by the time delta between those two amplitude spikes.

Knowing bounce and strike times by sound could greatly improve those positional metrics.

In practice, this won’t be easy. In my recordings, there’s a cacophony of strikes from nearby courts. You have to combine it with visual info, so you only pay attention to audio very near the visual strike point. And the amplitude spike of the court is barely detectable to my ear, and not at all on the far court. It might require additional frequency filters on that small window around the visual bounce point to extract ball noise. And there’s the speed of sound issue, but I imagine it’s either negligible, or we can measure distance from camera to the visual strike point to compensate for it.

The tighter we get positional tolerances the better, IMO. Each level opens up a new class of stat we can track.