A new kind of project

I did something new and fun this fall: I wrote a computer science paper. It's an analysis of the conditions under which a device equipped with a camera and an accelerometer can identify its position relative to two observed landmarks with known positions. Except for a measure zero set of singular cases with infinitely many solutions, there are always at most two solutions for device positions (this was previously known), and I found necessary and sufficient conditions for there to be a single solution. In particular, if the two landmarks are at the same altitude, there is always a single solution, unless the device is at the same altitude as the landmarks.

I implemented the algorithm on a phone (code here). In the screenshot, the markers 1 and 2 are landmarks, identified and outlined in green with OpenCV library code, and then the phone uses their positions and the accelerometer data to predict where the control markers 3 and 4 are on the screen, outlining them in red.

For someone like me who does some philosophy of science, it was an interesting experience to actually do a real experiment and collect data from it.

I am planning at some point to try to implement the algorithm using infrared LEDs under a TV and the accelerometer and infrared camera inside a right Nintendo Switch joycon. To that end, over the last couple of days I've reverse-engineered two of the joycon infrared camera blob identification modes.