Unstable Prediction when Transfer Training

[cchen225]

Hello,

I've been having issues where my resulting prediction on a mouse is static, flickers wildly on the y axis, and is also completely flattened on the y axis but the shape of the mouse seems fine on x and z axes (attached gif is zoomed out, and the image is zoomed in very far). Although I'm currently transfer training to a much larger arena setup and with 6 cameras rather than 3, I've had this same issue training on a setup more akin to what is displayed in the paper with a small 12" diameter cylindrical arena.

I believe this may be a calibration issue as after calibration it seems that in some views for some frames, the hand labeled keypoints are offset from where they should be. However, even after recalibrating I still seem to be getting the same issue. For extrinsics I use the L-frame method labeling 5 reference post coordinates for each view and for intrinsics I use the provided checkerboard method. The reprojections all seem fine.

I trained the COM UNet over 1200 epochs and also trained DANNCE over 600 epochs. Also there were only 52 labeled frames but the first few frames are all hand labeled such that I would at least expect the first few hundred frames to be stable, but this wasn't the case.

What could be the cause of this?