Skip to content

Commit

Permalink
Don't clear scalars in face tracking example (#4950)
Browse files Browse the repository at this point in the history
This is what caused the deadlocks/crashes we were after (which I'm glad
it did -- those have been fixed in #4949).

Even though they are not a problem anymore, I'm not sure why they're
here to begin with? Clearing scalars does nothing unless you
specifically intend to use them with latest-at queries (i.e. not plots,
which isn't the case here), otherwise it's just inefficient with no
upside (which I'd rather not teach users). Am I missing something?
:thinking:

---

I've updated the code to only clear faces when they disappear.

I might come up with a follow up PR soon so that the timeseries view interprets those clears as plot breakpoints.
  • Loading branch information
teh-cmc authored Jan 30, 2024
1 parent c2688ad commit 2c60146
Showing 1 changed file with 18 additions and 3 deletions.
21 changes: 18 additions & 3 deletions examples/python/face_tracking/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,13 +237,28 @@ def detect_and_log(self, image: npt.NDArray[np.uint8], frame_time_nano: int) ->
if self._video_mode
else self._detector.detect(image)
)
rr.log("video/landmarker/faces", rr.Clear(recursive=True))
rr.log("reconstruction/faces", rr.Clear(recursive=True))
rr.log("blendshapes", rr.Clear(recursive=True))

def is_empty(i): # type: ignore[no-untyped-def]
try:
next(i)
return False
except StopIteration:
return True

if is_empty(zip(detection_result.face_landmarks, detection_result.face_blendshapes)):
rr.log("video/landmarker/faces", rr.Clear(recursive=True))
rr.log("reconstruction/faces", rr.Clear(recursive=True))
rr.log("blendshapes", rr.Clear(recursive=True))

for i, (landmark, blendshapes) in enumerate(
zip(detection_result.face_landmarks, detection_result.face_blendshapes)
):
if len(landmark) == 0 or len(blendshapes) == 0:
rr.log(f"video/landmarker/faces/{i}/landmarks", rr.Clear(recursive=True))
rr.log(f"reconstruction/faces/{i}", rr.Clear(recursive=True))
rr.log(f"blendshapes/{i}", rr.Clear(recursive=True))
continue

# MediaPipe's keypoints are normalized to [0, 1], so we need to scale them to get pixel coordinates.
pts = [(math.floor(lm.x * width), math.floor(lm.y * height)) for lm in landmark]
keypoint_ids = list(range(len(landmark)))
Expand Down

0 comments on commit 2c60146

Please sign in to comment.