RICH Dataset joint selection

[mertkiray]

Hello,

Thank you for your great work. I wanted to ask how did you decide on which joints to select from the RICH dataset.
In this line you selected some joints from the available joints but I wonder how did you decide on these?

For example in SMPLX Unity project they select these joints.

Do you think how one can try to obtain these joints from your work? Is it feasible to create the datasets using those joints and finetune the model or what do you suggest?

Thanks!

[isarandi]

Hi! Thanks for your interest! What joints are you missing for the SMPL-X body? I aimed to include all body joints.
Note though that I'm currently working on a method to estimate full SMPL/SMPL-X models, so this will not be an issue.

[mertkiray]

So the selected joints for metrabs are:
['pelv', 'lhip', 'rhip', 'bell', 'lkne', 'rkne', 'spin', 'lank', 'rank', 'thor', 'ltoe', 'rtoe', 'neck', 'lcla', 'rcla', 'head', 'lsho', 'rsho', 'lelb', 'relb', 'lwri', 'rwri', 'jaw', 'leyehf', 'reyehf', 'lmiddle1', 'rmiddle1', 'nose', 'reye', 'leye', 'rear', 'lear', 'lto2', 'lto3', 'lhee', 'rto2', 'rto3', 'rhee', 'lthu', 'lmiddle', 'rthu', 'rmiddle']

Which I think translates to these joints from SMPLX joints named from this link:
['pelvis', 'left_hip', 'right_hip', 'spine1', 'left_knee', 'right_knee', 'spine2', 'left_ankle', 'right_ankle', 'spine3', 'left_foot', 'right_foot', 'neck', 'left_collar', 'right_collar', 'head', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'jaw', 'left_eye_smplhf', 'right_eye_smplhf', 'left_middle1', 'right_middle1', 'nose', 'right_eye', 'left_eye', 'right_ear', 'left_ear', 'left_big_toe', 'left_small_toe', 'left_heel', 'right_big_toe', 'right_small_toe', 'right_heel', 'left_thumb', 'left_middle', 'right_thumb', 'right_middle']

So this translates to these joints are missing in the SMPLX Unity project:

 Not found: nose, right_eye, left_eye, right_ear, left_ear, left_big_toe, left_small_toe, left_heel, right_big_toe, right_small_toe, right_heel, left_thumb, left_middle, right_thumb, right_middle

Which is fine as this is a limitation of the SMPLX fbx model in the unity.

But metrabs also lacks these joints that is available in the SMPLX fbx model:

Not found: left_index1, left_index2, left_index3, left_middle2, left_middle3, left_pinky1, left_pinky2, left_pinky3, left_ring1, left_ring2, left_ring3, left_thumb1, left_thumb2, left_thumb3, right_index1, right_index2, right_index3, right_middle2, right_middle3, right_pinky1, right_pinky2, right_pinky3, right_ring1, right_ring2, right_ring3, right_thumb1, right_thumb2, right_thumb3

Which looks like only lacking joints for metrabs <-> Unity model is most of the hand joints.

So my questions are:

1- Does my conversion between metrabs SMPLX joint names and original SMPLX joint names looks correct?
2- Do you know the difference between right_eye and right_eye_smplhf joints for example?
3- Do you plan to support the hand joints in the metrabs or do you think this is a limitation of the skeleton models and probably won't yield good results?
4- Can you explain the reasoning behind the naming convention? For example what is the meaning behind rto2, rto3 etc. Do you think it makes sense to create a documentation that maps these into a more accustomed joint naming convention?

Thank you :) If my question is not clear please let me know I can explain further.