Fix patch_off_object

[scottcanoe]

Problem Statement

In a small percentage of runs (<1%), Monty gets stuck bouncing between off-object observations early in the run. These experiments timeout after max_monty_steps with no matching steps performed. These timeouts occur mostly, but not exclusively, in experiments with distant agents. So far I've focused on distant agents. Here's a quick overview of what I've observed and done.

In distant agents experiments, get_good_view is called at the beginning of an episode. It moves the agent to a reasonable distance from the object and orients the agent so that the view finder's central pixel is on-object. However, the view finder's central area is not a perfect proxy for sensor module 0 (a patch), and this method does not guarantee that the patch will be on-object, even if the view finder is.

1. Because of the differences in resolutions/zooms of the patch and view finder, the central view finder's pixel is covered by about 10 pixels in the patch's image. If the view finder is aimed directly on an object's edge, the patch's central pixel can end up slightly off-object. If the agent goes the wrong way to try and find the object and ends up even further off-object, then the system gets stuck in a loop, bouncing back and forth between these two off-object observations, repeatedly calling fixme_undo_last_action. Example (object is "banana"):

2. A semantic map can contain "holes", and it can be problematic if the agent is oriented directly onto it. For example, the "chain" object has (real) topological holes in the links. If the agent is oriented onto a hole, the agent may then jumped to an off-object location, reverse this action, and ultimately get stuck in a fixme_undo_last_action loop. In another case, there was a small (spurious) hole in the object's semantic map of the knife, and in this episode, the system also got stuck in a fixme_undo_last_action loop. Here's the patch's semantic map on the part of the knife while holes:

Solutions

I've made several changes here.

1. Where get_good_view was called (at the beginning of an episode and after a jump), it is now called twice, first using the view finder and then sensor module 0. The second call allows for finer orienting movements that ensure that the central sensor patch ends up on-object.
2. Allowing the agent to move forward on the second pass with the sensor patch caused problems, so I added a parameter to get_good_view to facilitate disabling move-forward actions. Solutions (1) and (2) are the primary fix that solved problem (1).
3. I shrunk the size of the smoothing kernel in find_location_to_look_at to reduce the likelihood of orienting onto a semantic hole. The previous kernel was quite large, and the smearing of semantic information made it difficult to discourage orientation towards semantic holes. This fixed problem (2).
4. I've also added a fix that resets an LM's current_mlh between episodes. This wasn't affecting the system's behavior, but it did cause incorrect logging of these patch_off_object episodes.

These changes eliminated patch_off_object episodes for benchmark experiments tested except for the multi-object experiment and surface agent experiments. Those will have to be tackled in a future PR.

Benchmark updates for the 10- and 77-object experiments have been included in this PR with no substantial changes to report.

[scottcanoe]

Tagging @hlee9212 for visibility.

[scottcanoe]

Also noting here that I don't know what's causing the spurious semantic holes to sometimes show up (like in the "knife" example above), but I'll plan to take a closer look at that while working on the upcoming semantic sensor PR.

[nielsleadholm]

Really nice, thanks for the fix and the great documentation. Just added a function suggestion.

[nielsleadholm]

I wonder if this could be made a bit "dryer", and the underlying assumptions of the central sensor patch a bit more visible, by having an outer function, maybe called something like multi_step_get_good_view (or a better name), which takes no arguments (since we are hard-coding these anyways). This function then calls:

self.get_good_view("view_finder")
for patch_id in ("patch", "patch_0"):
    if patch_id in self._observation["agent_id_0"].keys():
        self.get_good_view(patch_id, allow_translation=False)
        break

which could be used on both lines 487 and 738.

[scottcanoe]

How about get_good_view_with_patch_refinement? It's a long name, but multi-step to me sounds like it could refer to multiple steps taken with the same sensor (as is partly done already with move forward).

[nielsleadholm]

Yeah good point, I like it!

[scottcanoe]

Updated.

[nielsleadholm]

@vkakerbeck I'm curious from when you've examined YCB objects in the past, whether you ever encountered holes in objects like the knife blade? I'm assuming this is a defect in the underlying mesh that emerged during the original 3D scanning of the YCB objects, since it's being picked up by the ground truth semantic sensor.

[vkakerbeck]

LGTM, thanks for the fix! Just a few questions so I understand all the changes.

Oh and yes, the YCB objects have tons of weird artifacts like that...

[vkakerbeck]

Why is this number hardcoded now?

[scottcanoe]

I just landed on a kernel size of 2 because it made the smoothed images look "reasonably" smooth, and solved the issue with holes on the object. I can see why we would want to kernel size to scale with the resolution, but then I thought what we really want to scale it by is the physical size on the object between pixels since presumably holes in the object are usually pretty small (in centimeters). This is something I can come back to, but I just left it since it seemed to be giving a good result when I looked at all the images during debugging.

[vkakerbeck]

Wha was the reason these needed to be added now?

[scottcanoe]

Since not resetting current_mlh was causing incorrect logging for patch_off_object episodes, I thought it made sense to include it here. More specifically, an episode could end up with patch_off_object but it would report that the most likely object was whatever the MLH was for the previous episode.

[vkakerbeck]

ah okay, good catch!

[vkakerbeck]

Do you have an idea why performance is worse now in terms of % correct and % used mlh?

[scottcanoe]

I was hoping to come up with a good answer for this, but ultimately I don't really know. I reran with a handful of different random seeds, sometimes recovering the roughly the same accuracy as before. Somehow the very slightly different starting conditions ended up having a noticeable change on accuracy, but I don't have much to say beyond that. It's something I'd like to investigate further in the near future. I'll be redoing all benchmarks for the semantic sensor PR, so maybe there's more I can learn then.