Skip to content

Commit

Permalink
Enhance rendering options and add new features
Browse files Browse the repository at this point in the history
- Added new color options (RED, GREEN, BLUE) to the Color enum.
- Renamed `set_material` to `keep_material` for clarity.
- Updated function parameters to improve usability.
- Added support for additional shading options.
- Improved handling of shadow settings.
- Introduced new images for various rendering examples.
- Enabled visual debugging with `blenderproc debug`.
  • Loading branch information
Matthias Humt committed Dec 6, 2024
1 parent 441007d commit 36ff474
Show file tree
Hide file tree
Showing 17 changed files with 115 additions and 21 deletions.
99 changes: 95 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,103 @@ The following options can be added to:
* render the mesh as **depth** image: `--depth`
* render the mesh as **point cloud** from projected **depth** image: `--pcd --depth`

| Mesh | Point cloud | Depth |
|---------------------------------|-----------------------------------------------------|-------------------------------------------------------|
You can test you render settings using any of the `Blender` primitives (`monkey`, `cube`, `sphere`, `cone`,
`cylinder`, ...) as the first argument.

| Mesh | Point cloud | Depth |
|----------------------------|--------------------------|----------------------------------|
| ![mesh](examples/mesh.png) | ![pcd](examples/pcd.png) | ![mesh_depth](examples/mesh.png) |
| `--obj_path suzanne` | `--mesh_as_pcd` | `--depth` |
| `--obj_path suzanne` | `--pcd` | `--pcd --depth` |

## Basic Options

* `--resolution`: Change the resolution of the rendered image (default: `512x512`)
* `--normalize`: Normalize and center the object to fit into a unit cube (`True` by default)
* `--rotate`: Rotate the object using `XYZ` Euler angles (default: `0,0,-35`)
* `--show`: Show the rendered image in a window (`True` if `--save` is not provided)

## Additional Options

Some examples of additional options to customize the rendering are shown below.

### Color

To change the color of the rendered object, use the `--color` option using either any of the predefined colors (e.g.
`pale_violet`), choosing from those at random (`random_color`), a completely random color (`random`), or a
three-tuple of RGB values in range 0-1 (e.g. `(0.8,0.5,0.2)`).
Point clouds can additionally be colored using any of the
[`matplotlib` colormaps](https://matplotlib.org/stable/users/explain/colors/colormaps.html). The background color can be
changed using the `--bg_color` option.

| Mesh | Point cloud | Background |
|----------------------------------|--------------------------------|--------------------------------------|
| ![mesh](examples/mesh_color.png) | ![pcd](examples/pcd_color.png) | ![mesh_depth](examples/bg_color.png) |
| `--color bright_blue` | `--pcd --color viridis` | `--bg_color pale_turquoise` |

### Background

By default, the background is transparent. To change this, use the `--bg_color` option as shown above. Additionally,
`--notransparent` can be used to render the backdrop object.

| Backdrop | Colored backdrop |
|--------------------------------|---------------------------------------|
| ![mesh](examples/backdrop.png) | ![pcd](examples/backdrop_colored.png) |
| `--notransparent` | `--notransparent --bg_color pale_red` |

## Light

The default light intensity for meshes is `bright` (`0.7`) and `very_bright` (`1.0`) for point clouds. Use a value
between 0 and 1 or `very_dark`, `dark`, `medium`, `bright`, or `very_bright` to change the light intensity.

| Very Dark | Dark | Medium |
|---------------------------------|---------------------------|------------------------------------|
| ![mesh](examples/very_dark.png) | ![pcd](examples/dark.png) | ![mesh_depth](examples/medium.png) |
| `--light very_dark` | `--light dark` | `--light medium` |

### Shadow

Shadows are rendered by default. To disable them, use the `--noshadow` option. To make the shadow softer, use
`--shadow soft` or --shadow=hard` for a harder shadow.

| Soft shadow | Hard shadow | No shadow |
|-----------------------------------|----------------------------------|--------------------------------------|
| ![mesh](examples/shadow_soft.png) | ![pcd](examples/shadow_hard.png) | ![mesh_depth](examples/noshadow.png) |
| `--shadow soft` | `--shadow hard` | `--noshadow` |

### Shading

The default shading is `flat` for meshes and `smooth` for point clouds. To change this, use the `--shade` option.

| Smooth shading | Auto-smooth shading |
|------------------------------|----------------------------------|
| ![mesh](examples/smooth.png) | ![pcd](examples/auto-smooth.png) |
| `--shade smooth` | `--shade auto` |

### Animations

To create an animation, use the `--animate` option. The `--frames` option can be used to specify the number of frames
(default: `72`). To keep transparency, which is not supported by GIF, use `.mp4` as file extension.

| Turn (default, loops) | Tumble |
|----------------------------|-----------------------------|
| ![mesh](examples/turn.gif) | ![pcd](examples/tumble.gif) |
| `--animate` | `--animate tubmle` |

### Further Options

Some additional useful options include:

* `--roughness`: Change the roughness of the object. Meshes use `0.5` and point clouds use `0.9` by default.
* `--ao`: Apply ambient occlusion
* `--fstop`: Enable depth of field with a given f-stop
* `--keep_material`: Keep your custom material (only works for `.blend` files)
* `--verbose`: Enable verbose logging during execution
* `--seed`: Set a seed for the random number generator. Useful for random colors or the tumble animation.

## Debugging

## Examples
`BlenderProc` supports visual debugging inside `Blender` using `blenderproc debug` instead of `blenderproc run`.
Adding `--debug` will further disable rendering and only setup the scene.

## Credits

Expand Down
Binary file added examples/auto-smooth.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/backdrop.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/backdrop_colored.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/bg_color.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/medium.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/mesh_color.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/noshadow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/pcd_color.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/shadow_hard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/shadow_soft.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/smooth.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/tumble.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/turn.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/very_dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 20 additions & 17 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@
class Color(Enum):
WHITE = (1, 1, 1)
BLACK = (0, 0, 0)
RED = (1, 0, 0)
GREEN = (0, 1, 0)
BLUE = (0, 0, 1)
PALE_VIOLET = (0.342605, 0.313068, 0.496933)
PALE_TURQUOISE = (0.239975, 0.426978, 0.533277)
PALE_GREEN = (0.165398, 0.558341, 0.416653)
Expand Down Expand Up @@ -62,7 +65,6 @@ class Shadow(Enum):
MEDIUM = 'medium'
SOFT = 'soft'
VERY_SOFT = 'very_soft'
NONE = 'none'


class Engine(Enum):
Expand Down Expand Up @@ -100,10 +102,10 @@ def run(obj_path: str | Tuple[str, str],
gravity: bool = False,
animate: Optional[Animation | str | bool] = None,
shade: Shading | str = Shading.FLAT,
set_material: bool = True,
keep_material: bool = False,
color: Optional[Tuple[float, float, float] | Color | str] = None,
roughness: Optional[float] = None,
mesh_as_pcd: bool = False,
pcd: bool = False,
point_size: Optional[float] = None,
point_shape: Shape | str = Shape.SPHERE,
cam_location: Tuple[float, float, float] = (1.5, 0, 1),
Expand All @@ -116,7 +118,7 @@ def run(obj_path: str | Tuple[str, str],
transparent: bool | float = True,
look: Optional[Look | str] = None,
exposure: float = 0,
shadow: Optional[Shadow | str] = None,
shadow: Optional[Shadow | str | bool] = None,
ao: Optional[bool | float] = None,
engine: Engine | str = Engine.CYCLES,
noise_threshold: Optional[float] = None,
Expand All @@ -141,10 +143,10 @@ def run(obj_path: str | Tuple[str, str],
gravity: Whether to enable physics-based gravity simulation
animate: Animation type to apply (turn, tumble) or False for static render
shade: Shading style to apply to the object
set_material: Whether to apply default materials
keep_material: Whether to keep the custom material or apply the default one
color: Color for the object in RGB format
roughness: Material roughness value
mesh_as_pcd: Treat mesh vertices as point cloud
pcd: Create a point cloud by sampling points from the surface of the mesh
point_size: Size of points when rendering point clouds
point_shape: Shape to use for point cloud visualization
cam_location: Camera position in 3D space
Expand Down Expand Up @@ -187,11 +189,12 @@ def run(obj_path: str | Tuple[str, str],
noise_threshold=noise_threshold or 0.01,
samples=samples or 100)

point_shape = Shape.SPHERE if animate and Animation(animate) in [Animation.TURN, Animation.TUMBLE] else point_shape
animate = Animation.TURN if isinstance(animate, bool) else Animation(animate) if animate else None
point_shape = Shape.SPHERE if animate in [Animation.TURN, Animation.TUMBLE] else point_shape
obj = setup_obj(obj_path=obj_path,
normalize=normalize,
mesh_as_pcd=mesh_as_pcd,
set_material=set_material,
mesh_as_pcd=pcd,
set_material=not keep_material,
color=color,
cam_location=cam_location,
roughness=roughness,
Expand All @@ -207,14 +210,15 @@ def run(obj_path: str | Tuple[str, str],
gravity = False

offset = np.array([0, 0, -0.05])
if animate and Animation(animate) is Animation.TUMBLE:
if animate is Animation.TUMBLE:
if gravity:
logger.warning("Disabling gravity for tumble animation.")
gravity = False
offset = np.array([0, 0, -0.6])

if gravity or (backdrop and not (transparent and shadow and Shadow(shadow) is Shadow.NONE)):
shadow_strength = Strength.OFF if shadow and Shadow(shadow) is Shadow.NONE else Strength.MEDIUM
no_shadow = isinstance(shadow, bool) and not shadow
if gravity or (backdrop and not (transparent and no_shadow)):
shadow_strength = Strength.OFF if no_shadow else Strength.MEDIUM
setup_backdrop(obj=obj,
shadow_strength=shadow_strength,
transparent=transparent,
Expand All @@ -228,7 +232,7 @@ def run(obj_path: str | Tuple[str, str],
light = light or (Light.BRIGHT if is_mesh else Light.VERY_BRIGHT)
light = light if isinstance(light, float) else Light[light.upper()].value if isinstance(light, str) else light.value
make_lights(obj=obj,
shadow=shadow or (Shadow.MEDIUM if is_mesh else Shadow.SOFT),
shadow=(Shadow.MEDIUM if is_mesh else Shadow.SOFT) if shadow is None or no_shadow else Shadow(shadow),
light_intensity=light)
make_camera(obj=obj,
location=cam_location,
Expand All @@ -237,7 +241,6 @@ def run(obj_path: str | Tuple[str, str],

show = show or save is None
if animate:
animate = Animation.TURN if isinstance(animate, bool) else Animation(animate)
save = Path(animate.value).with_suffix('.gif') if save is None else Path(save)
make_animation(obj=obj,
save=save,
Expand Down Expand Up @@ -783,10 +786,10 @@ def init_mesh(obj: bproc.types.MeshObject,
obj: The BlenderProc mesh object to initialize.
shade: The shading type to apply to the object. Can be 'FLAT', 'SMOOTH', or 'AUTO'.
"""
logger.debug(f"Setting shading mode to {Shading(shade).name}")
if Shading(shade) is Shading.AUTO:
obj.add_auto_smooth_modifier()
else:
obj.set_shading_mode(Shading(shade).name)
obj.set_shading_mode(Shading(shade).name)


def _pointcloud_with_geometry_nodes(links,
Expand Down Expand Up @@ -1229,7 +1232,7 @@ def render(bg_color: Optional[Tuple[float, float, float] | Color | str] = None,
image = Image.alpha_composite(background, image)

if save:
image.save(save.parent / f'{save.stem}_{i:04d}{save.suffix}')
image.save(Path(save).resolve())
if show:
image.show()
images.append(image)
Expand Down

0 comments on commit 36ff474

Please sign in to comment.