diff --git a/proposals/Readme.md b/proposals/Readme.md index ba0f886..e82cd2b 100644 --- a/proposals/Readme.md +++ b/proposals/Readme.md @@ -19,3 +19,4 @@ See the [OpenUSD Proposals Status](https://github.com/orgs/PixarAnimationStudios - [Namespace Editing](https://github.com/PixarAnimationStudios/OpenUSD-proposals/blob/main/proposals/namespace_editing/README.md) ![Status:Published](https://img.shields.io/badge/published-green) ([current PRs and issues](https://github.com/PixarAnimationStudios/USD-proposals/pulls?q=is%3Apr++label%3Ausd-namespace-editing+)) - [Multiple Animation support](multiple-animations/README.md) ![Status:Draft](https://img.shields.io/badge/draft-blue) ([current PRs and issues](https://github.com/PixarAnimationStudios/USD-proposals/pulls?q=is%3Apr++label%3Ausd-multiple-animation-support+)) - [Image plane support](image-planes/README.md) ![Status:Draft](https://img.shields.io/badge/draft-blue) ([current PRs and issues](https://github.com/PixarAnimationStudios/USD-proposals/pulls?q=is%3Apr++label%3Ausd-image-planes-support+)) +- [Phyical Cameras](physical-cameras/README.md) ![Status:Draft](https://img.shields.io/badge/draft-blue) ([current PRs and issues](https://github.com/PixarAnimationStudios/USD-proposals/pulls?q=is%3Apr++label%3Ausd-physical-cameras-support+)) diff --git a/proposals/physical-cameras/README.md b/proposals/physical-cameras/README.md new file mode 100644 index 0000000..bef5eb2 --- /dev/null +++ b/proposals/physical-cameras/README.md @@ -0,0 +1,388 @@ + +![Status:Draft](https://img.shields.io/badge/Draft-blue) +## Physical Cameras for USD + +## Contents +- [Introduction](#introduction) +- [Use Cases](#use-cases) +- [Relationship to Image Plane proposal](#Relationship-to-Image-Plane-Proposal) +- [Schema](#Proposed-PhysicalCamera-schema) + +## Introduction + +This schema usefully bridges the description of physical cameras to USD scenes. + + +https://github.com/mmp/pbrt-v4/blob/master/src/pbrt/cameras.h + +PBRT defines Orthographic, Perspective, Spherical, and Realistic cameras. + +## Use Cases + +### Interesting Looking Models + +The fx company has got a set of lenses in its +library. These lenses are described by individual USD files that can be +layered into a USD scene that records details of a shoot using those lenses. + +The camera and rig are composed of classes in this schema and are custom for +every shot, but the lenses are reused between shots. Every morning the lenses +are characterized, and if new values are recorded, they are added to the +individual lens files as variants, and the current variant is set to latest. + +Before shots are archived, the variant in use is set appropriately, although +during the shoot, the lens variant will typically be set to "current". + +### Likeable Puppets + +The stop motion animation company has got a large collection of camera bodies +and lenses. These bodies and lenses travel around and are mixed and matched +all day long during shoots on multiple sets, and when images are recorded, +the combination for any given shot is constructed via layering those cameras +and bodies, and associated with the image plane data, so that the plates and +unique camera set ups always travel together. + +### Bellam Blvd Previs + +The previs company has got a camera kit they use in all their work. This kit +consists of a single USD file with 8 variants, including Doll House, Over The +Shoulder, and Out in the Desert. These variants are selected on a shot by shot +basis, and characteristics of the individual lenses and bodies are overridden +as needed. + +### Beloved Lens Co + +The Beloved Lens Co has sent us a box with it's latest offerings, Darling 16, +Darling 24, a Gamin 8, and a Super Snoot 300. They've also kindly supplied USD +files for these lenses, complete with both a custom lens model that properly +describes the unique characteristics of their lenses, and also a best match +PhysicalCameraLensModelPinhole for use in realtime previs scouting. + + +## Relationship to Image Plane proposal + +Image Plane proposal is here: [https://github.com/PixarAnimationStudios/OpenUSD-proposals/tree/main/proposals/image-planes](https://github.com/PixarAnimationStudios/OpenUSD-proposals/tree/main/proposals/image-planes) + +These should be complementary, and any information related to elements +in this proposal, should be migrated to this proposal. Otherwise, the Image +Plane proposal specifically describes the mapping of a capture image to an +image plane, and the Physical Camera schema describes the mapping of objects +in a scene to the image plane. So that's the pivot between them. + +## Proposed PHysicalCamera schema + +``` +class PhysicalCamera "PhysicalCamera" ( + doc = """Physically modeled camera. + + The physical camera is a primitive meant to be a target for construction of + a physical camera model through applied, and multiply applied schemas. + + The physical camera is a GeomXformable, and thus inherits the transform + schema. The physical camera also inherits the UsdGeomImageable schema, + which provides facilities for controlling visibility and rendering of the + camera. + + The physical camera provides a number of attributes that are meant to be + authored by a user or by a DCC import process. These attributes are + generally meant to be authored once, and then left alone. + + The concept is that the PhysicalCamera is just like a real camera that may + be used and measure on a real life stage, and it's attributes recorded in + a usd file. A real camera is a body, with a lens, and a film back, all + interchangeable. + + This base schema then, describes the body, which is little more than a + transform, and is otherwise a target for a collection of film backs, and + lenses, which are themselves schemas that are applied to the PhysicalCamera. + + In addition to transform properties, the base camera includes attributes + describing the aperture at the front of the body. The camera body currently + models the aperture and shutter as a non-interchangeable component. + + The aperture describes both the shape of the pupil of the aperture and also + the shutter, and the distance of the physical aperture from the film back. + + Currently the shape of the aperture is modeled as a straight sided polygon. + In the future, an arbitrary mask could be specified to use in convolution + or raytracing. + + shutter angle is the shutter duration * frame rate * 360 degrees, and not + computed here. + + """ +) +{ + float flange_distance = 52.0 ( + doc = '''distance from film back to lens flange, in mm''' + ) + float shutter_open = 0.0 ( + doc = '''offset in seconds from start of exposure''' + ) + float shutter_duration = 0.02 ( + doc = '''duration of exposure, in seconds, the default is 1/50th of + a second''' + ) + float iris = 6.25 ( + doc = '''the default aperture is 6.25mm, corresponding to f8 for a + 50mm lens''' + ) +} + +class PhysicalCameraSensor "PhysicalCameraSensor" ( + doc=""" + The PhysicalCameraSensor schema describes the film back of the camera. The + film back is the sensor that is exposed to light, and is the primary + determinant of the field of view of the camera, by describing the geometry + of the plane where an image is to be resolved. + + Since a camera may have multiple film backs, the PhysicalCameraSensor is + a multi-apply schema, and may be applied multiple times to a PhysicalCamera. + + The sensor's coordinate system has its center at (0, 0) and its bounds + are -1 to 1. + The enlarge property is a multiplicative value; and shift is an additive + value in the sensor's coordinate system. + + Shift and enlarge can be used to create projection matrices for + subregions of an image. For example, if the default sensor is to be + rendered as four tiles, a matrix for rendering the upper left quadrant + can be computed by setting enlarge to { 2, 2 }, and + shift to millimeters{-17.5f}, millimeters{-12.25f}. + + The default sensor aperture is as for 35mm DSLR. + + A handedness of -1 indicates a left handed camera. + + The PhysicalCameraSensor does describe the electrical or + photochemical characteristics of a sensor, it's geometric. + The electrical or photochemical characteristics are to be described + either here or in another class within this schema. + """ +) +{ + float2 shift = (0.0, 0.0) ( + doc = '''shift in millimeters''' + ) + float2 enlarge = (1.0, 1.0) ( + doc = '''enlarge multiplicative''' + ) + float2 aperture = (36.0, 24.0) ( + doc = '''aperture in millimeters''' + ) + float handedness = 1.0 ( + doc = '''-1 for left handed, 1 for right handed''' + ) +} + +class PhysicalCameraLens "PhysicalCameraLens" ( + doc=""" + The PhysicalCameraLens schema describes the lens of the camera. The lens + is the optical system that focuses light onto the film back. + + Since a camera may have multiple lenses, the PhysicalCameraLens is + a multi-apply schema, and may be applied multiple times to a PhysicalCamera. + + Fundamentally, the lens is a mapping from a point in space to a point on + the film back, with an associated transmittance value, and so the + PhysicalCameraLens provides some common measurements, but it's expected that + the explicit modeling will occur in a prim referenced from the PhysicalCameraLens. + + float aperture_distance; // distance from image plane to aperture, mm + unsigned int shutter_blades; // default 7 + + """ +) +{ + rel lensModelPrim ( + doc = '''The lens model prim''' + ) + float iris = 6.25 ( + doc = '''The default iris corresponds to f8 for a 50mm lens''' + ) + float aperture_distance = 50.0 ( + doc = '''distance from image plane to aperture, mm''' + ) + unsigned int shutter_blades = 7 ( + doc = '''default 7''' + ) + float fstop = 8.0 ( + doc = '''fstop''' + ) + float2 clippingRange = (1, 1000000) ( + doc = '''Near and far clipping distances in scene units''' + ) +} + +class PhysicalCameraLensModelPinhole "PhysicalCameraLensModelPinhole" ( + doc=""" + The PhysicalCameraLensModelPinhole schema describes a pinhole lens, and is A + sample schema that may be refered to via the lensModelPrim relationship. + + A and B are on the sensor plane. + + There is a point at infinity, O, on the axis of the lens, whose parallel + rays converge on B. + a + ---------------------------+---------------+ A + b| | | + O--------------------------+---------------+ B + c| | | + ---------------------------+---------------+ + infinity lens sensor plane + C + + h = A-B is half the sensor plane aperture. + f = b-B is the focal length + + There is a point C at infinity, whose parallel rays through a, b, and c, + converge on the edge of the sensor plane, at A. + + The field of view of the lens, at infinity, can therefore be approximated by + + fov = 2 atan((h/2) / f) + + Given a field of view, the assumption of a focus at infinity, and a sensor + aperture, the focal length may be calculated accordingly: + + f = tan(fov/2) / (h/2) + + an anamorphic lens' horizontal field of view must take squeeze into account: + + fov = 2 atan((s * h/2) / f) + + The hyperfocal distance is useful for determining what ranges are in focus. + + If a camera is focused at infinity, objects from the hyperfocal distance to + infinity are in focus. + + O----------------H----------------------------| + Infinity hyperfocal distance Sensor plane + + If a camera is focussed at the hyperfocal distance, then objects + from 1/2 H to infinity are in focus. + + O----------------H----------H/2---------------| + Infinity hyperfocal distance Sensor plane + + Given a focus distance, and the hyperfocal distance, the in focus range is + given by + + Dn = hd / (h + (d - f)) + Df = hd / (h - (d - f)) + + Given a circle of confusion, such as 0.002mm as might be typical for + 35mm film, the hyperfocal distance can be determined. + + h = f^2/(fstop * CoC) + + Note that this struct does not model lens imperfections such as focus + breathing, vignetting, distortion, flaring, spherical aberration or + chromatic aberration. Nor does it model field curvature or diffraction. + """ +) +{ + float focal_length = 50.0 ( + doc = '''focal length in millimeters''' + ) + float focus_distance = 1000.0 ( + doc = '''focus distance in millimeters''' + ) + float transmittance = 1.0 ( + doc = '''transmittance''' + ) + float squeeze = 1.0 ( + doc = '''squeeze''' + ) +} + +class PhysicalCameraExposure "PhysicalCameraExposure" ( + doc = """ + Describes the exposure settings common to still and cinema cameras. + + These settings are used to scale the brightness of the illumination incident + on the camera system. The scaling factor, S, is computed as: + S = exposure_time * iso * 2^exposure_compensation * responsivity_compensation / (100 * fnumber^2) + """ + inherits = + customData = { + token apiSchemaType = "singleApply" + } +) +{ + float physcam:exposure:fnumber = 1.0 ( + doc = """The lens aperture f-number""" + ) + + float physcam:exposure:iso = 100 ( + doc = """The speed rating of the sensor or film""" + ) + + float physcam:exposure:exposure_time = 1.0 ( + doc = """The exposure time in seconds""" + ) + + float physcam:exposure:exposure_compensation = 0.0 ( + doc = """Adjusts the brightness of the image by a factor of + 2^exposure_compensation + """ + ) + + float physcam:exposure:responsivity_compensation = 1.0 ( + doc = """Scalar multiplier that represents the overall scaling of incident + exposure by the camera system, including loss by the lens and scaling + by the camera ISP to generate an output signal + """ + ) +} + +class PhysicalCameraSensorRGBResponsitivity "PhysicalCameraSensorRGBResponsitivity" ( + doc = """ + Describes an RGB camera's native colour space in terms of the colour matching functions + for the red, green and blue primaries, and a matrix for transforming from the resulting + camera RGB value to CIE XYZ. For more information on the use of these values see + https://github.com/wetadigital/physlight/blob/main/examples/physlight_camera_model.ipynb + """ + inherits = + customData = { + token apiSchemaType = "singleApply" + } +) +{ + float2[] physcam:responsivity:red = [...default to XYZ X_bar] ( + doc = """Colour matching function for calculating the output of the camera + RGB red value given an incident spectral exposure, by convolution. The CMF + is represented as a list of [wavelength, value] pairs, where wavelength is + in nanometers. Values at intermediate wavelengths are calculated by linear + interpolation, and values at wavelengths not within the domain of the given + samples are calculated by linear extrapolation. + """ + ) + + float2[] physcam:responsivity:green = [...default to XYZ Y_bar] ( + doc = """Colour matching function for calculating the output of the camera + RGB green value given an incident spectral exposure, by convolution. The CMF + is represented as a list of [wavelength, value] pairs, where wavelength is + in nanometers. Values at intermediate wavelengths are calculated by linear + interpolation, and values at wavelengths not within the domain of the given + samples are calculated by linear extrapolation. + """ + ) + + float2[] physcam:responsivity:blue = [...default to XYZ Z_bar] ( + doc = """Colour matching function for calculating the output of the camera + RGB blue value given an incident spectral exposure, by convolution. The CMF + is represented as a list of [wavelength, value] pairs, where wavelength is + in nanometers. Values at intermediate wavelengths are calculated by linear + interpolation, and values at wavelengths not within the domain of the given + samples are calculated by linear extrapolation. + """ + ) + + matrix3d physcam:responsivity:cam_to_xyz = [1 0 0 0 1 0 0 0 1] ( + doc = """3x3 matrix representing the transformation from Camera RGB to + CIE XYZ + """ + ) +} +```