Depth-guided generation
Generate posed RGB views from depth context and target cameras.
atlasChisel generates posed RGB views from optional posed depth context, a
required text prompt, a set of target cameras, and optional posed photographs
of the scene that the generated views stay consistent with. A context frame
may also carry the view's known pixels, a mask, and a depth mask, in which
case the task keeps the known pixels and depth and generates the rest of the
view.
Account availability:
POST /api/v2/tasks:atlasChiselrequires access to this task on your account.
When to use it
Use this task when you have a graybox, blockout, or another geometric layout
expressed as posed depth images and want Marble to render coherent RGB views at
the same cameras. You can also leave contextFrames empty to generate from the
prompt alone.
The request combines:
contextFrames: zero or one posed frame per target camera, in target order, each carrying depth, the view's known image, or both, optionally with a mask and a depth mask.sourceFrames: zero or more posed RGB photographs of the scene, each an image and its camera.targetCameras: 1 to 32 pinhole cameras for the views to generate.prompt: a required description of the environment, materials, lighting, and style.modelParameters: optionalseedandnumStepsoverrides.voxelSizeandvoxelGridOrientation: optional controls for simplifying the depth geometry onto a grid of cubes (voxels).
atlasChisel serves one model and does not accept a model selector. The
model keeps each frame's known depth and image, limited to the maskAsset
when one is set, and generates the rest. Start with the documented defaults
and change one sampling or voxel control at a time.
Keep the prompt, seed, and camera set with the operation so a result can be
reproduced.
Camera and depth requirements
Every context and target camera must be a 1280 x 720 pinhole camera. Each context depth raster must also be exactly 1280 x 720 and aligned with its camera. The public endpoint does not resize or crop depth maps for you.
When context is present, provide exactly one frame per target camera in the
same order. Each frame carries camera and at least one of depth and
imageAsset:
depth.depthAssetfor a linear-depth EXR, ordepth.logDepthAssetfor an inverted log-depth PNG (see Encode context depth)imageAsset, the view's known pixels at 1280 x 720- optionally
maskAsset, an 8-bit grayscale PNG that is white where the frame's image and depth are known and black where the model is free to generate; frames with images and no masks are fully known, and frames with masks and no images keep their depth inside the mask only - optionally
depth.maskAsset, an 8-bit grayscale PNG on the depth grid that is white where the frame's depth is known and black where the model is free to generate the geometry
A depth.maskAsset masks the depth apart from the image, and the frame's
maskAsset then marks only the image, so it needs an imageAsset beside it.
Without a depth.maskAsset, the maskAsset marks the depth too. For
example, to keep the full depth of a scene with the known image limited to
one object, send an all-white depth.maskAsset and the object's mask as
maskAsset. Both masks read a pixel as known above half intensity.
A frame with an imageAsset and no depth keeps its image inside the
maskAsset and lets the model generate the geometry along with the rest of
the view. This is the frame shape atlasMasked takes, so
a view-completion request moves to atlasChisel by sending its frames as
contextFrames, with the same cameras as targetCameras and a prompt that
describes the finished scene (atlasChisel's prompt enhancer reads text
only, so an edit instruction would be rewritten without the images). Leave
voxelSize unset with these frames: there is no depth to quantize, and the
request is rejected.
Send each of depth, imageAsset, maskAsset, and depth.maskAsset on
every context frame or on none; a request that mixes frames with and without
one is rejected.
Do not send depth.confidenceAsset to atlasChisel; the endpoint rejects it
instead of silently ignoring it. Keep every camera and depth image in one
coordinate convention and scale. See
Cameras and posed images for the pose
convention, and Euler angles for
converting the Chisel editor's rotation fields.
Three.js EXRLoader: If client code decodes or rewrites a linear-depth EXR before upload, account for
EXRLoader's bottom-up CPU row order. The task expects depth aligned to the camera's top-left image grid. See Decode linear-depth EXR rows for a reference accessor.
Encode context depth
Each depth pixel is planar depth: the distance from the camera to the surface measured along the camera's optical axis (view-space z). It is not the length of the ray through the pixel. Use the same distance unit as the camera positions; the service normalizes the scene scale, so any consistent unit works.
If your renderer or raycaster reports distance along the ray, convert each
pixel before encoding. For ray distance d at pixel center (u, v):
Send each frame's depth in one of two encodings:
| Field | File | Pixel value |
|---|---|---|
depth.depthAsset | Single-channel EXR (image/x-exr) | Planar depth in world units |
depth.logDepthAsset with zMin and zMax | 8-bit grayscale PNG (image/png) | Inverted log depth: 255 is zMin, darker is farther |
A log-depth PNG value p from 1 to 255 decodes to:
To encode, set zMin and zMax to the frame's nearest and farthest surface
depth, write round(255 * (1 - ln(z / zMin) / ln(zMax / zMin))), and clamp
each surface to at least 1 so the farthest one does not collide with the
no-surface value 0. zMin and zMax must be finite with
0 < zMin <= zMax. A 16-bit PNG is rejected, because the encoding is defined
on 8-bit values. If a log-depth PNG has an unsupported bit depth, the operation
fails with INVALID_INPUT before model inference. Correct the encoding before
submitting another request.
Mark sky and empty space
Mark every pixel with no surface behind it, such as sky, as no surface:
- in a log-depth PNG, write
0; - in an EXR, write
0,NaN, or infinity.
The service reads any depth that is not finite and positive as no surface.
Atlas's own Chisel editor sends depth this way: planar view-space z, with 0
wherever nothing was drawn.
Do not model the sky as a distant sphere or dome. Any finite positive depth is a surface, so the model reads the dome as far geometry, not as sky. The model also sees depth on a log scale normalized over every context frame in the request together, so a dome kilometers away becomes the farthest surface in the scene and squeezes all real geometry into a narrow band of the depth range.
Keep generated views consistent with photographs
When you already have posed photographs of the scene, pass them in
sourceFrames. Each entry carries an imageAsset and its camera, the same
posed RGB frame shape atlasMasked takes, with no maskAsset and no depth.
The model uses these photographs as visual references for the materials,
colors, and lighting of surfaces visible from the target cameras.
- Cameras must be in the same coordinate convention and scale as the target cameras and any context depth, but they need not match a target camera.
- Any resolution works. Each photograph is cover-resized and center-cropped onto 1280 x 720 before it reaches the model, so keep the subject away from the edges of a photograph with a different aspect ratio.
- Request sizes must satisfy
3 * contextFrames.length + sourceFrames.length + targetCameras.length <= 128. With one context frame per target, 32 targets leave no room for source photographs; 30 targets leave room for 8 photographs.
Prompt and voxel controls
prompt must be non-empty whether or not context depth is present. Leave
enhancePrompt at true to expand your prompt into a detailed scene
description, or set it to false to use your wording unchanged.
voxelSize sets the side length of each cube used to simplify the depth
geometry. It uses normalized scene units, not your input's distance units:
Marble rescales the scene so the 90th percentile of inverse depth is
approximately 1. Omit it or set it to 0 to keep the original geometry. The
recommended range is 0 to 0.4; values around 0.05 to 0.15 are useful starting
points for blockouts.
With a nonzero voxelSize, voxelGridOrientation: "world" (the default)
aligns the grid to the request's world axes so walls aligned with those axes
stay flat. "gauge" aligns the grid to the average camera pose after the scene
is recentered.
Handle the result
The task returns a long-running operation. On success, frames contains one
generated image and camera per target camera, in request order.
Set returnDepth: true only with at least two target cameras whose centers are
not all identical:
- A request with one target camera and
returnDepth: trueis rejected with HTTP422. Context depth frames and source photographs do not count toward the two targets, so a one-camera run cannot return depth. - Targets that all share one center (a pure-rotation rig) are accepted. With no
anchoring photograph, the operation can still report success with a depth
scale of
0.0orNaN, because it has no translation baseline.
Returned depth is reconstructed from the generated targets, not copied from
the context. A source photograph joins that reconstruction as an anchor, and
gets no depth back, only when its image is 1280 x 720 and its camera's
principal point is centered. The depth comes back from the generation call
itself, and context frames that carry an image and no maskAsset join that
reconstruction too, up to 19 of them, with the depth under a depth.maskAsset
left out. With more than 19 such frames, or with any sourceFrames, the
generated views are reconstructed in a separate step afterwards.
Example completed output
After the operation reports done: true, its task-specific result is in
operation.response:
When returnDepth is true, each generated frame also carries a depth
buffer. Its depth can contain NaN where the reconstruction found no reliable
geometry; see
Missing values in returned depth.
Use the Operations polling pattern, and open the atlasChisel reference for the live request schema, defaults, and response types.