NativeImage

The Image component owns display and terminal-protocol selection. NativeImage owns image decode, transformed pixels, and native image handles.

This advanced API is for code that must inspect or transform pixels before display. Use the Image component to load and display an encoded source without direct pixel ownership.

Every materialized image uses top-left, straight-alpha, sRGB RGBA8 pixels. The API is supported in the Bun and Node.js runtimes listed in Runtime and platform support.

Load and dispose an image#

import { NativeImage } from "@opentui/core"

const image = await NativeImage.load("./image.webp")

try {
  const raw = image.raw("rgba8")
  console.log(raw.width, raw.height, raw.stride, raw.data)
} finally {
  image.dispose()
}

raw() returns an owned JavaScript copy. Disposing image does not invalidate that copy.

Accepted sources#

Format detection reads encoded bytes. It does not trust a filename, URL suffix, response header, or file extension.

decode(), fromRgba(), and fromPixels() copy your data into native memory. imageInfo() returns metadata without retaining an image handle. It can decode pixels when validation needs pixel inspection.

For fromRgba() and fromPixels(), stride is the byte distance between row starts. It defaults to width * 4 and must fit a complete row. The input view must contain at least (height - 1) * stride + width * 4 bytes. Padding after the final row is optional.

Pixel import options#

interface PixelImportOptions {
  owner?: ResourceContext
  stride?: number
  format?: "rgba8" | "bgra8"
  alpha?: "straight" | "opaque"
  colorSpace?: "srgb"
}

fromPixels() defaults to format: "rgba8", alpha: "straight", and colorSpace: "srgb". It copies rows from a top-left origin into packed native RGBA8 storage. The same pass converts BGRA to RGBA and detects transparency.

alpha: "opaque" ignores source alpha and writes 255 to every destination alpha channel. It does not unpremultiply RGB values. The API does not support premultiplied alpha or color spaces other than sRGB.

Pass a subarray directly to import part of a mapped buffer. The native call does not retain or change the view. Keep the source stable during the call. After the call returns, you can reuse, detach, or unmap the buffer.

const image = NativeImage.fromPixels(mappedPixels, width, height, {
  format: "bgra8",
  stride: bytesPerRow,
  alpha: "opaque",
})

The image reports format: "raw-rgba" and colorStatus: "explicit-srgb", even for BGRA input.

Load options#

interface ImageLoadOptions {
  owner?: ResourceContext
  signal?: AbortSignal
  fetch?: (input: URL, init?: RequestInit) => Promise<Response>
}

signal cancels file, response-body, or fetch acquisition. An abort throws signal.reason. fetch replaces globalThis.fetch for fetched URLs.

Paths, blobs, and response bodies are fully buffered before native decode. load() rejects a non-success HTTP status before it reads the body.

Choose a Context owner#

Each image belongs to one native Context. The Context owns the image storage and enforces object and lease limits. JavaScript holds a checked handle to the image, not a raw pointer. Set owner in ImageCreateOptions to choose the Context:

interface ImageCreateOptions {
  owner?: ResourceContext
}

decode(), fromRgba(), and imageInfo() accept these options. ImageLoadOptions, PixelImportOptions, and NativeImagePoolOptions also include owner. Pass renderer.nativeScene.resourceContext to create an image in the renderer’s Context.

Without owner, these methods share one automatic Context that starts with 4,096 object slots. Derived images, retained handles, and raw-pixel leases keep that Context alive. The last release destroys it. You control the lifetime of an explicit ResourceContext.

This example releases the raw lease before destroying its Context:

import { NativeImage, ResourceContext } from "@opentui/core"

const owner = new ResourceContext({ objectCapacity: 4, renderCellsMax: 1 })
try {
  const image = NativeImage.fromPixels(Uint8Array.of(255, 0, 0, 255), 1, 1, { owner })
  try {
    const raw = image.takeRaw()
    try {
      console.log(Array.from(raw.data).join(","))
    } finally {
      raw.dispose()
    }
  } finally {
    image.dispose()
  }
} finally {
  owner.destroy()
}

Output:

255,0,0,255

After you destroy an explicit Context, its remaining image wrappers throw on use. Their dispose() calls still complete wrapper cleanup. While a raw-pixel lease is live, owner.destroy() throws NativeError with ContextBusy. Release the lease, then call destroy() again. Read Resources and ownership for checked image limits and transfer rules.

Format behavior#

NativeImage does not expose animation frames or timing. GIF becomes one image. Animated WebP reports an unsupported feature.

Metadata#

image.info() and imageInfo() return ImageInfo:

explicit-srgb means the source supplied supported color metadata or the image came from explicit RGBA pixels. assumed-srgb means OpenTUI treated untagged source values as sRGB.

For PNG, supported sRGB cICP takes precedence. Otherwise, iCCP takes precedence over sRGB, gAMA, and cHRM. Unsupported cICP falls through to lower-priority metadata. OpenTUI converts supported RGB or grayscale ICC monitor profiles to sRGB with Little CMS.

Pixel access#

Both raw result types include data, width, height, stride, format, colorSpace: "srgb", and alpha: "straight".

copyTo() defaults to RGBA8 with stride width * 4. A custom stride must fit a row. The destination must fit all rows.

Transfer native pixels#

const image = await NativeImage.load(new Blob([encodedImage]))

try {
  const raw = image.takeRaw()
  try {
    consumeRgba(raw.data, raw.width, raw.height, raw.stride)
  } finally {
    raw.dispose()
  }
} finally {
  image.dispose()
}

takeRaw() consumes the NativeImage. Later image access throws. image.dispose() is still safe and does nothing.

OwnedRawImage.data aliases native memory. Do not dispose the OwnedRawImage while any code still uses data. You must call OwnedRawImage.dispose(). Repeated calls are safe. It releases the lease and invalidates data and all saved aliases. The lease counts against the Context’s lease-count and lease-byte limits. It keeps an automatic Context alive and blocks destruction of an explicit Context.

takeRaw() needs the only native reference to the image storage. If another retained handle or a same-Context buffer placement also holds the storage, takeRaw() throws an Error. Release those references before you transfer the pixels.

Share or copy a handle#

Dispose every retained, cloned, decoded, and transformed handle separately. Disposing one retained handle does not invalidate another retained handle.

Drawing within the same Context retains immutable storage without copying it. Using an image in another Context clones it once into that Context, and the wrapper caches the clone. dispose() and takeRaw() release the cached clones. A destination buffer that retains a clone does not pin the original pixels. ImageRenderable.source also retains the source image’s storage, so that reference can block takeRaw().

Reuse pixel storage#

NativeImagePool reuses a fixed number of native pixel buffers, called slots, for frames of the same size. Each successful publication copies the pixels into a free slot and returns an immutable NativeImage.

import { ImageRenderable, NativeImagePool } from "@opentui/core"

// The renderer's Context lets the view bind each frame without a copy.
const pool = new NativeImagePool({ width: 640, height: 480, capacity: 3, owner: renderer.nativeScene.resourceContext })
const view = new ImageRenderable(renderer, { width: 80, height: 24 })
renderer.root.add(view)

function onPixels(pixels: Uint8Array): boolean {
  const frame = pool.publishRgba(pixels)
  if (!frame) return false
  try {
    view.source = frame
  } finally {
    frame.dispose()
  }
  return true
}

// Stop the producer before teardown.
view.destroy()
pool.dispose()

The ImageRenderable.source setter retains the frame synchronously. It requests a render when the load settles, so you do not need to call requestRender().

Each publication creates a new JavaScript wrapper and native handle. This prevents source equality checks and terminal-protocol caches from hiding a new frame.

• new NativeImagePool({ width, height, capacity?, owner? }) fixes the dimensions, slot count, and optional Context owner. Capacity defaults to 3 and must be an integer from 1 through 8. Native image dimension and memory limits apply on the first publication.
• publishRgba(pixels, stride = width * 4) copies top-left, straight-alpha, sRGB RGBA8 bytes without retaining caller memory. It accepts row padding. The input must include every row’s pixels, but padding after the final row is optional.
• publishPixels(pixels, options?) accepts the same PixelImportOptions as NativeImage.fromPixels(). It converts RGBA8 or BGRA8 with straight or opaque alpha directly into a reusable slot, without a JavaScript staging buffer. The pool constructor selects the Context owner. A publication’s owner option does not override it.
• The pool allocates slots only as needed, up to capacity * width * height * 4 bytes of pixel storage. Reuse does not allocate another pixel buffer. Encoding and rendering can allocate separately.
• The pool can reuse a slot only after all published handles and same-Context buffer placements release it. Retained images, renderable sources, and buffer snapshots keep their original pixels. Dispose frames when you no longer need them.
• Both publication methods return null if every slot is still in use. They do not allocate pixel storage, queue, block, or replace a frame. Drop the incoming frame or retry after consumers release a slot. Invalid input throws instead of returning null.
• Both methods finish synchronously, with no acquisition token or pending operation to cancel. To cancel input, stop calling them.
• dispose() releases the pool’s references and rejects future publications. Repeated calls are safe. Published frames remain valid and still need disposal. If a native release fails, dispose() throws before it marks the pool disposed. Call dispose() again to retry.
• Pools cannot resize. Create a new pool for new dimensions. Then dispose the old pool.
• takeRaw() cannot transfer a frame while the pool owns its slot. Use raw() or copyTo(), or dispose the pool first.

Transform images#

Operations do not mutate the source. Each successful operation returns a new NativeImage.

Resize kernels are "area", "default", "triangle", "cubic-bspline", "catmull-rom", "mitchell", and "nearest". The default is "area".

extend() defaults all sides to zero and the background to transparent [0, 0, 0, 0]. Each background channel is an integer from 0 through 255.

Composite options default to left: 0, top: 0, blend: "source-over", and opacity: 1. Blend modes are "source-over", "source", and "destination-over". Opacity must be finite and in 0..1. Negative offsets clip the overlay to the base image.

const source = await NativeImage.load("photo.jpg")

try {
  const thumbnail = source.resize({ width: 320 })
  try {
    useImage(thumbnail)
  } finally {
    thumbnail.dispose()
  }
} finally {
  source.dispose()
}

Encoded PNG retention#

ensureEncodedPng() makes encoded PNG data available for low-level native consumers. Raw and transformed images can need an encode at this point. The method returns void and leaves the image usable.

Opaque, orientation-free PNG input can retain its original encoded bytes without decoding pixels immediately. A direct unchanged Kitty placement can use those bytes. Any pixel read, pixel operation, Sixel or block rendering, or Kitty crop, resize, or opacity operation materializes pixels.

This lazy path means corrupt PNG pixel data can pass initial metadata validation and fail when the first pixel path runs.

Errors and limits#

ImageLoadError covers source acquisition. It exposes code, source, and optional HTTP status.

ImageError covers native decode and operations. It exposes numeric status and one of these codes:

• invalid-handle, unsupported-format, unsupported-color-space, or malformed-data
• dimension-limit, memory-limit, invalid-argument, or out-of-memory
• output-too-small, internal-error, or unsupported-feature
• busy for retained native pool slots. Pool publication methods return null instead of throwing this status.

JavaScript option and geometry validation uses TypeError and RangeError. All image methods except dispose() throw after disposal or transfer.

Context failures throw NativeError with a NativeStatus value, such as ObjectLimit, LeaseLimit, or LeaseBytesLimit. These errors are separate from the image codec’s ImageError codes.

OpenTUI keeps the eight most recently used validated ICC profiles and sRGB transforms. The cache key includes complete decompressed profile bytes and RGB or grayscale mode. The native library clears the cache when its final client releases it.

Next#

• Image owns normal display and protocol choice.
• FrameBuffer displays an existing or transformed image.
• Buffer API defines drawImage() placement ownership.
• Lifecycle and cleanup defines native-handle cleanup.
• Runtime and platform support lists native targets.
• API and symbol index lists every image export.