Current product framing
Rasterift is now a hosted FastAPI demo with a bundled default video, upload support, Original/Glyph/Pixel playback, WebSocket transformed streams, FFmpeg audio extraction, and production grid caps for Render. It still does not introduce a DCT, wavelet, residual, or coefficient-domain video syntax. The point of the pipeline is to expose each representation boundary: source container, decoded frame, Glyph grid, Pixel field, adaptive payload, WebSocket message, JavaScript decoder state, display queue, canvas output, audio extraction, cached browser source, and telemetry.
Request and selection model
The browser keeps a compact playback state: selected library id, selected render family, current stream state, volume, frame queue, codec decoder, timing clock, and measured payload statistics. GET /videos returns the bundled startup video plus any uploaded items with dimensions, FPS, frame count, duration, file size, and a source URL. Selecting a row or mode calls POST /select, where FastAPI resolves the id, updates application state, and builds the queue entry that the next WebSocket connection will consume.
The UI has three separate playback surfaces. Original Source uses a native <video> element. Glyph and Pixel hide that element and draw to a canvas from WebSocket frames. The selected mode controls not only rendering, but also grid sizing, codec cell width, audio synchronization, throughput accounting, and which visual layer is visible.
Upload and library lifecycle
Uploads are sent as multipart form data to POST /upload. The server normalizes the filename, stores it under RASTERIFT_DATA_DIR/uploads, probes it with OpenCV, rejects unreadable streams, and returns a stable library id. Deletion is restricted to uploaded files through DELETE /videos; the bundled startup source is protected so the public demo always has something to play.
Original playback also passes through a compatibility layer. If the selected file is already a conservative browser source, the server returns it directly. If it uses a codec that OpenCV can decode but browser video elements may not, /source creates a cached H.264/AAC MP4 under RASTERIFT_DATA_DIR/source_cache, keyed by path, size, and modification time. On Render, durable uploads require a persistent disk mounted at /data; otherwise upload storage is reset when the service restarts or redeploys.
OpenCV frame decode and grid sizing
Rasterift Glyph and Rasterift Pixel both start by opening the selected video with OpenCV. The server reads width, height, FPS, and frame count, then derives rows from the column count and aspect ratio. Glyph compensates for character geometry by using fewer rows than a square-pixel grid would require. Pixel treats each cell as a square color sample, so the same source becomes a denser reduced raster.
Local CLI runs keep the original higher-resolution defaults. The hosted Render entry point applies production budgets: a 24 FPS cap, a Glyph cell budget, and a Pixel cell budget. That keeps square and portrait uploads from expanding into very large transformed streams while preserving the Original, Glyph, and Pixel comparison.
Rasterift Glyph render path
In live Glyph mode, each decoded frame is resized into both a small grayscale matrix and a matching BGR colour grid. Brightness values are quantized into mapper lookup indexes, producing characters that approximate local luminance. The source colour for the same cell is converted from OpenCV BGR into RGB and packed as [char, R, G, B] so the browser can draw each glyph in the correct sampled colour while the invisible text layer preserves selectable character content.
The paper's measured Glyph benchmark remains useful context, but the hosted product measures the selected file directly. Live Colour Glyph carries RGB ink per cell and may use a smaller production grid than local CLI defaults, so the throughput card reports the current stream rather than a fixed benchmark number.
Pixel render path
Pixel mode uses a direct reduced-raster path instead of glyph selection, streaming BGR color cells. The browser receives three bytes per cell, swaps BGR into RGBA order, writes into an ImageData buffer, and paints that buffer with putImageData. CSS stretches the small canvas to the player while preserving hard pixel edges.
Pixel intentionally carries more visual detail than Glyph, so it is capped more aggressively in hosted production. The local default can still use a 450-column grid, but Render scales large uploads to the configured cell budget so Pixel remains playable for visitors.
Adaptive codec and timing
For binary modes, the WebSocket uses an adaptive codec. Each message carries frame_index, a codec tag, and a payload. Tag 0 is a raw framebuffer, tag 1 is a zlib-compressed full framebuffer, and tag 2 is a zlib-compressed delta made from changed cell indexes plus changed cell values. The encoder picks the smallest practical representation for each frame and forces periodic keyframes for resynchronization.
Audio is served separately through /audio, where FFmpeg extracts an MP3 stream from the selected file. The client now primes that audio from the user's play click before the transformed WebSocket stream starts, which avoids browser autoplay blocks for Glyph and Pixel. When audio is available, it becomes the master clock: the visual queue buffers frames, drops frames that are too far behind, and waits on frames that arrive early.
Throughput measurement model
The throughput cards are source-normalized so they match the paper's accounting boundary. Original Source is measured as selected file bytes divided by source duration. Glyph and Pixel are measured as observed visual payload bytes per frame multiplied by the source FPS. Network framing, TCP/IP overhead, TLS, retransmission, browser cache behavior, initialization messages, and audio bytes are intentionally excluded because the paper measures application visual payload rather than total network capture.
This is why the numbers should be read as representation cost, not as a browser download speed test. Original is a compressed MP4 container. Glyph and Pixel are transformed visual streams. Raw RGB is the uncompressed baseline: width times height times three color bytes times FPS. The UI should therefore support the paper's cautious conclusion: reduced raw transport, explicit representation boundaries, and no claim of codec competitiveness.
