This guide will show how to continuously retrieve image capture data from a PipeOut endpoint and display it in the user interface using the HTML Canvas.

Create a React Component that wraps an HTML Canvas element.

import React, { Component, RefObject } from "react";

export interface CanvasViewProps {
    width: number;
    height: number;
}

class CanvasView extends Component<CanvasViewProps> {
    private _CanvasRef: RefObject<HTMLCanvasElement>;

    constructor(props: CanvasViewProps) {
        super(props);

        this._CanvasRef = React.createRef();
    }

    render() {
        return (
            <canvas 
                ref={this._CanvasRef} 
                width={this.props.width} 
                height={this.props.height} />
        );
    }
}

export default CanvasView;Code language: JavaScript (javascript)

This defines a React Component named CanvasView whose state specifies the width and height of the HTML Canvas that it renders. The component maintains a reference to the rendered HTML Canvas element. It will use this reference to update the content displayed in the Canvas.

Add a public method named ClearFrameImage to the CanvasView.

public ClearFrameImage() {
    const canvas: HTMLCanvasElement | null = this._CanvasRef.current;

    if (canvas != null) {
        const context: CanvasRenderingContext2D | null = canvas.getContext("2d");

        if (context != null) {
            context.fillStyle = "black";
            context.fillRect(0, 0, canvas.width, canvas.height);
        }
    }
}Code language: JavaScript (javascript)

This method sets all the pixels of the content displayed by the Canvas to the color black.

Add a public method named UpdateFrameImage to the CanvasView.

public UpdateFrameImage(sourceData: Uint32Array, columnCount: number, rowCount: number) {
    const canvas: HTMLCanvasElement | null = this._CanvasRef.current;

    if (canvas != null) {
        const context: CanvasRenderingContext2D | null = canvas.getContext("2d", {
            alpha: false
        });

        if (context != null) {
            const frameImageData: ImageData = context.createImageData(
                columnCount,
                rowCount
            );

            const targetImageData: Uint8ClampedArray = frameImageData.data;

            // Copy the pixel data from the source to the target
            for (let pixelIndex = 0; pixelIndex < sourceData.length; pixelIndex++) {
                const sourcePixelValue: number = sourceData[pixelIndex];
                const pixelByteOffset: number = pixelIndex * 4;
                
                targetImageData[pixelByteOffset] = sourcePixelValue & 0xff;               // Red
                targetImageData[pixelByteOffset + 1] = (sourcePixelValue >> 8) & 0xff;    // Green
                targetImageData[pixelByteOffset + 2] = (sourcePixelValue >> 16) & 0xff;   // Blue
                targetImageData[pixelByteOffset + 3] = 0xff;                              // Alpha
            }

            context.putImageData(frameImageData, 0, 0);
        }
    }
}Code language: JavaScript (javascript)

This method will be called to update the content displayed in the Canvas. It takes pixel source data and copies it to the target frame image data buffer retrieved from the HTML Canvas context.

Add the React Component lifecycle componentDidMount method.

componentDidMount() {
    this.ClearFrameImage();
}Code language: JavaScript (javascript)

This will cause the Canvas content to be cleared every time the React Component is mounted.

Add the React Component lifecycle componentDidUpdate method.

componentDidUpdate(
        prevProps: Readonly<CanvasViewProps>,
        _prevState: Readonly<NonNullable<unknown>>,
        _snapshot?: NonNullable<unknown>
    ): void {
        if (this.props.width !== prevProps.width || this.props.height !== prevProps.height) {
            this.ClearFrameImage();
        }
    }Code language: JavaScript (javascript)

This will cause the Canvas content to be cleared whenever the width and or height properties are changed.

Create a React Component named FrontPanel that manages CanvasView.

import React, { Component, RefObject } from "react";

import { 
    IFrontPanel,
    WorkQueue
} from "@opalkelly/frontpanel-alloy-core";

import "./FrontPanel.css";

import CanvasView from "./CanvasView";

export interface FrontPanelProps {
    name: string;
}

export interface FrontPanelState {
    device: IFrontPanel;
}

class FrontPanel extends Component<FrontPanelProps, FrontPanelState> {

    private readonly _WorkQueue: WorkQueue = new WorkQueue();

    private readonly _CanvasViewRef: RefObject<CanvasView>;

    constructor(props: FrontPanelProps) {
        super(props);

        this._CanvasViewRef = React.createRef();

        this.state = { 
            device: window.FrontPanel
        };
    }

    render() {
        return (
            <div className="ControlPanel">
                <CanvasView 
                    ref={this._CanvasViewRef}
                    width={1024} 
                    height={768} />
            </div>
        );
    }
}

export default FrontPanelCode language: JavaScript (javascript)

This defines a React Component named FrontPanel that maintains a reference to the CanvasView component created previously. It will use this reference to update the image displayed by the CanvasView.

Add a private method named RetrieveImage to the FrontPanel component.

private async RetrieveImage(): Promise<void> {
    await this._WorkQueue.Post(async () => {
        const dataSize = 1024 * 768 * 4;     // 1024x768 pixels, 4 bytes per pixel
        const data: ArrayBuffer = await this.state.device.readFromPipeOut(0xa0, dataSize);
        
        if (this._CanvasViewRef.current != null) {
            const imageData = new Uint32Array(data);
            this._CanvasViewRef.current.UpdateFrameImage(imageData, 1024, 768);
        }
    });
}Code language: JavaScript (javascript)

In this example, the data retrieved from the PipeOut endpoint is assumed to be arranged in 1024 columns by 768 rows of pixels. Each pixel is assumed to consist of 4 bytes.

Add a private method named ContinuousImageCaptureLoop to the FrontPanel component.

private _IsStopPending: boolean = false;
private _TimeoutId?: number;
private _CurrentOperation?: Promise<void>;

private async ContinuousImageCaptureLoop(): Promise<void> {
    const start: number = performance.now();

    await this.RetrieveImage();

    const elapsed: number = performance.now() - start;

    if (!this._IsStopPending) {
        const delay: number = 33 - elapsed;

        this._TimeoutId = window.setTimeout(
            () => {
                this._CurrentOperation = this.ContinuousImageCaptureLoop();
            },
            delay > 33 ? delay : 33
        );
    }
}Code language: JavaScript (javascript)

This method will cause the operation that retrieves and displays the image to occur periodically. It attempts to ensure that an update operation is initiated every 33 milliseconds.

Add the React Component lifecycle componentDidMount and componentWillUnmount methods.

componentDidMount(): void {
    this._CurrentOperation = this.ContinuousImageCaptureLoop();
}

async componentWillUnmount(): Promise<void> {
    this._IsStopPending = true;

    if (this._TimeoutId !== undefined) {
        clearTimeout(this._TimeoutId);
        this._TimeoutId = undefined;
    }

    await this._CurrentOperation;

    this._IsStopPending = false;
}Code language: JavaScript (javascript)

These methods will cause the ContinuousImageCaptureLoop to start automatically when the React Component is mounted and stop it when the component will unmount.

import React, { Component, RefObject } from "react";

export interface CanvasViewProps {
    width: number;
    height: number;
}

class CanvasView extends Component<CanvasViewProps> {
    private _CanvasRef: RefObject<HTMLCanvasElement>;

    constructor(props: CanvasViewProps) {
        super(props);

        this._CanvasRef = React.createRef();
    }

    componentDidMount() {
        this.ClearFrameImage();
    }

    componentDidUpdate(
        prevProps: Readonly<CanvasViewProps>,
        _prevState: Readonly<NonNullable<unknown>>,
        _snapshot?: NonNullable<unknown>
    ): void {
        if (this.props.width !== prevProps.width || this.props.height !== prevProps.height) {
            this.ClearFrameImage();
        }
    }

    render() {
        return (
            <canvas 
                ref={this._CanvasRef} 
                width={this.props.width} 
                height={this.props.height} />
        );
    }

    public ClearFrameImage() {
        const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
    
        if (canvas != null) {
            const context: CanvasRenderingContext2D | null = canvas.getContext("2d");
    
            if (context != null) {
                context.fillStyle = "black";
                context.fillRect(0, 0, canvas.width, canvas.height);
            }
        }
    }

    public UpdateFrameImage(sourceData: Uint32Array, columnCount: number, rowCount: number) {
        const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
    
        if (canvas != null) {
            const context: CanvasRenderingContext2D | null = canvas.getContext("2d", {
                alpha: false
            });
    
            if (context != null) {
                const frameImageData: ImageData = context.createImageData(
                    columnCount,
                    rowCount
                );
    
                const targetImageData: Uint8ClampedArray = frameImageData.data;
    
                // Copy the pixel data from the source to the target
                for (let pixelIndex = 0; pixelIndex < sourceData.length; pixelIndex++) {
                    const sourcePixelValue: number = sourceData[pixelIndex];
                    const pixelByteOffset: number = pixelIndex * 4;
                    
                    targetImageData[pixelByteOffset] = sourcePixelValue & 0xff;               // Red
                    targetImageData[pixelByteOffset + 1] = (sourcePixelValue >> 8) & 0xff;    // Green
                    targetImageData[pixelByteOffset + 2] = (sourcePixelValue >> 16) & 0xff;   // Blue
                    targetImageData[pixelByteOffset + 3] = 0xff;                              // Alpha
                }
    
                context.putImageData(frameImageData, 0, 0);
            }
        }
    }
}

export default CanvasView;Code language: JavaScript (javascript)

import React, { Component, RefObject } from "react";

import { 
    IFrontPanel,
    WorkQueue
} from "@opalkelly/frontpanel-alloy-core";

import "./FrontPanel.css";

import CanvasView from "./CanvasView";

export interface FrontPanelProps {
    name: string;
}

export interface FrontPanelState {
    device: IFrontPanel;
}

class FrontPanel extends Component<FrontPanelProps, FrontPanelState> {

    private readonly _WorkQueue: WorkQueue = new WorkQueue();

    private readonly _CanvasViewRef: RefObject<CanvasView>;

    private _IsStopPending: boolean = false;
    private _TimeoutId?: number;
    private _CurrentOperation?: Promise<void>;

    constructor(props: FrontPanelProps) {
        super(props);

        this._CanvasViewRef = React.createRef();

        this.state = { 
            device: window.FrontPanel
        };
    }

    componentDidMount(): void {
        this._CurrentOperation = this.ContinuousImageCaptureLoop();
    }
    
    async componentWillUnmount(): Promise<void> {
        this._IsStopPending = true;
    
        if (this._TimeoutId !== undefined) {
            clearTimeout(this._TimeoutId);
            this._TimeoutId = undefined;
        }
    
        await this._CurrentOperation;
    
        this._IsStopPending = false;
    }

    render() {
        return (
            <div className="ControlPanel">
                <CanvasView 
                    ref={this._CanvasViewRef}
                    width={1024} 
                    height={768} />
            </div>
        );
    }

    private async RetrieveImage(): Promise<void> {
        await this._WorkQueue.Post(async () => {
            const dataSize = 1024 * 768 * 4;     // 1024x768 pixels, 4 bytes per pixel
            const data: ArrayBuffer = await this.state.device.readFromPipeOut(0xa0, dataSize);
            
            if (this._CanvasViewRef.current != null) {
                const imageData = new Uint32Array(data);
                this._CanvasViewRef.current.UpdateFrameImage(imageData, 1024, 768);
            }
        });
    }

    private async ContinuousImageCaptureLoop(): Promise<void> {
        const start: number = performance.now();

        await this.RetrieveImage();

        const elapsed: number = performance.now() - start;

        if (!this._IsStopPending) {
            const delay: number = 33 - elapsed;

            this._TimeoutId = window.setTimeout(
                () => {
                    this._CurrentOperation = this.ContinuousImageCaptureLoop();
                },
                delay > 33 ? delay : 33
            );
        }
    }
}

export default FrontPanelCode language: JavaScript (javascript)

.ControlPanel {
    display: flex;
    flex-direction: column;
    background-color: white;
    border-radius: 8px;
    padding: 10px;
    gap: 8px;
    margin: 10px;
}Code language: CSS (css)