How to use HTML Canvas to Display Image Capture Frames
This guide will show how to continuously retrieve image capture data from a PipeOut endpoint and display it in the user interface using the HTML Canvas.
![](https://docs.opalkelly.com/wp-content/uploads/2024/06/FPAlloy-Camera-Example.png)
Create a CanvasView Component
Create a React Component that wraps an HTML Canvas element.
import React, { Component, RefObject } from "react";
export interface CanvasViewProps {
width: number;
height: number;
}
class CanvasView extends Component<CanvasViewProps> {
private _CanvasRef: RefObject<HTMLCanvasElement>;
constructor(props: CanvasViewProps) {
super(props);
this._CanvasRef = React.createRef();
}
render() {
return (
<canvas
ref={this._CanvasRef}
width={this.props.width}
height={this.props.height} />
);
}
}
export default CanvasView;
Code language: JavaScript (javascript)
This defines a React Component named CanvasView
whose state specifies the width and height of the HTML Canvas that it renders. The component maintains a reference to the rendered HTML Canvas element. It will use this reference to update the content displayed in the Canvas.
Add a Method to Clear the Canvas
Add a public method named ClearFrameImage
to the CanvasView
.
public ClearFrameImage() {
const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
if (canvas != null) {
const context: CanvasRenderingContext2D | null = canvas.getContext("2d");
if (context != null) {
context.fillStyle = "black";
context.fillRect(0, 0, canvas.width, canvas.height);
}
}
}
Code language: JavaScript (javascript)
This method sets all the pixels of the content displayed by the Canvas to the color black.
Add a Method to Update the Content of the Canvas
Add a public method named UpdateFrameImage
to the CanvasView
.
public UpdateFrameImage(sourceData: Uint32Array, columnCount: number, rowCount: number) {
const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
if (canvas != null) {
const context: CanvasRenderingContext2D | null = canvas.getContext("2d", {
alpha: false
});
if (context != null) {
const frameImageData: ImageData = context.createImageData(
columnCount,
rowCount
);
const targetImageData: Uint8ClampedArray = frameImageData.data;
// Copy the pixel data from the source to the target
for (let pixelIndex = 0; pixelIndex < sourceData.length; pixelIndex++) {
const sourcePixelValue: number = sourceData[pixelIndex];
const pixelByteOffset: number = pixelIndex * 4;
targetImageData[pixelByteOffset] = sourcePixelValue & 0xff; // Red
targetImageData[pixelByteOffset + 1] = (sourcePixelValue >> 8) & 0xff; // Green
targetImageData[pixelByteOffset + 2] = (sourcePixelValue >> 16) & 0xff; // Blue
targetImageData[pixelByteOffset + 3] = 0xff; // Alpha
}
context.putImageData(frameImageData, 0, 0);
}
}
}
Code language: JavaScript (javascript)
This method will be called to update the content displayed in the Canvas. It takes pixel source data and copies it to the target frame image data buffer retrieved from the HTML Canvas context.
Clear the CanvasView Content when the Component Mounts
Add the React Component lifecycle componentDidMount
method.
componentDidMount() {
this.ClearFrameImage();
}
Code language: JavaScript (javascript)
This will cause the Canvas content to be cleared every time the React Component is mounted.
Clear the CanvasView Content when the Component Resizes
Add the React Component lifecycle componentDidUpdate
method.
componentDidUpdate(
prevProps: Readonly<CanvasViewProps>,
_prevState: Readonly<NonNullable<unknown>>,
_snapshot?: NonNullable<unknown>
): void {
if (this.props.width !== prevProps.width || this.props.height !== prevProps.height) {
this.ClearFrameImage();
}
}
Code language: JavaScript (javascript)
This will cause the Canvas content to be cleared whenever the width and or height properties are changed.
Create a Parent Component for the CanvasView
Create a React Component named FrontPanel
that manages CanvasView
.
import React, { Component, RefObject } from "react";
import {
IFrontPanel,
WorkQueue
} from "@opalkelly/frontpanel-alloy-core";
import "./FrontPanel.css";
import CanvasView from "./CanvasView";
export interface FrontPanelProps {
name: string;
}
export interface FrontPanelState {
device: IFrontPanel;
}
class FrontPanel extends Component<FrontPanelProps, FrontPanelState> {
private readonly _WorkQueue: WorkQueue = new WorkQueue();
private readonly _CanvasViewRef: RefObject<CanvasView>;
constructor(props: FrontPanelProps) {
super(props);
this._CanvasViewRef = React.createRef();
this.state = {
device: window.FrontPanel
};
}
render() {
return (
<div className="ControlPanel">
<CanvasView
ref={this._CanvasViewRef}
width={1024}
height={768} />
</div>
);
}
}
export default FrontPanel
Code language: JavaScript (javascript)
This defines a React Component named FrontPanel
that maintains a reference to the CanvasView
component created previously. It will use this reference to update the image displayed by the CanvasView
.
Add a Method to Retrieve and Display Image
Add a private method named RetrieveImage
to the FrontPanel
component.
private async RetrieveImage(): Promise<void> {
await this._WorkQueue.Post(async () => {
const dataSize = 1024 * 768 * 4; // 1024x768 pixels, 4 bytes per pixel
const data: ArrayBuffer = await this.state.device.readFromPipeOut(0xa0, dataSize);
if (this._CanvasViewRef.current != null) {
const imageData = new Uint32Array(data);
this._CanvasViewRef.current.UpdateFrameImage(imageData, 1024, 768);
}
});
}
Code language: JavaScript (javascript)
In this example, the data retrieved from the PipeOut endpoint is assumed to be arranged in 1024 columns by 768 rows of pixels. Each pixel is assumed to consist of 4 bytes.
Add a Method to Periodically Retrieve the Current Image
Add a private method named ContinuousImageCaptureLoop
to the FrontPanel
component.
private _IsStopPending: boolean = false;
private _TimeoutId?: number;
private _CurrentOperation?: Promise<void>;
private async ContinuousImageCaptureLoop(): Promise<void> {
const start: number = performance.now();
await this.RetrieveImage();
const elapsed: number = performance.now() - start;
if (!this._IsStopPending) {
const delay: number = 33 - elapsed;
this._TimeoutId = window.setTimeout(
() => {
this._CurrentOperation = this.ContinuousImageCaptureLoop();
},
delay > 33 ? delay : 33
);
}
}
Code language: JavaScript (javascript)
This method will cause the operation that retrieves and displays the image to occur periodically. It attempts to ensure that an update operation is initiated every 33 milliseconds.
Configure the Image Capture Loop to Start and Stop
Add the React Component lifecycle componentDidMount
and componentWillUnmount
methods.
componentDidMount(): void {
this._CurrentOperation = this.ContinuousImageCaptureLoop();
}
async componentWillUnmount(): Promise<void> {
this._IsStopPending = true;
if (this._TimeoutId !== undefined) {
clearTimeout(this._TimeoutId);
this._TimeoutId = undefined;
}
await this._CurrentOperation;
this._IsStopPending = false;
}
Code language: JavaScript (javascript)
These methods will cause the ContinuousImageCaptureLoop
to start automatically when the React Component is mounted and stop it when the component will unmount.
Complete Example
Canvas.tsx
import React, { Component, RefObject } from "react";
export interface CanvasViewProps {
width: number;
height: number;
}
class CanvasView extends Component<CanvasViewProps> {
private _CanvasRef: RefObject<HTMLCanvasElement>;
constructor(props: CanvasViewProps) {
super(props);
this._CanvasRef = React.createRef();
}
componentDidMount() {
this.ClearFrameImage();
}
componentDidUpdate(
prevProps: Readonly<CanvasViewProps>,
_prevState: Readonly<NonNullable<unknown>>,
_snapshot?: NonNullable<unknown>
): void {
if (this.props.width !== prevProps.width || this.props.height !== prevProps.height) {
this.ClearFrameImage();
}
}
render() {
return (
<canvas
ref={this._CanvasRef}
width={this.props.width}
height={this.props.height} />
);
}
public ClearFrameImage() {
const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
if (canvas != null) {
const context: CanvasRenderingContext2D | null = canvas.getContext("2d");
if (context != null) {
context.fillStyle = "black";
context.fillRect(0, 0, canvas.width, canvas.height);
}
}
}
public UpdateFrameImage(sourceData: Uint32Array, columnCount: number, rowCount: number) {
const canvas: HTMLCanvasElement | null = this._CanvasRef.current;
if (canvas != null) {
const context: CanvasRenderingContext2D | null = canvas.getContext("2d", {
alpha: false
});
if (context != null) {
const frameImageData: ImageData = context.createImageData(
columnCount,
rowCount
);
const targetImageData: Uint8ClampedArray = frameImageData.data;
// Copy the pixel data from the source to the target
for (let pixelIndex = 0; pixelIndex < sourceData.length; pixelIndex++) {
const sourcePixelValue: number = sourceData[pixelIndex];
const pixelByteOffset: number = pixelIndex * 4;
targetImageData[pixelByteOffset] = sourcePixelValue & 0xff; // Red
targetImageData[pixelByteOffset + 1] = (sourcePixelValue >> 8) & 0xff; // Green
targetImageData[pixelByteOffset + 2] = (sourcePixelValue >> 16) & 0xff; // Blue
targetImageData[pixelByteOffset + 3] = 0xff; // Alpha
}
context.putImageData(frameImageData, 0, 0);
}
}
}
}
export default CanvasView;
Code language: JavaScript (javascript)
FrontPanel.tsx
import React, { Component, RefObject } from "react";
import {
IFrontPanel,
WorkQueue
} from "@opalkelly/frontpanel-alloy-core";
import "./FrontPanel.css";
import CanvasView from "./CanvasView";
export interface FrontPanelProps {
name: string;
}
export interface FrontPanelState {
device: IFrontPanel;
}
class FrontPanel extends Component<FrontPanelProps, FrontPanelState> {
private readonly _WorkQueue: WorkQueue = new WorkQueue();
private readonly _CanvasViewRef: RefObject<CanvasView>;
private _IsStopPending: boolean = false;
private _TimeoutId?: number;
private _CurrentOperation?: Promise<void>;
constructor(props: FrontPanelProps) {
super(props);
this._CanvasViewRef = React.createRef();
this.state = {
device: window.FrontPanel
};
}
componentDidMount(): void {
this._CurrentOperation = this.ContinuousImageCaptureLoop();
}
async componentWillUnmount(): Promise<void> {
this._IsStopPending = true;
if (this._TimeoutId !== undefined) {
clearTimeout(this._TimeoutId);
this._TimeoutId = undefined;
}
await this._CurrentOperation;
this._IsStopPending = false;
}
render() {
return (
<div className="ControlPanel">
<CanvasView
ref={this._CanvasViewRef}
width={1024}
height={768} />
</div>
);
}
private async RetrieveImage(): Promise<void> {
await this._WorkQueue.Post(async () => {
const dataSize = 1024 * 768 * 4; // 1024x768 pixels, 4 bytes per pixel
const data: ArrayBuffer = await this.state.device.readFromPipeOut(0xa0, dataSize);
if (this._CanvasViewRef.current != null) {
const imageData = new Uint32Array(data);
this._CanvasViewRef.current.UpdateFrameImage(imageData, 1024, 768);
}
});
}
private async ContinuousImageCaptureLoop(): Promise<void> {
const start: number = performance.now();
await this.RetrieveImage();
const elapsed: number = performance.now() - start;
if (!this._IsStopPending) {
const delay: number = 33 - elapsed;
this._TimeoutId = window.setTimeout(
() => {
this._CurrentOperation = this.ContinuousImageCaptureLoop();
},
delay > 33 ? delay : 33
);
}
}
}
export default FrontPanel
Code language: JavaScript (javascript)
FrontPanel.css
.ControlPanel {
display: flex;
flex-direction: column;
background-color: white;
border-radius: 8px;
padding: 10px;
gap: 8px;
margin: 10px;
}
Code language: CSS (css)