System Design

FrontPanel connects logic inside an FPGA to software on a host. From the gateware side, that connection appears as a set of interfaces carrying data in and out. From the host’s side, it appears as API calls.

Direction is named from the perspective of the device throughout this documentation and in the interface names themselves. An In interface moves data into the gateware; an Out interface moves data out of it.

The link between the two is not on-chip. Data crosses an external interface — USB or PCI Express — with its own clock domain, its own protocol, its own latency, and its own failure modes. Those properties are inconsequential at low transfer rates and decisive at high ones. Most of this page concerns them.

Classic and AXI interfaces

A FrontPanel board presents one of two interfaces to the gateware, depending on its hardware generation: the Classic endpoint set, or standard AXI. The FrontPanel SDK supports both. The vocabulary differs between them, and so does the way a design connects to them. What the communication does is the same.

FrontPanel introduced its endpoint set in 2005: Wires, Triggers, and Pipes, joined later by the Register Bridge and Block-Throttled Pipes. Each endpoint type has a fixed behavior and a reserved address range, up to 32 instances of each, and the host addresses them by endpoint. AXI was not broadly deployed in FPGA design at that time, and the endpoint set was defined independently of it.

FrontPanel AXI boards present standard AXI instead. The predefined endpoint types are gone. You lay out a register map on AXI4-Lite, a memory map on AXI4-Full, and streams on AXI4-Stream, and the host reaches them with generic AXI reads and writes to addresses you assign. Because the design interface is standard AXI, the surrounding AXI ecosystem — interconnects, CSR generators, memory controllers, third-party IP — connects to it directly.

Underneath, both interfaces provide the same four operations: holding state, signalling events, addressing registers, and moving bulk data. A design partitioned sensibly for one partitions sensibly for the other, and every system-level consideration in the rest of this page applies to both.

The correspondence between the two interfaces is shown in the table below. On AXI boards each equivalent is built rather than instantiated; the register patterns are given in the Classic to AXI Migration Guide.

What it doesClassic endpointAXI equivalent
Host to FPGA control bitsWireInWrite-only AXI-Lite register
FPGA to host status bitsWireOutRead-only AXI-Lite register
Host to FPGA one-shotTriggerInWrite-only AXI-Lite register, self-clearing
FPGA to host eventTriggerOutRead-only AXI-Lite register, clear-on-read
Addressable 32-bit registersRegister BridgeAXI-Lite register space, or AXI-Full for a large map
Host to FPGA bulkPipeInAXI-Stream m_axis, or an AXI-Full write
FPGA to host bulkPipeOutAXI-Stream s_axis, or an AXI-Full read
Throttled bulkBlock-Throttled PipeAXI-Stream with per-beat ready and valid

The sections that follow are organized by what the communication does rather than by what it is called. Where the two interfaces behave differently, the difference is stated.

Control and Status

Much of the interaction between host and gateware is control and status: setting a mode, starting an operation, reading a counter, learning whether something finished. This is a control and status register (CSR) interface, and both FrontPanel interfaces provide one.

On Classic devices the CSR interface is distributed across three endpoint types: wires hold state, triggers carry events, and the Register Bridge holds addressed values. On AXI devices all three are register patterns on a single AXI4-Lite interface.

State

A Wire is an asynchronous connection carrying the current value of a signal. A WireIn is an input to the gateware; a WireOut is an output from it. Wires fill the position of pushbuttons, DIP switches, LEDs, and hexadecimal displays: they are not synchronous to the design, and they convey present state rather than events.

Wires are updated by polling, at a rate the host determines. Even at the fastest update period the bandwidth consumed is negligible. All wires transfer together: reading any WireOut captures all 64 of them, and writing any WireIn updates all 64. They are not synchronous to each other, but they are updated at the same instant.

Group updates on AXI

AXI has no equivalent to the Classic group update. A CSR is written one register at a time, so control fields spread across several registers take effect one after another rather than at the same instant. Where a design requires them to change together, the requirement is carried in the register map itself — the fields packed into a single register, or a commit bit the design acts on.

On AXI, a WireIn is a plain write-only AXI-Lite register and a WireOut is a plain read-only one.

Events

A Trigger is a synchronous connection conveying a single event rather than a state. A TriggerIn creates a signal asserted for exactly one clock cycle; the synchronization clock is yours to choose, and the module performs the clock-domain crossing. A TriggerOut signals the host when a rising edge is detected, determined by the signal’s state from one clock cycle to the next, which means glitches are not detected.

Because the host polls, a TriggerOut is latched: once set it remains set until the next poll clears it. Multiple occurrences between two polls are indistinguishable from one.

On AXI both are register patterns rather than dedicated hardware. A TriggerIn is a write-only register that clears to zero every cycle, so each written bit pulses for a single clock. A TriggerOut is a read-only register whose bits set on an event and clear when read. Both patterns ship in the Counters sample.

Neither pattern crosses clock domains. Both operate in the clock domain of the AXI4-Lite interface, and reaching a different domain takes an explicit crossing — an AXI clock converter ahead of the register block, or synchronizing logic on the signals the registers drive.

Addressed Registers

The Register Bridge provides an addressable read/write access point into the design, with 32-bit address and 32-bit data ports and separate read and write strobes. It is available on USB 3.0 Classic devices.

On AXI devices, AXI4-Lite is itself an addressed register interface, and the map is whatever you assign. For a map large enough that single-beat access becomes the bottleneck, AXI4-Full carries the same addressing with bursts.

Whichever interface is in use, treat an individual register access as atomic and design any multi-register operation around that assumption. This matters most for flow control, discussed below.

Bulk Data

Where control and status move individual values, bulk transfer moves quantities of data: memory contents, captured frames, streaming acquisition, computed results.

A Pipe transmits a series of bytes to or from the design. The host controls the transaction in both directions, so from the gateware’s point of view a Pipe is always a subordinate. Transfers occur at the endpoint’s clock rate, which is fixed by the device generation:

  • Gen 1 (USB 2.0) — 48 MHz
  • Gen 2 (USB 3.0) — 100.8 MHz
  • Gen 2 (PCIe) — 50 MHz
  • Gen 3 (AXI) — 100 MHz

Crossing from that clock into the design’s own is best handled with a FIFO.

A Block-Throttled Pipe is a Pipe that the design can throttle, by asserting back-pressure at block-size intervals rather than per beat. Throughput is the same as a standard Pipe. On PCI Express devices, Block-Throttled Pipes are treated as standard Pipes.

AXI4-Stream is the AXI counterpart, with per-beat TVALID and TREADY handshaking in place of block-level throttling. okHost presents one stream in each direction; reaching more than one stream endpoint is a routing problem solved with an AXI4-Stream Interconnect.

AXI4-Full has no Classic counterpart. It carries addressed, bursted access to memory and memory-like targets, and is the right choice when the host needs random access to a region rather than an ordered flow. The distinction between the two is addressing: a stream carries data in order with no address, while a memory map is addressed and may be accessed in any order.

Throughput figures by device generation are summarized in the Introduction, with measurement conditions and per-platform results on Performance.

Flow Control

An endpoint that cannot produce or accept data as fast as the host transfers it must be able to say so. Both interfaces provide a mechanism. They differ in granularity, and in what happens when the mechanism is relied on too heavily.

One way in which an external link departs from an internal bus is the cost of stalling it. On-chip, back-pressure is free and instantaneous. Across USB or PCI Express, a stalled transfer holds resources on the host, in the driver, and in the device firmware, and a stall held long enough becomes an error rather than a delay.

Classic: Block-Level Back-Pressure

A Block-Throttled Pipe applies back-pressure through the EP_READY signal to the USB microcontroller, recognized at block-size intervals. It moderates a transfer running near the maximum rate the interface supports, removing the overhead of negotiating each block at a higher level.

Held long enough, back-pressure induces a timeout. Because the mechanism sits at a low level, the consequences are correspondingly low-level: data loss, data corruption, or loss of synchronization with the device. Back-pressure is appropriate for slowing a transfer temporarily, not for pausing one indefinitely.

AXI: Per-Beat Back-Pressure

AXI4-Stream carries TVALID and TREADY on every beat, so back-pressure is native to the protocol and finer-grained than the Classic block interval. okHost bounds every operation with a timeout and returns the byte count reached, so a subordinate that stalls returns a partial result rather than hanging the host.

AXI introduces a consideration of its own. okHost does not move data continuously; it transfers in chunks with brief idle gaps between them, an artifact of the USB microcontroller bridging the link. A fixed-rate endpoint that cannot pause — a free-running ADC, a continuous DAC — needs a FIFO sized to absorb those gaps. Sizing guidance is on AXI System Design.

Negotiating at a Higher Level

When the required bandwidth is a modest fraction of what the link provides, there is room to negotiate transfers explicitly using control and status registers, rather than relying on low-level back-pressure. The cost of checking a buffer level or a threshold before starting a transfer is small relative to the transfer itself, and it moves the decision to a layer where a delay is a delay rather than an exception.

Example: Low-Latency Handshaking

Consider a link providing about 300 MB/s and a design streaming 200 MB/s from an ADC or image sensor. Two thirds of the available bandwidth is committed, leaving little room for routine polling to negotiate each block. Low-level throttling is the appropriate mechanism here: the firmware and the design handle the handshaking, and because the source is continuous, data is reliably present and transfers are unlikely to time out. On Classic devices that is a Block-Throttled Pipe, negotiating at block-size intervals; on AXI devices it is an AXI4-Stream, negotiating per beat.

Example: High-Latency Handshaking

Consider instead 75 MB/s arriving in large packets with idle periods between them. Bandwidth headroom is ample and the idle periods are an opportunity: the host can poll a status register to learn when a packet is available and start the transfer only when it can proceed to completion without delay. On Classic devices that status is carried on wires and triggers, with the bulk transfer on a Pipe; on AXI devices it is carried on AXI4-Lite registers, with the transfer on AXI4-Stream or AXI4-Full. This is the approach taken in the Camera Reference Design, where images are captured to a frame buffer and the host polls for buffer occupancy before reading.

The difference between the two examples is not the data rate. It is how much of the link is committed, and therefore whether the system can afford to negotiate.

Timeout

A timeout is a bounded wait: a period after which a system stops waiting and proceeds, either reporting an error or returning what it has. Timeouts arise at several layers of a FrontPanel system, and they mean different things at each.

Three of those layers are worth distinguishing:

  • A request the FrontPanel API makes of the host operating system, hardware, or driver may time out.
  • An application’s request to the FrontPanel API may time out, against resource limits reached in servicing it.
  • The application’s own operation may time out to its user, against limits of the platform it runs on.

Responsibility runs upward. Each layer understands the resource limits and behavior of the layer below it, and communicates error and timeout conditions to the layer above in terms that layer can act on. A timeout that is an exception at one layer is often a normal, expected result at the next.

System Layering with the FrontPanel SDK

Operating System and Hardware Layer

A timeout at this layer is an error condition and should not occur while the device, the host, and the link are all operating correctly. Data may be lost, and recovering may require some level of reset negotiation with the device.

Timeout support and behavior are not uniform across host hardware and operating systems. The period itself, whether the condition is reported at all, and the state the transfer is left in all vary by platform. The layer above therefore treats these as exceptions rather than as results to be interpreted.

FrontPanel SDK Layer

Some FrontPanel API methods report a timeout originating at the operating system or hardware layer. These are exceptions: the firmware may be left in an inconsistent state, and data may be lost.

A design can also induce a timeout at this layer, by holding off a transfer the host has already started. The two interfaces differ in what the host is told when that happens.

On Classic devices, sustained back-pressure on a Block-Throttled Pipe is the path that reaches the SDK. The EP_READY handshake runs between the USB microcontroller and the gateware, below the level at which the API can intervene, so a stall held past the timeout surfaces as an exception with the transfer in an undefined state — data loss, data corruption, or loss of synchronization with the device. The call reports the failure rather than the point the transfer reached. The throttle is intended to moderate a transfer running near the maximum rate the interface supports, removing the cost of negotiating each block at a higher level; it is not a means of pausing one. A design that needs to hold a transfer off for longer than a few block intervals has, by that fact, the bandwidth headroom for the wire and trigger negotiation described under Flow control, and that is the approach to take.

On AXI devices, okHost bounds every operation with its own timeout and returns the count reached, so a Subordinate that stalls or misbehaves yields a partial result rather than a hung host. On reads and streams, the region of the buffer that was never filled carries 0xE7, marking how far the data actually got. On writes, the byte count and the transaction count can be compared to localize the fault: a full byte count with no responses points at the B channel, while a low byte count points at the AW or W handshake. These counts are the first diagnostic, ahead of any hardware instrumentation. See AXI System Design for the counts each call returns and the triage sequence that follows from them.

Application Layer

The application layer is where application software and application hardware meet: software calls the FrontPanel API, and the design answers on the interfaces it exposes. Whether to implement a timeout here is a system design decision, and the FrontPanel system design neither requires nor prevents one.

Some applications never need a timeout at this layer. Others expect one as a normal condition: an application reading periodic data from a remote instrument over a radio link would treat a sustained absence of data as a timeout to surface to its operator, with nothing below it having failed. Where an application does implement one, its period is bounded from below by the layers underneath it — a timeout shorter than the operations it supervises will fire on transfers that were going to complete.

The Host Side

Every interface described above has a corresponding host-side operation.

Applications written against the FrontPanel API call methods that map to the interfaces directly: Classic designs use the endpoint methods, and AXI designs use generic reads and writes on the AXI-Lite, AXI-Full, and AXI-Stream interfaces. Applications built for the FrontPanel Platform reach the same interfaces through the Platform API, and the React component library binds common controls and indicators to them.

A structure worth adopting on both sides is a driver layer: a class holding the register map in one place and exposing domain operations rather than raw addresses. The map then exists once on the host and once in the gateware, and the two can be read against each other. See AXI System Design for the pattern and a worked example.