By the end of this tutorial you will have successfully used the SZG-PCIEX4 and a SYZYGY carrier board to initiate DMA transfers between your FPGA gateware and host PC. The steps will guide you through the process of generating the gateware, configuring your host system, bringing up your hardware, and successfully running the test procedure to validate your system.

This tutorial utilizes Xilinx’s DMA/Bridge Subsystem for PCI Express IP’s example design along with Xilinx’s provided example drivers. This tutorial will use the Ubuntu operating system, but Windows 10 drivers are also available.

• A supported SYZYGY TXR4 carrier board:
  • XEM8320
  • ECM1900 + BRK1900
  • XEM8350 + BRK8350
• SZG-PCIE4X PCI Express 4-lane interface card
• One of the following supported Ubuntu distributions: 
  • 16.04 LTS
  • 18.04 LTS
  • 20.04 LTS

• Xilinx PCI Express DMA Drivers and Software Guide for Linux and Windows (AR65444)
• DMA Subsystem for PCI Express (Product Page)
• XDMA GitHub repository Linux driver and application sources.
• Pins Reference

The DMA/Bridge Subsystem for PCI Express IP’s example design is generated by Vivado. The example design implements a 4KByte BRAM buffer that is read or written to via DMA. The PCIe signals travel through the Samtec cable to a SZG-PCIE4 connected to the PCIe slot on the motherboard of the HOST PC. Xilinx’s provided drivers/applications initiate the DMA transfers to and from the BRAM on the SYZYGY carrier device. 

1. Open a New Vivado project.
2. Select the device family for your Opal Kelly product.
3. Configure the DMA/Bridge Subsystem for PCI Express IP from the IP wizard. 
4. Select a lane width of x4.
5. Select the maximum link speed available through the GUI.
6. Select Mode “Advanced”.
7. Select the “GT Settings” tab.
8. Select “true” for “Disable GT Channel LOC Constraint. See “Vivado IP Constraint Automation” under “Restrictions” at SZG-PCIEX4 for more information regarding why disabling the LOC constraints is necessary.
9. All other settings can be kept at their default. Press “OK” to create the IP.
10. Right click on the IP in the hierarchy view and select “Open IP Example Design”.

1. Open up the example design’s constraint file.
2. Remove the LOC constraints for ports sys_rst_n, sys_clk_p, and sys_clk_n.
3. Open your Opal Kelly board’s Pins page.
4. Drop down the “DISPLAY OPTIONS” menu.
5. Select “CONNECTOR”.
6. Select the TXR4 port on your SYZYGY carrier board that will be used with the SZG-PCIEX4.
7. Have available the “Pinout” section from SZG-PCIEX4.
8. The following schematic nets and ports are to be connected:
   1. PCIE_PERST_B_VIO (net) = sys_rst_n (port)
   2. PCIE_REFCLKP (net) = sys_clk_p (port)
   3. PCIE_REFCLKN (net) = sys_clk_n (port)
9. Find the Pin Num from “Pinout” for the nets listed above. Find that pin’s row from the PIN column from step 6 Use the FPGA PIN listed to replace the removed constraints from step 2.
10. Follow the same procedure to add in the following constraints:

set_property LOC <FPGA PIN> [get_ports pci_exp_txp[0]]
set_property LOC <FPGA PIN> [get_ports pci_exp_txn[0]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxp[0]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxn[0]]
 
set_property LOC <FPGA PIN> [get_ports pci_exp_txp[1]]
set_property LOC <FPGA PIN> [get_ports pci_exp_txn[1]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxp[1]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxn[1]]
 
set_property LOC <FPGA PIN> [get_ports pci_exp_txp[2]]
set_property LOC <FPGA PIN> [get_ports pci_exp_txn[2]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxp[2]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxn[2]]
 
set_property LOC <FPGA PIN> [get_ports pci_exp_txp[3]]
set_property LOC <FPGA PIN> [get_ports pci_exp_txn[3]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxp[3]]
set_property LOC <FPGA PIN> [get_ports pci_exp_rxn[3]]
Code language: HTML, XML (xml)

11. Generate the bitstream.

1. Acquire the sources at the following Git repository: XMA
2. Follow the README’s instructions for installation.

The PCIe device must be powered and available at the time the host PC enumerates the PCIe bus at power on. Follow the following procedure to configure your hardware correctly:

1. Power-off all devices. 
2. Place the SZG-PCIEX4 into a capable PCIe slot on your host PC.
3. Connect the provided Samtec cable between the SYZYGY connectors on the SZG-PCIEX4 and your SYZYGY carrier.
4. Power-on your SYZYGY carrier.
5. Program the PCIe DMA bitfile into your SYZYGY carrier. 
6. Power-on your host PC. 

1. Run sudo lspci -vvv to see verbose information about the devices enumerated onto the PCIe bus of the host PC. The device will be enumerated under the name “Xilinx Corporation Device”.
2. Run the following Xilinx provided tests from the XMA Git repository:
   1. run_tests.sh
   2. perform_hwcount.sh