Altera Cyclone V SoC Development Kit Reference Platform Porting Guide

Introduction

The Altera Cyclone V SoC Development Kit Reference Platform Porting Guide describes the hardware and software design of the Altera® Cyclone® V SoC Development Kit Reference Platform (c5soc) for use with the Altera Software Development Kit (SDK) for OpenCL™ (AOCL).

Before proceeding, it is recommended to familiarize yourself with the following documents:

1. Altera SDK for OpenCL Cyclone V SoC Getting Started Guide
2. Altera SDK for OpenCL Custom Platform Toolkit User Guide
3. Cyclone V Device Handbook, Volume 3: Hard Processor System Technical Reference Manual

Additional information is available on the Cyclone V SoC Development Kit and SoC Embedded Design Suite page of the Altera website.

? Attention: This guide assumes an in-depth understanding of the Altera SDK for OpenCL Custom Platform Toolkit User Guide. It focuses on the differences between AOCL support on the Cyclone V SoC Development Kit and a generic AOCL Custom Platform, rather than detailing the implementation of a Custom Platform.

Related Information

• Altera SDK for OpenCL Cyclone V SoC Getting Started Guide
• Altera SDK for OpenCL Custom Platform Toolkit User Guide
• Cyclone V Device Handbook, Volume 3: Hard Processor System Technical Reference Manual
• Cyclone V SoC Development Kit and SoC Embedded Design Suite page on the Altera website

Overview of the Cyclone V SoC Development Kit Reference Platform

The Cyclone V SoC Development Kit Reference Platform is available with the Altera SDK for OpenCL.

(1) OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission of the Khronos Group™.

(2) The Altera SDK for OpenCL is based on a published Khronos Specification and has passed the Khronos Conformance Testing Process. Current conformance status can be found at www.khronos.org/conformance.

© 2015 Altera Corporation. All rights reserved. Altera, Arria, Cyclone, Enpirion, MAX, MegaCore, NIOS, Quartus, and Stratix are trademarks of Altera Corporation. All other trademarks are the property of their respective holders. Altera warrants product performance to current specifications but reserves the right to make changes without notice. Altera assumes no liability for the application or use of information, products, or services described herein, except as expressly agreed in writing. Customers are advised to obtain the latest device specifications before relying on published information.

Cyclone V SoC Development Kit Reference Platform Board Variants

AOCL support for the Cyclone V SoC Development Kit leverages the following board features to maximize performance:

1. FPGA device with the FPGA core logic.
2. Hard processor system (HPS) with dual-core ARM® Cortex®-A9 CPU.
3. Shared physical memory between the CPU and the FPGA core fabric.

The Altera SDK for OpenCL Cyclone V SoC Development Kit Reference Platform includes two board variants:

• c5soc board: This default board provides access to two DDR memory banks. The HPS DDR is accessible by both the FPGA and the CPU. The FPGA DDR is accessible only by the FPGA.
• c5soc_sharedonly board: This variant offers only HPS DDR connectivity, with the FPGA DDR inaccessible. It is more area-efficient and suitable for prototyping boards with a single DDR memory bank.

To target the c5soc_sharedonly board variant when compiling an OpenCL kernel, use the --board c5soc_sharedonly option in your aoc command.

Refer to the Altera SDK for OpenCL Programming Guide for more information on the --board <board_name> option.

Related Information

• Compiling a Kernel for a Specific FPGA Board (--board <board_name>)

Content of the Cyclone V SoC Development Kit Reference Platform

The Cyclone V SoC Development Kit Reference Platform consists of the following files and directories:

Relevant Features of the Cyclone V SoC Development Kit

The following features of the Cyclone V SoC Development Kit are relevant to the Altera SDK for OpenCL:

• Dual-core ARM Cortex-A9 CPU running 32-bit Linux.
• Advanced eXtensible Interface (AXI) bus between the HPS and the FPGA core fabric.
• Two hardened DDR memory controllers, each connecting to a 1 gigabyte (GB) DDR3 SDRAM.
• One DDR controller is accessible to the FPGA core only (FPGA DDR).
• The other DDR controller is accessible to both the HPS and the FPGA (HPS DDR), enabling shared memory between the CPU and FPGA core.
• The CPU can reconfigure the FPGA core fabric.

Cyclone V SoC Development Kit Reference Platform Design Goals and Decisions

Altera bases the reference platform implementation on several design goals and decisions. Consider these when porting the platform to your SoC board.

c5soc Design Goals:

1. Provide the highest possible bandwidth between kernels on the FPGA and the DDR memory system(s).
2. Ensure FPGA computations (OpenCL kernels) do not interfere with other CPU tasks, such as servicing peripherals.
3. Maximize FPGA resources for kernel computations, minimizing interface components.

High-Level Design Decisions:

1. The Reference Platform uses hard DDR memory controllers with the widest possible configuration (256 bits).
2. The FPGA communicates directly with the HPS DDR memory controller, bypassing the AXI bus and L3 switch within the HPS. This direct communication optimizes bandwidth to DDR and prevents FPGA computations from interfering with CPU communications.
3. Scatter-gather direct memory access (SG-DMA) is not part of the FPGA interface logic. Data is stored in the shared HPS DDR for efficiency. Direct access to CPU memory by the FPGA is more efficient than DMA, saving FPGA area and simplifying the Linux kernel driver.

Porting the Reference Platform to Your SoC Board

⚠️ Warning: Memory transfer between the shared HPS DDR system and the FPGA-only DDR system is very slow. Use this method only for very small amounts of data.

The host and device perform non-DMA data transfer via the HPS-to-FPGA (H2F) bridge using a single 32-bit port. This is sufficient as the Linux kernel can only issue single 32-bit read/write requests without DMA.

Control signals from the host to the device use a lightweight H2F (LH2F) bridge, ideal for low-bandwidth signals.

Tasks for Porting:

1. Select either the single DDR memory or dual DDR memory version of the c5soc Reference Platform as the starting point.
2. Update pin locations in the ALTERAOCLSDKROOT/board/c5soc/<board_variant>/top.qsf file. Replace ALTERAOCLSDKROOT with the Altera SDK for OpenCL installation path and <board_variant> with c5soc_sharedonly (one DDR) or c5soc (two DDR).
3. Update DDR settings for HPS and/or FPGA SDRAM blocks in the ALTERAOCLSDKROOT/board/c5soc/<board_variant>/system.qsys file.
4. Ensure the design placement is timing clean for guaranteed timing closure. To port the c5soc board partition (acl_iface_partition.qxp):
   1. Remove acl_iface_partition.qxp from the ALTERAOCLSDKROOT/board/c5soc/c5soc directory.
   2. Enable the acl_iface_region LogicLock™ region by changing the Tcl command:

      set_global_assignment -name LL_ENABLED OFF -section_id acl_iface_region

      to

      set_global_assignment -name LL_ENABLED ON -section_id acl_iface_region

   3. Compile an OpenCL kernel for your board.
   4. Adjust the LogicLock region size and location if necessary.
   5. Export the partition as the acl_iface_partition.qxp Quartus Prime Exported Partition File once timing is clean.

   Importing this file into the top-level design fulfills the requirement for a board design with a guaranteed timing closure flow, as described in the Altera SDK for OpenCL Custom Platform Toolkit User Guide.

For detailed instructions on modifying and preserving board partitions, refer to the Quartus® Prime Incremental Compilation for Hierarchical and Team-Based Design chapter of the Quartus Prime Standard Edition Handbook.

Updating a Ported Reference Platform

In the current Cyclone V SoC Development Kit Reference Platform, the HPS block is part of the nonkernel logic partition and cannot be exported within the .qxp file. To update an existing Custom Platform modified from a previous c5soc version:

1. Implement the QXP preservation flow.
2. Update the SD flash card image for the latest runtime environment.
3. Update the board_spec.xml file to enable automigration.

Altera SDK for OpenCL versions 14.1 and later use board_spec.xml for board information and automatic updates. Updating this file helps AOCL distinguish between unpreserved and QXP-based Custom Platforms.

Refer to Custom Platform Automigration for Forward Compatibility in the Altera SDK for OpenCL Custom Platform Toolkit User Guide for more information.

Steps to implement QXP preservation flow:

1. Create a partition around the HPS in the .qsf Quartus Prime Settings File before partitioning the nonkernel logic. Example commands for manual partitioning and setting HPS_PARTITION type are provided.
2. When exporting the partition for acl_iface_partition, use the --incremental_compilation_export_flatten=off option to leave the HPS partition as a blackbox. Example Quartus CDB commands are shown.

After excluding the HPS, import the .qxp file and compile your design.

Update the SD flash card image:

1. Mount the FAT32 and ext3 partitions of the existing image as loop-back devices. Refer to Step 2 in Building an SD Flash Card Image for instructions.
2. In the /home/root/opencl_arm32_rte directory, remove previous RTE files.
3. Download and unpack the current RTE version into the /home/root/opencl_arm32_rte directory.
4. Update the ACL_DRIVER_VERSION in <path_Custom_Platform>/driver/version.h to the current AOCL and driver versions (e.g., 15.1.x).
5. Rebuild the driver.
6. Delete hardware folders of your Custom Platform and copy the platform and updated driver to /home/root/opencl_arm_rte/board.
7. Copy the Altera.icd file from /home/root/opencl_arm32_rte to /etc/OpenCL/vendors.
8. Unmount and test the new image. Refer to Steps 8-11 in Building an SD Flash Card Image for details.

Related Information

• Building an SD Flash Card Image on page 1-10
• Custom Platform Automigration for Forward Compatibility

Software Support for Shared Memory

Shared physical memory between the FPGA and CPU is preferred for OpenCL kernels on SoCs. Since the FPGA accesses shared physical memory directly, it does not have access to the CPU's page tables that map user virtual addresses to physical page addresses.

Hardware-wise, OpenCL kernels connect directly to the HPS DDR memory controller. Software support for shared physical memory involves:

1. Standard CPU memory allocation functions (like malloc()) cannot allocate regions usable by the FPGA. These functions allocate memory contiguous in virtual address space, but not necessarily physically contiguous. User-space applications on Linux lack the ability to allocate physically-contiguous memory, so the Linux kernel driver must handle this.
2. The OpenCL SoC Linux kernel driver uses the mmap() function to allocate and map shared physical memory into user space. It utilizes the standard Linux kernel call dma_alloc_coherent() for requesting physically-contiguous memory regions.
3. By default, dma_alloc_coherent() allocates no more than 0.5 MB of physically-contiguous memory. To enable larger allocations, the contiguous memory allocator (CMA) feature of the Linux kernel must be enabled and the kernel recompiled.

For the Cyclone V SoC Development Kit Reference Platform, CMA manages 512 MB of the 1 GB physical memory. This value can be adjusted based on application needs. While dma_alloc_coherent() may not always allocate the full 512 MB, it typically provides approximately 450 MB.

4. CPU caching can affect visibility of writes to OpenCL kernels. The mmap() function in the OpenCL SoC Linux kernel driver uses pgprot_noncached() or remap_pf_range() to disable caching for this memory region.
5. After dma_alloc_coherent() allocates physically-contiguous memory, mmap() returns the virtual address. The host application uses this virtual address, while OpenCL kernels require physical addresses. The Linux kernel driver maps virtual to physical addresses.

The aocl_mmd_shared_mem_alloc() MMD API call handles these queries:

• mmap() allocates memory and returns the virtual address.
• An additional query maps the returned virtual address to a physical address.

The aocl_mmd_shared_mem_alloc() call returns two addresses: the virtual address and the physical address (via device_ptr_out).

⚠️ Warning: The driver can only map virtual addresses returned by mmap() to physical addresses. Requesting the physical address for any other virtual pointer returns NULL.

⚠️ Warning: The Altera SDK for OpenCL runtime libraries assume shared memory is the first memory listed in board_spec.xml. The physical address obtained by the Linux kernel driver becomes the Avalon® address for the OpenCL kernel to the HPS SDRAM.

Use the clCreateBuffer() call to allocate shared memory as a device buffer:

• For the two-DDR board variant (shared and nonshared memory), clCreateBuffer() allocates shared memory if CL_MEM_USE_HOST_PTR is specified. Other flags allocate to nonshared memory.
• For the one-DDR board variant (only shared memory), clCreateBuffer() always allocates shared memory.

Currently, 32-bit Linux on ARM CPU supports shared memory in AOCL runtime libraries. Other environments (e.g., x86_64 Linux, 64-bit Windows) do not.

C5soc does not distinguish between shared and nonshared memory due to:

• History: Heterogeneous memory support was unavailable when shared memory support was created.
• Uniform interface: To maintain consistency across heterogeneous computing vendors, Altera uses the same interface as other board vendors for memory allocation and usage.

FPGA Reconfiguration

For Altera SoCs, the CPU can reconfigure the FPGA core fabric without interrupting CPU operation. The FPGA Manager hardware block handles this. A Linux kernel driver provides easy access to the FPGA Manager.

• To view FPGA core status, use the command: cat /sys/class/fpga/fpga0/status

The Altera SDK for OpenCL program utility uses this interface to program the FPGA. When reprogramming an FPGA core with a running CPU, the utility performs these tasks:

1. Disable all communication bridges (H2F and LH2F) between the FPGA and HPS before reprogramming. Reenable them after completion.
2. Ensure the link between the FPGA and HPS DDR controller is disabled during reprogramming.
3. Disable FPGA interrupts during reprogramming and notify the driver to reject any interrupts.

Consult the program utility's source code for implementation details.

⚠️ Warning: Do not change the HPS DDR controller configuration while the CPU is running, as this may cause fatal errors by altering the configuration during active memory transactions. Reprogramming the FPGA core with an image using a different HPS DDR configuration while the CPU is running is not permitted.

The OpenCL system and the Golden Hardware reference design (SoC Embedded Design Suite - EDS) configure the HPS DDR to a single 256-bit mode. CPU components like the branch predictor or page table prefetcher may issue DDR commands even when the CPU appears idle. Therefore, boot time is the only safe time to set the HPS DDR controller configuration. U-boot must load a raw binary file (.rbf) image. Enabling HPS DDR with unused FPGA ports and changing configurations afterward can lead to issues. The OpenCL Linux kernel driver no longer includes logic for setting the HPS DDR controller configuration.

The SW3 DIP switches on the Cyclone V SoC Development Kit control the expected format of the .rbf image (compressed/encrypted). C5soc and the Golden Hardware Reference Design use compressed but unencrypted .rbf images. SW3 DIP switch settings in the Altera SDK for OpenCL Cyclone V SoC Getting Started Guide match this configuration.

Related Information

• HPS-FPGA Interfaces
• Configuring the SW3 Switches

FPGA System Architecture Details

Support for the Cyclone V SoC Development Kit Reference Platform is based on the Stratix® V Reference Platform (s5_ref), available with the Altera SDK for OpenCL. The c5soc Qsys system and kernel driver organization are similar to s5_ref.

The following FPGA core components are common to both c5soc and s5_ref:

• VERSION_ID block
• Rest mechanism
• Memory bank divider
• Cache snoop interface
• Kernel clock
• Control register access (CRA) blocks

? Attention: The OpenCL system does not use the FPGA-to-HPS (F2H) bridge. Refer to the HPS-FPGA Interfaces section in the Cyclone V Device Handbook, Volume 3: Hard Processor System Technical Reference Manual for details.

Building an SD Flash Card Image

The Altera Cyclone V SoC is a full system-on-a-chip. Altera recommends delivering the system definition as an SD flash card image, which users can write to a micro SD card to prepare the SoC board for use.

You can build an SD flash card image by following instructions on the GSRD - SD Card page on the RocketBoards.org website. Alternatively, modify the image provided with the Cyclone V SoC Development Kit Reference Platform.

The c5soc linux_sd_card_image.tgz file is located in ALTERAOCLSDKROOT/board/c5soc, where ALTERAOCLSDKROOT is the AOCL installation path.

? Attention: Root or sudo privileges are required to modify the SD flash card image.

The following steps describe creating the linux_sd_card_image.tgz image from the Golden System Reference Design (GSRD) SD flash card image:

Note: To create the image from the c5soc image, perform all applicable tasks in this procedure.

1. Begin with the GSRD SD flash card image version 14.0.
2. Mount the FAT32 and ext3 partitions of the image as loop-back devices. To mount a partition:
   1. Determine the partition's byte start using /sbin/fdisk -lu image_file. (Example: Partition 1, W95 FAT, block offset 2121728; byte offset = 512 * 2121728 = 1086324736 bytes).
   2. Identify a free loop device (e.g., /dev/loop0) with losetup -f.
   3. Assign the flash card image to the loop device: losetup /dev/loop0 image_file -0 1086324736.
   4. Mount the loop device: mount /dev/loop0 /media/disk1. /media/disk1 will now be a mounted FAT32 partition.
   5. Repeat steps a-d for the ext3 partition.
3. Place the unpacked aclrte_arm32 package into /home/root/opencl_arm32_rte on the ext3 partition.
4. Delete your Custom Platform's hardware folder(s) and place the Custom Platform into the board subdirectory of opencl_arm32_rte.
5. Create init_opencl.sh in /home/root with the following content:

   export ALTERAOCLSDKROOT=/home/root/opencl_arm32_rte
   export AOCL_BOARD_PACKAGE_ROOT=$ALTERAOCLSDKROOT/board/<board_name>
   export PATH=$ALTERAOCLSDKROOT/bin:$PATH
   export LD_LIBRARY_PATH=$ALTERAOCLSDKROOT/host/arm32/lib:$LD_LIBRARY_PATH
   insmod $AOCL_BOARD_PACKAGE_ROOT/driver/aclsoc_drv.ko

   The AOCL user runs source ./init_opencl.sh to load environment variables and the OpenCL Linux kernel driver.

6. If updating the preloader, DTS files, or Linux kernel, follow instructions in the Altera SoC Embedded Design Suite User Guide to recompile and update files on the mounted FAT32 partition.

Recompiling the Linux Kernel and the OpenCL Linux Kernel Driver

To run OpenCL applications on the Cyclone V SoC board, you must first compile the OpenCL Linux kernel driver and the Linux kernel. Kernel recompilation is necessary to enable CMA.

? Attention: You will likely need to update the preloader if your Custom Platform has different pin usages than those in c5soc.

Remember: If you recompile the Linux kernel, use the same source files to recompile the Linux kernel driver. A mismatch will prevent the driver from loading. Ensure CMA is enabled.

Refer to Recompiling the Linux Kernel and the OpenCL Linux Kernel Driver for more information.

1. Click the GSRD v14.0 - Compiling Linux link on the RocketBoards.org website Resources page for instructions on downloading and rebuilding the Linux kernel source code.
2. The build process creates arch/arm/configs/socfpga_defconfig. Add the following lines to this file:

   CONFIG_MEMORY_ISOLATION=y
   CONFIG_CMA=y
   CONFIG_DMA_CMA=y
   CONFIG_CMA_DEBUG=y
   CONFIG_CMA_SIZE_MBYTES=512
   CONFIG_CMA_SIZE_SEL_MBYTES=y
   CONFIG_CMA_ALIGNMENT=8
   CONFIG_CMA_AREAS=7

   CONFIG_CMA_SIZE_MBYTES sets the upper limit for physically contiguous memory. Increase this value if more memory is needed.

   ? Attention: The total physical memory available to the ARM processor on the SoC board is 1 GB. Altera does not recommend setting the CMA manager close to 1 GB.

3. Run make ARCH=arm socfpga_deconfig.
4. Run make ARCH=arm zImage. The resulting image is in arch/arm/boot/zImage.
5. Run export CROSS_COMPILE=arm-linux-gnueabihf-.
6. Place the zImage file into the FAT32 partition of the flash card image.
7. To recompile the OpenCL Linux kernel driver, set the KDIR value in the driver's Makefile to the directory containing the Linux kernel source files.
8. Type make.
9. To verify CMA is enabled, power up the SoC board and run grep init_cma /proc/kallsyms. CMA is enabled if the output is non-empty.

Steps for testing the SD flash card image:

1. Write the uncompressed image to a micro SD flash card.
2. Insert the card into the SoC board.
3. Power up the board.
4. Invoke the aocl diagnose utility command.

Related Information

• GSRD - SD Card page on the RocketBoards.org website
• Altera SoC Embedded Design Suite User Guide
• OpenCL Design Examples page on the Altera website
• Recompiling the Linux Kernel and the OpenCL Linux Kernel Driver on page 1-11
• Querying the Device Name of Your FPGA Board (diagnose)

Known Issues

There are limitations when using the Altera SDK for OpenCL with the Cyclone V SoC Development Kit Reference Platform:

1. Vendor and board names reported by CL_DEVICE_VENDOR and CL_DEVICE_NAME strings of the clGetDeviceInfo() call cannot be overridden.
2. If the host allocates constant memory in shared DDR (HPS DDR) and modifies it after kernel execution, the data may become outdated. This is because the FPGA core cannot snoop on CPU-to-HPS DDR transactions. To prevent this, implement one of the following workarounds:
   • Do not modify constant memory after initialization.
   • If multiple __constant data sets are required, create multiple constant memory buffers.
   • If available, allocate constant memory in the FPGA DDR on your accelerator board.
3. The AOCL utility on ARM only supports the program and diagnose utility commands.

Document Revision History

The flash, install, and uninstall utility commands are not applicable to the Cyclone V SoC Development Kit for the following reasons:

• Install/Uninstall: The install utility requires compiling the aclsoc_drv Linux kernel driver and enabling it on the SoC. The development machine already has SoC Linux kernel sources, which may differ from the SoC's sources, making the location unknown to the AOCL user. The uninstall utility is also unavailable. Delivering aclsoc_drv is challenging as the default distribution lacks Linux kernel include files or the GNU Compiler Collection (GCC).
• Flash Utility: Requires placing a .rbf file on the FAT32 partition of the micro SD card. This partition is not mounted on board power-up. Updating requires a flash card reader and the development machine.

When switching between Altera Offline Compiler Executable files (.aocx) for different board variants (c5soc and c5soc_sharedonly), use the AOCL program utility to load the .aocx file for the new variant the first time. Running the host application with a new variant while the FPGA contains an image from another variant may cause a fatal error.

The .qxp file does not include interface partition assignments, as Quartus Prime software consistently meets timing requirements for this partition.

When powering up the board, its media access control (MAC) address is random. To set a static MAC address:

• During U-Boot power-up, press any key to enter the U-Boot prompt.
• Type setenv ethaddr 00:07:ed:00:00:03 (or any desired MAC address) at the prompt.
• Type saveenv.
• Reboot the board.

Revision History