AC460 Application Note

1 Revision History

The revision history describes the changes that were implemented in the document. The changes are listed by revision, starting with the most current publication.

1.1 Revision 1.0

Revision 1.0 is the first publication of this document.

2 Overview

2.1 Introduction

This application note describes how to build Mobile Industry Processor Interface (MIPI) Camera Serial Interface 2 (CSI-2) receive solutions using SmartFusion®2 SoC FPGA and IGLOO®2 FPGA devices. It guides the reader on how to build the design using Libero® SoC software, which I/O interfaces to use. It also lists some of the capabilities and limitations of the device with reference to implementing the MIPI CSI-2 protocol.

Figure 1: MIPI CSI-2 Based System
A system diagram showing an Image Sensor connected via MIPI CSI-2 lanes (Lane 0-3) and a Clock line to an FPGA. The FPGA processes this data and outputs parallel digital video with sync and clock to an HD LCD Monitor. An I2C interface is shown for configuration.

In the preceding figure, the MIPI CSI-2 interface consists of one or more high-speed serial unidirectional differential data pairs and a high-speed serial clock from the transmitter (image sensor) to the receiver (FPGA). The MIPI CSI-2 specification defines High Speed (HS) and Low Power (LP) modes of operation. This application note focuses on high-speed operations only, and does not consider low-power operations, as the latter is often not required for interfacing with most applications configured for MIPI CSI-2 receive only. The bi-directional low-speed I2C interface is used for control and configuration. Image sensors are typically configured by updating registers that are accessible by the FPGA through the I2C interface. For more information about the MIPI CSI-2 interface, see the MIPI CSI-2 Specification by MIPI Alliance Group.

SmartFusion2 and IGLOO2 SoC FPGAs come with high-speed differential receivers that can be configured to receive MIPI CSI-2 high-speed data. The SmartFusion2 device has a built-in ARM Cortex-M3-based microcontroller subsystem (MSS) that can be used to configure the image sensor over an I2C bus. See the DS0128: IGLOO2 and SmartFusion2 Datasheet for more information about SmartFusion2 and IGLOO2 FPGA devices.

The Libero design software can be used to build image processing application designs using easy-to-use SmartHDL schematic-based design entry tools, or standard HDL coding methods. The Libero design software comes with all of the required building blocks to build a MIPI CSI-2 high-speed receiver. Subsequent sections in this document go into the details of building a successful MIPI CSI-2 reference design.

2.2 Reference

• MIPI Alliance Specification for Camera Serial Interface 2 (CSI-2)
• UG0557: SmartFusion2 SoC FPGA Advanced Development Kit User Guide
• DS0128: IGLOO2 and SmartFusion2 Datasheet
• http://www.microsemi.com/products/fpga-soc/technology-solutions/imaging

3 Design Overview

The following figure shows a MIPI CSI-2 reference design block diagram for SmartFusion2 and IGLOO2 devices. This reference design includes a MIPI CSI-2 receiver that interfaces with a MIPI image sensor to de-serialize high-speed serial data to raw sensor parallel data. The received RAW Bayer-pattern pixel data is then converted to a 24-bit RGB format. There are several image processing steps before the RGB pixel data is sent to the display controller block. The display controller block generates the required display timing sync signals based on image output configurations.

Figure 2: MIPI CSI-2 Reference Design Block Diagram
A block diagram illustrating the MIPI CSI-2 Reference Design. It shows an Image Sensor feeding high-speed differential data and clock into the FPGA. Inside the FPGA, the MIPI CSI-2 IP includes a Deserializer/Aligner (LVDS 1:8) and a MIPI Data Decoder. Other blocks like MSS/I2C Core, Bayer Conversion, Video Pipe, Edge Detection, Alpha Mixer, and Sharpening are also present. The output is parallel digital video with sync and clock to an HD LCD Monitor. An I2C interface is used for configuration.

3.1 De-serializer and Aligner Block (LVDS RX 1:8)

The De-serializer and Aligner block as implemented in SmartFusion2 and IGLOO2 devices for a single lane is shown in the following figure. This block receives high-speed serial MIPI data, de-serializes and converts it to 8-bit parallel data. The aligner block bit aligns the data to the correct 8-bit demarcations in the serial stream. The single lane example design can be extended to four lanes.

Figure 3: LVDS RX 1:8 Example Design Diagram
A diagram of the LVDS RX 1:8 module for a single lane. Inputs include DATA_LANE0_IN and CLOCK_IN. It uses DDR_IN, Deserializer, and Fabric CCC blocks. Outputs are RD_DATA_A (8-bit parallel data) and Training_Pattern. It generates Serial Clock and Parallel Clock.

A DDR_IN Libero macro is instantiated in the design to clock data at both, positive and negative edges of the reference clock. A clock control PLL CCC macro is instantiated in the design to derive both, a de-skewed serial clock and a bit parallel clock (/4 of serial clock) named clk_x. The clock conditioning circuit (CCC) is configured for a serial clock (named clk_4x) to be presented on the GL0 pin. This GL0 serial clock output is fed back as a feedback clock to the CCC to cancel out any routing delays that may skew the clock before capturing serial data inside the FPGA fabric.

The Deserializer block hunts within the incoming serial stream for short-packet reception defined by the MIPI protocol. The state machine looks for an identifying pattern, 2 bits at a time, clocked out of the DDR input block and moves to the next state machine as the pattern matching steps are followed. If at any time the pattern matching fails, the state machine moves back to an Idle state and starts all over again until a valid pattern is matched for all 16 bits. This state machine is shown in the following figure. Once a complete pattern is detected, the align signal goes up to indicate the next block, the sync decoder, that the 8-bit data from the Deserializer block is known to be aligned with the correct demarcation in the serial stream.

Figure 4: Data Aligner FSM Diagram
A Finite State Machine (FSM) diagram for the Data Aligner. It has states like IDLE and PATTERN0 through PATTERN7. Transitions occur based on matching input patterns (e.g., tr_pat[1:0] == {q_r,q_f}). The FSM moves to 'Aligned' state upon successful pattern matching and 'Reset' to return to IDLE if a pattern fails.

3.2 MIPI Sync Decoder

The MIPI Sync decoder module block diagram is shown in the following figure. The sync decoder module decodes the Start of Line, Start of Frame, End of Line, and End of Frame of the image signals along with the RAW pixel data from the Image sensor. The sync decoder block also extracts and aligns data striped across multiple MIPI lanes in mulit-lane MIPI configurations.

Figure 5: MIPI Sync Decoder Block
A block diagram of the MIPI Sync Decoder. It takes 'Lane' input and uses Control Signal Generation to produce Line_Start, Line_End, Frame_Start, and Frame_End signals. It also includes a Lane Selection MUX (for Lanes A, B, C, D), a Line Buffer, and outputs Pixel_Data and Pixel_Out, along with a Clock.

4 Libero Implementation

This section describes Libero implementation of a single-lane MIPI design that is available for download at the following location: http://www.microsemi.com/form/87-design-files-for-mipi-csi-2-camera-sensor-interface.

Once the reference design is opened in the Libero design software, navigating to the Design Hierarchy window shows the design hierarchy as shown in the following figure.

Figure 6: Design Hierarchy of MIPI Reference Design
A screenshot or representation of a hierarchical view in the Libero design software. It shows the design structure, highlighting modules like AR0330_CAM_TOP, AR0330_MIPI_IF, cam_ar0331_mipi, LVDS_RX_1_8, and ar0330_mipi_sync_decoder.

The highlighted section is the MIPI CSI-2 design module, which consists of two sub-modules: LVDS_RX_1_8 LVDS receiver and ar0330_mipi_sync_decoder MIPI Sync decoder block. As mentioned earlier, the LVDS receiver module performs serial-to-parallel data conversion, and the sync decoder block extracts image data information including start/end of video frame and lines.

The following figure shows a SmartHDL canvas view of a single-lane MIPI receiver and decoder implementation.

Figure 7: Top-level MIPI CSI-2 Libero Design per Single Lane
A SmartHDL canvas view showing the top-level MIPI CSI-2 design for a single lane. It depicts the interconnection between the LVDS_RX_1_8 module and the ar0330_mipi_sync_decoder_0 module. It shows input ports like CLK_PAD_P, CLK_PAD_N, DATA_PADP_0, DATA_PADN_0 and output ports like RDATA_A[7:0], CFA_PixelOut_o[7:0], dg_frame_start, etc.

In the preceding figure, CLK_PAD_P and CLK_PAD_N constitute a high-speed differential clock. DATA_PADP_0 and DATA_PADN_0 are high-speed differential data ports in DDR format. The LVDS_RX_1_8 module deserializes the high-speed serial data coming from the image sensor down to 8-bit parallel data and generates an align signal for each lane as shown in the preceding figure. The parallel data output and parallel clock outputs from the deserializer are routed to the ar0330_mipi_sync_decoder module, which decodes the parallel data and generates all of the required video timing signals such as Line_start, Line_end, frame_start, frame_end and extracts the RAW pixel data. The RAW8 pixel data is sent to the next block as CFA_PixelOut_o[7:0] along with a valid signal, CFA_PixelOut_Valid_o.

The following figure shows how to use INBUF_DIFF, DDR_IN, CLKBUF_DIFF, and FCCC blocks to interface with high-speed MIPI receive data.

Figure 8: LVDS RX 1:8 Libero Design per Single Lane
A diagram illustrating the LVDS RX 1:8 Libero design components. It shows the flow through INBUF_DIFF, DDR_IN, RX_TOP, FCCC, and CLKBUF_DIFF blocks, connecting differential input pads (PADP, PADN) to internal signals and data outputs (RDATA A-D).

MIPI signals use SLVS-200 signaling levels for high-speed transmission. These signals are received by differential I/O buffers of SmartFusion2 and IGLOO2 devices. The differential I/Os must be configured to LVDS 2.5 V. It is recommended to use MSIOD LVDS 2.5 V configuration to achieve best I/O performance. It is also recommended to select the differential MSIOD pairs close to each other to get deterministic and maximum performance.

It is recommended to route back the serial clock output of the CCC as feedback to the PLL to cancel out clock skews introduced by global clock buffers and internal clock routing delays that might introduce clock skews.

5 Limitations of MIPI CSI-2 on SmartFusion2 and IGLOO2 Devices

1. MIPI CSI-2 can support speeds of up to 1Gbps or higher in high-speed mode. SmartFusion2 and IGLOO2 devices, however, can support only a maximum speed of 700Mbps on LVDS I/Os (MSIODs). Also, extreme care must be taken to match clock-data skews both external to the FPGA and inside the FPGA, to achieve the best performance.

2. MIPI CSI-2 supports two modes, namely high speed (HS) and low power (LP) modes. In low power modes all differential signals act as single-ended LVCMOS signals and require additional I/O circuitry to switch between LP and HS modes. Only HS mode can be supported by SmartFusion2 and IGLOO2 devices using LVDS I/Os.

When both, HS and LP modes must be supported, it is recommended to use an external device such as Meticom MC20901 DPHY chip to convert SLVS MIPI I/O signals to LVDS for HS data and LVCMOS low power signals. The following example block diagram shows how to use the Meticom device to interface an external Image sensor with SmartFusion2 and IGLOO2. In this example, the high-speed signals are routed to the MSIOD LVDS 2.5 V differential pairs of the FPGA while the low-power signals are routed to LVCMOS 2.5 V signals. Also, care should be taken to route high-speed signals to the same FPGA I/O bank and the I/O selection must minimize clock-data skew.

Figure 9: Using an External MIPI Interface Device to Support both HS and LP Modes
A block diagram showing an external MIPI interface device (METICOM MC20901) used to support both High-Speed (HS) and Low-Power (LP) modes. It depicts an Image Sensor connected to the MC20901 via MIPI lanes and clock. The MC20901 then outputs HS signals to SmartFusion2/IGLOO2 FPGA's LVDS I/Os and LP signals to LVCMOS I/Os. An I2C interface is used for configuration.

6 Resource Utilization

The following table shows resource utilization for implementing MIPI CSI-2 receive only in the SmartFusion2 and IGLOO2 family of devices. These design utilization numbers do not include image processing IP that may be required to further process received sensor data. This table shows information for only the Deserializer/Aligner and MIPI decoder blocks:

7 Conclusion

This application note provides an overview of how to use SmartFusion2 and IGLOO2 devices to design MIPI CSI-2 interfaces. It also discusses a few limitations of SmartFusion2 and IGLOO2 devices in this context, and offers alternatives to overcome them.