intel OPAE FPGA Linux Device Driver Architecture

OPAE Intel FPGA Linux Device Driver Architecture

OPAE Intel FPGA tsav tsheb muab kev sib tshuam rau cov neeg siv-qhov chaw siv los teeb tsa, sau npe, qhib, thiab nkag mus rau FPGA accelerators ntawm cov platforms nruab nrog Intel FPGA cov kev daws teeb meem thiab ua kom cov txheej txheem tswj hwm qib xws li FPGA reconfiguration, tswj fais fab, thiab virtualization.

Kho vajtse Architecture

Los ntawm OS's point of view, FPGA kho vajtse zoo li ib qho khoom siv PCIe li niaj zaus. Lub FPGA lub cim xeeb ntaus ntawv tau teeb tsa siv cov ntaub ntawv teev tseg ua ntej (Device Feature List). Cov yam ntxwv txhawb nqa los ntawm FPGA ntaus ntawv tau nthuav tawm los ntawm cov ntaub ntawv no, raws li tau piav qhia hauv qab no hauv daim duab hauv qab no:

FPGA PCIe ntaus ntawv

Tus neeg tsav tsheb txhawb nqa PCIe SR-IOV los tsim Virtual Functions (VFs) uas tuaj yeem siv los muab cov accelerators rau cov tshuab virtual.

Intel Corporation. Txhua txoj cai. Intel, Intel logo, thiab lwm yam Intel cov cim yog cov cim lag luam ntawm Intel Corporation lossis nws cov koom haum. Intel lav qhov kev ua tau zoo ntawm nws cov FPGA thiab cov khoom siv semiconductor rau cov kev qhia tshwj xeeb tam sim no raws li Intel tus qauv warranty tab sis muaj txoj cai los hloov cov khoom thiab cov kev pabcuam txhua lub sijhawm yam tsis muaj ntawv ceeb toom. Intel xav tias tsis muaj lub luag haujlwm lossis kev lav phib xaub uas tshwm sim los ntawm daim ntawv thov lossis siv cov ntaub ntawv, cov khoom lag luam, lossis cov kev pabcuam uas tau piav qhia hauv no tshwj tsis yog raws li tau pom zoo hauv kev sau ntawv los ntawm Intel. Intel cov neeg siv khoom tau qhia kom tau txais qhov tseeb version ntawm cov cuab yeej tshwj xeeb ua ntej tso siab rau cov ntaub ntawv tshaj tawm thiab ua ntej muab xaj rau cov khoom lossis cov kev pabcuam.

Lwm lub npe thiab cov npe yuav raug lees paub tias yog cov cuab yeej ntawm lwm tus.

Virtualized FPGA PCIe Ntaus

FPGA Management Cav (FME)
FPGA Management Engine ua lub zog thiab kev tswj xyuas thermal, kev qhia yuam kev, kev teeb tsa, kev qhia txog kev ua haujlwm, thiab lwm yam kev ua haujlwm. Txhua FPGA muaj ib qho FME, uas ib txwm nkag los ntawm Lub Cev Muaj Zog (PF). Cov neeg siv-chaw thov tuaj yeem tau txais kev nkag mus rau FME tshwj xeeb siv qhib(), thiab tso nws siv ze() raws li tus neeg siv muaj cai (hauv paus).

Chaw nres nkoj
Ib qhov chaw nres nkoj sawv cev rau qhov sib cuam tshuam ntawm FPGA cov ntaub ntawv zoo li qub ("FPGA Interface Manager (FIM)") thiab thaj chaw hloov kho ib nrab uas muaj Accelerator Function (AF). Chaw nres nkoj tswj kev sib txuas lus los ntawm software mus rau lub accelerator thiab nthuav tawm cov yam ntxwv xws li rov pib dua thiab kho qhov teeb meem. Lub PCIe ntaus ntawv yuav muaj ntau qhov chaw nres nkoj, thiab txhua qhov chaw nres nkoj tuaj yeem nthuav tawm los ntawm VF los ntawm kev muab nws siv FPGA_FME_PORT_ASSIGN ioctl ntawm FME ntaus ntawv.

Accelerator Function (AF) Unit

• Lub Chaw Ua Haujlwm Accelerator (AF) tau txuas nrog rau Chaw nres nkoj thiab nthuav tawm thaj tsam 256K MMIO los siv rau kev tswj xyuas tshwj xeeb ntawm cov npe.
• Cov neeg siv-qhov chaw thov tuaj yeem tau txais kev nkag mus rau AFU txuas rau Chaw nres nkoj los ntawm kev siv qhib() ntawm Chaw nres nkoj ntaus ntawv, thiab tso nws siv ze().
• Cov neeg siv-chaw thov tuaj yeem mmap() accelerator MMIO cheeb tsam.

Ib nrab Reconfiguration
Raws li tau hais los saum toj no, accelerators tuaj yeem rov kho dua los ntawm ib nrab ntawm kev teeb tsa ntawm Accelerator Function (AF) file. Lub Accelerator Function (AF) yuav tsum tau tsim rau qhov tseeb FIM thiab tsom mus rau thaj chaw zoo li qub (Port) ntawm FPGA; txwv tsis pub, lub reconfiguration ua hauj lwm yuav ua tsis tau tejyam thiab tejzaum nws ua rau system instability. Qhov kev sib raug zoo no tuaj yeem tshawb xyuas los ntawm kev sib piv cov interface ID sau tseg hauv AF header tiv thaiv tus interface ID nthuav tawm los ntawm FME los ntawm sysfs. Qhov kev kuaj no feem ntau yog ua los ntawm cov neeg siv qhov chaw ua ntej hu rau kev teeb tsa IOCTL.

Nco tseg:
Tam sim no, txhua qhov kev pab cuam software nkag mus rau FPGA, suav nrog cov khiav hauv virtualized host, yuav tsum raug kaw ua ntej sim ua ib feem ntawm kev teeb tsa. Cov kauj ruam yuav yog:

1. Unload tus tsav tsheb ntawm tus qhua
2. Tshem tawm VF ntawm tus qhua
3. Disable SR-IOV
4. Ua ib nrab reconfiguration
5. Qhib SR-IOV
6. Txuas VF rau tus qhua
7. Load tus tsav tsheb hauv qhua

FPGA Virtualization
Txhawm rau txhawm rau nkag mus rau lub nrawm nrawm los ntawm cov ntawv thov khiav hauv VM, qhov chaw AFU qhov chaw nres nkoj yuav tsum tau muab rau VF siv cov kauj ruam hauv qab no:

1. Lub PF muaj tag nrho AFU chaw nres nkoj los ntawm lub neej ntawd. Txhua qhov chaw nres nkoj uas yuav tsum tau muab rov qab rau VF yuav tsum xub tso tawm ntawm PF los ntawm FPGA_FME_PORT_RELEASE ioctl ntawm FME ntaus ntawv.
2. Thaum N cov chaw nres nkoj raug tso tawm los ntawm PF, cov lus txib hauv qab no tuaj yeem siv los pab SRIOV thiab VFs. Txhua VF tsuas muaj ib qho chaw nres nkoj nrog AFU. ncha N > PCI_DEVICE_PATH/sriov_numvfs
3. Dhau los ntawm VFs rau VMs.
4. AFU nyob rau hauv VF tuaj yeem siv tau los ntawm cov ntawv thov hauv VM (siv tib tus tsav tsheb hauv VF).

Nco tseg:
Ib qho FME tsis tuaj yeem raug xa mus rau VF, yog li PR thiab lwm yam kev tswj hwm tsuas yog muaj los ntawm PF.

Lub koom haum tsav tsheb

PCIe Module Device Driver

Lub koom haum tsav tsheb

Cov khoom siv FPGA tshwm sim li PCIe li niaj zaus; yog li, FPGA PCIe ntaus ntawv tsav tsheb (intel-FPGA-PCI.ko) ib txwm thauj khoom ua ntej thaum kuaj pom FPGA PCIe PF lossis VF. Tus neeg tsav tsheb no ua lub luag haujlwm hauv kev tsim kho hauv cov tsav tsheb architecture. Nws:

• Tsim ib qho FPGA ntim khoom ua niam txiv ntawm cov khoom siv tshwj xeeb.
• Taug kev los ntawm Cov Ntaus Ntaus Ntaus Ntaus, uas yog siv hauv PCIe ntaus ntawv BAR nco, txhawm rau tshawb pom cov khoom siv tshwj xeeb thiab lawv cov yam ntxwv thiab tsim cov khoom siv platform rau lawv hauv qab lub thawv ntim khoom.
• Txhawb SR-IOV.
• Qhia txog cov cuab yeej cuab tam cov cuab yeej cuab tam, uas ua haujlwm paub daws teeb meem rau cov yam ntxwv sib luag thiab nthuav tawm cov haujlwm zoo sib xws rau cov cuab yeej tsav tsheb.

PCIe Module Device Driver Functions

• Muaj PCIe discovery, ntaus ntawv enumeration, thiab feature discovery.
• Tsim sysfs cov npe rau cov khoom siv niam txiv, FPGA Management Engine (FME), thiab Chaw nres nkoj.
• Tsim lub platform tsav tsheb, ua rau Linux ntsiav thauj lawv cov platform module tsav tsheb.

FME Platform Module Device Driver

• Kev tswj fais fab thiab thermal, kev qhia yuam kev, kev qhia txog kev ua haujlwm, thiab lwm yam kev ua haujlwm hauv vaj tse. Koj tuaj yeem nkag mus rau cov haujlwm no ntawm sysfs interfaces nthuav tawm los ntawm FME tsav tsheb.
• Ib nrab Reconfiguration. Tus neeg tsav tsheb FME sau npe rau FPGA Tus Thawj Coj thaum lub sijhawm PR sub-feature pib; ib zaug nws tau txais FPGA_FME_PORT_PR ioctl los ntawm koj, nws invokes qhov kev sib txuas ua haujlwm los ntawm FPGA Tus Thawj Saib Xyuas kom ua tiav cov kev teeb tsa ib nrab ntawm bitstream rau qhov chaw nres nkoj muab.
• Chaw nres nkoj tswj rau virtualization. Tus tsav tsheb FME qhia ob lub ioctls, FPGA_FME_PORT_RELEASE, uas tso tawm qhov chaw nres nkoj muab los ntawm PF; thiab FPGA_FME_PORT_ASSIGN, uas muab qhov chaw nres nkoj rov qab rau PF. Thaum Chaw nres nkoj raug tso tawm los ntawm PF, nws tuaj yeem raug xa mus rau VF los ntawm SR-IOV interfaces muab los ntawm PCIe tsav tsheb. Yog xav paub ntxiv, xa mus rau "FPGA Virtualization".

FME Platform Module Device Driver Functions

• Tsim FME cim ntaus ntawv node.
• Tsim FME sysfs files thiab siv FME sysfs file accessors.
• Siv cov FME ntiag tug feature sub-drivers.
• FME private feature sub-drivers:
  • FME Header
  • Thermal Management
  • Tswj fais fab
  • Ntiaj teb no yuam kev
  • Ib nrab Reconfiguration
  • Ntiaj teb no Performance

Chaw nres nkoj Platform Module Device Driver
Zoo ib yam li tus tsav tsheb FME, FPGA Chaw nres nkoj (thiab AFU) tus tsav tsheb (intel-fpga-afu. ko) raug soj ntsuam ib zaug Port platform ntaus ntawv tsim. Lub luag haujlwm tseem ceeb ntawm cov qauv no yog muab kev sib txuas rau cov neeg siv-qhov chaw siv kom nkag mus rau tus kheej cov neeg siv nrawm, suav nrog kev pib pib tswj ntawm Chaw nres nkoj, AFU MMIO thaj av xa tawm, DMA tsis pom kev pabcuam, UMsg (1) ceeb toom, thiab cov chaw taws teeb debug ua haujlwm ( saib saum toj no).

UMsg tsuas yog txhawb nqa los ntawm Acceleration Stack rau Intel Xeon® Processor nrog Integrated FPGA.

Chaw nres nkoj Platform Module Device Driver Functions

• Tsim qhov chaw nres nkoj cim ntaus ntawv node.
• Tsim qhov chaw nres nkoj sysfs files thiab siv qhov chaw nres nkoj sysfs file accessors.
• Siv cov chaw nres nkoj ntiag tug feature sub-drivers.
• Chaw nres nkoj private feature sub-drivers:
  • Chaw nres nkoj Header
  • AFU
  • Chaw nres nkoj yuam kev
  • UAS (2)
  • Teeb liab Kais

Daim ntawv thov FPGA Device Enumeration
Tshooj lus no qhia txog yuav ua li cas cov ntawv sau npe FPGA ntaus ntawv los ntawm sysfs hierarchy nyob rau hauv /sys/class/fpga. Hauv example hauv qab no, ob lub Intel FPGA li tau nruab rau hauv tus tswv tsev. Txhua FPGA ntaus ntawv muaj ib qho FME thiab ob qhov chaw nres nkoj (AFUs). Rau txhua lub FPGA ntaus ntawv, ib phau ntawv qhia ntaus ntawv yog tsim nyob rau hauv /sys/class/fpga:

/sys/class/fpga/intel-fpga-dev.0
/sys/class/fpga/intel-fpga-dev.1

Txhua lub node muaj ib qho FME thiab ob qhov chaw nres nkoj (AFUs) ua cov khoom siv me me:
/sys/class/fpga/intel-fpga-dev.0/intel-fpga-fme.0
/sys/class/fpga/intel-fpga-dev.0/intel-fpga-port.0
/sys/class/fpga/intel-fpga-dev.0/intel-fpga-port.1
/sys/class/fpga/intel-fpga-dev.1/intel-fpga-fme.1
/sys/class/fpga/intel-fpga-dev.1/intel-fpga-port.2
/sys/class/fpga/intel-fpga-dev.1/intel-fpga-port.3

Feem ntau, FME / Port sysfs interfaces muaj npe raws li hauv qab no:
/sys/class/fpga/intel-fpga-dev.i/intel-fpga-fme.j/
/sys/class/fpga/intel-fpga-dev.i/intel-fpga-port.k/

nrog kuv txuas ntxiv suav nrog txhua lub thawv ntim khoom, j sib txuas ua tus lej FME thiab k sib txuas ua tus lej ntawm txhua qhov chaw nres nkoj.

Cov cuab yeej siv rau ioctl() thiab mmap() tuaj yeem raug xa mus los ntawm:
/dev/intel-fpga-fme.j
/dev/intel-fpga-port.k

PCIe Driver Enumeration
Tshooj lus no muab tshaj tawmview ntawm tus lej ntws rau ntaus ntawv suav ua los ntawm intel-fpga-pci.ko. Cov ntaub ntawv tseem ceeb thiab cov haujlwm tseem ceeb tau hais tseg. Tshooj lus no zoo tshaj plaws ua raws li thaum viewnrog rau qhov chaws code (pcie.c).

Enumeration Data Structures

enum fpga_id_type {
PARENT_ID,
FME_ID,
PORT_ID,
FPGA_ID_MAX
};
static struct idr fpga_ids[FPGA_ID_MAX];
struct fpga_chardev_info {
const char * npe;
dev_t devt;
};
struct fpga_chardev_info fpga_chrdevs[] = {
{ .name = FPGA_FEATURE_DEV_FME },
{ .name = FPGA_FEATURE_DEV_PORT },
};
static struct class *fpga_class;
static struct pci_device_id cci_pcie_id_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_RCiEP0_MCP),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_VF_MCP),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_RCiEP0_SKX_P),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_VF_SKX_P),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_RCiEP0_DCP),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIe_DEVICE_ID_VF_DCP),},
{0,}
};
static struct pci_driver cci_pci_driver = {
.name = DRV_NAME,
.id_table = cci_pcie_id_tbl,
.probe = cci_pci_probe,
.remove = cci_pci_remove,
.sriov_configure = cci_pci_sriov_configure
};
struct cci_drvdata {
int device_id;
struct device *fme_dev;
struct mutex xauv;
struct list_head port_dev_list;
int tso_port_num;
struct list_head thaj tsam;
};
struct build_feature_devs_info {
struct pci_dev *pdev;
void __ioem *ioaddr;
void __ioem *ioend;
int current_bar;
void __ioem *pfme_hdr;
struct device *parent_dev;
struct platform_device *feature_dev;
};

Kev suav sau Flow

• ccidrv_init()
  • Pib pib fpga_ids siv idr_init().
  • Pib pib fpga_chrdevs[i].devt siv alloc_chrdev_region().
  • Pib pib fpga_class siv class_create().
  • pci_register_driver(&cci_pci_driver);
• cci_pci_probe()
  • Qhib lub PCI ntaus ntawv, thov nkag mus rau nws cov cheeb tsam, teeb tsa PCI tus tswv hom, thiab teeb tsa DMA.
• cci_pci_create_feature_devs() build_info_alloc_and_init()
  • faib cov struct build_feature_devs_info, pib nws.
    .parent_dev yog teem rau niam txiv sysfs directory (intel-fpga-dev.id) uas muaj FME thiab Port sysfs directory.
• parse_feature_list()
  • Taug kev BAR0 Device Feature List kom pom FME, Chaw nres nkoj, thiab lawv tus kheej nta.
• parse_feature() parse_feature_afus() parse_feature_fme()
  • Thaum ntsib FME:
• build_info_create_dev()
  • Muab lub platform ntaus ntawv rau FME, khaws cia hauv build_feature_devs_info.feature_dev.
  • feature_dev.id yog pib rau qhov tshwm sim ntawm idr_alloc(fpga_ids[FME_ID],
  • feature_dev.parent yog teem rau build_feature_devs_info.parent_dev.
  • faib cov array ntawm cov khoom siv hauv feature_dev.resource.
• Faib tus qauv feature_platform_data, pib nws, thiab khaws tus taw tes rau hauv feature_dev.dev.platform_data
  • create_feature_instance() build_info_add_sub_feature()
  • Pib pib feature_dev.resource[FME_FEATURE_ID_HEADER].
  • feature_platform_data_add()
  • Initialize feature_platform_data.features[FME_FEATURE_ID_HEADER], txhua yam tab sis .fops.
• parse_feature() parse_feature_afus() parse_feature_port()
  • Thaum ntsib qhov chaw nres nkoj:
• build_info_create_dev()
  • Muab lub platform ntaus ntawv rau Chaw nres nkoj, khaws cia hauv build_feature_devs_info.feature_dev.
  • feature_dev.id yog pib rau qhov tshwm sim ntawm idr_alloc(fpga_ids[PORT_ID],
  • feature_dev.parent yog teem rau build_feature_devs_info.parent_dev.
  • faib cov array ntawm struct resource hauv feature_dev.resource.
  • Faib tus qauv feature_platform_data, pib nws, thiab khaws tus taw tes rau hauv feature_dev.dev.platform_data
• build_info_commit_dev()
  • Ntxiv cov struct feature_platform_data.node rau Chaw nres nkoj rau cov npe ntawm cov chaw nres nkoj hauv struct cci_drvdata.port_dev_list
• create_feature_instance() build_info_add_sub_feature()
  • Pib pib feature_dev.resource[PORT_FEATURE_ID_HEADER].
• feature_platform_data_add()
  • Initialize feature_platform_data.features[PORT_FEATURE_ID_HEADER], txhua yam tab sis .fops.
• parse_feature() parse_feature_afus() parse_feature_port_uafu()
  • Thaum ntsib AFU:
• create_feature_instance() build_info_add_sub_feature()
  • Initialize feature_dev.resource[PORT_FEATURE_ID_UAFU].
• feature_platform_data_add()
  • Initialize feature_platform_data.features[PORT_FEATURE_ID_UAFU], txhua yam tab sis .fops.
• parse_feature() parse_feature_private() parse_feature_fme_private()
  • Thaum ntsib FME ntiag tug feature:
• create_feature_instance() build_info_add_sub_feature()
  • Pib pib feature_dev.resource[id].
• feature_platform_data_add()
  • Initialize feature_platform_data.features[id], txhua yam tab sis .fops.
• parse_feature() parse_feature_private() parse_feature_port_private()
• Thaum ntsib qhov chaw nres nkoj ntiag tug: * create_feature_instance() build_info_add_sub_feature() * Initialize feature_dev.resource[id]. * feature_platform_data_add() pib feature_platform_data.features[id], txhua yam tab sis .fops.
• parse_ports_from_fme()
  • Yog tias tus tsav tsheb thauj khoom ntawm lub cev muaj nuj nqi (PF), ces:
• Khiav qhov parse_feature_list() ntws ntawm txhua qhov chaw nres nkoj tau piav qhia hauv FME header.
• Siv BAR hais nyob rau hauv txhua qhov chaw nres nkoj nkag hauv header.

FME Platform Device Initialization
Tshooj lus no muab tshaj tawmview ntawm tus lej ntws rau FME ntaus ntawv pib ua los ntawm intel-fpga-fme.ko. Cov ntaub ntawv tseem ceeb thiab kev ua haujlwm yog highlited. Tshooj lus no zoo tshaj plaws ua raws li thaum viewnrog rau qhov chaws code (fme-main.c).

FME Platform Device Data Structures

struct feature_ops {
int (*init)(struct platform_device *pdev, struct feature *feature);
int (*uinit)(struct platform_device *pdev, struct feature *feature);
long (*ioctl)(struct platform_device *pdev, struct feature *feature,
unsigned int cmd, unsigned ntev arg);
int (*test)(struct platform_device *pdev, struct feature *feature);
};
struct feature {
const char * npe;
int resource_index;
void __ioem *ioaddr;
struct feature_ops *ops;
};
struct feature_platform_data {
struct list_head node;
struct mutex xauv;
unsigned ntev dev_status;
struct cdev cdev;
struct platform_device *dev;
unsigned int disable_count;
void * ntiag tug;
int num;
int (*config_port)(struct platform_device *, u32, bool);
struct platform_device *(*fpga_for_each_port)(struct platform_device *,
void *, int (*match)(struct platform_device *, void *)); qauv
feature nta[0];
};
struct perf_object {
rau cov menyuam id;
const struct attribute_group **attr_groups;
struct device *fme_dev;
struct list_head node;
struct list_head cov me nyuam;
struct kobject kobj;
};
struct fpga_fme {
u8 port_id;
u64 pwm;
struct device *dev_err;
struct perf_object *perf_dev;
struct feature_platform_data *pdata;
};

FME Platform Device Initialization Flow

FME Initialization Flow

• fme_probe() fme_dev_init()
  • Pib ib qho struct fpga_fme thiab khaws cia rau hauv qhov feature_platform_data.private teb.
• fme_probe() fpga_dev_feature_init() feature_instance_init()
  • Txuag ib qho struct feature_ops rau hauv feature_platform_data.features rau txhua qhov muaj nyob.
  • Hu rau qhov kev xeem ua haujlwm, yog tias muaj, los ntawm cov qauv.
  • Hu rau init muaj nuj nqi los ntawm struct.
• fme_probe() fpga_register_dev_ops()
  • Tsim lub FME cim ntaus ntawv node, sau npe tus qauv file_ kev ua haujlwm.

Chaw nres nkoj Platform Device Initialization
Tshooj lus no muab tshaj tawmview ntawm tus lej ntws rau qhov chaw nres nkoj pib pib ua los ntawm intel-fpga-afu.ko. Cov ntaub ntawv tseem ceeb thiab cov haujlwm tseem ceeb tau hais tseg. Tshooj lus no zoo tshaj plaws ua raws li thaum viewnyob rau hauv lub accompanying source code (afu.c).

Chaw nres nkoj Platform Device Data Structures

struct feature_ops {
int (*init)(struct platform_device *pdev, struct feature *feature);
int (*uinit)(struct platform_device *pdev, struct feature *feature);
long (*ioctl)(struct platform_device *pdev, struct feature *feature,
unsigned int cmd, unsigned ntev arg);
int (*test)(struct platform_device *pdev, struct feature *feature);
};
struct feature {
const char * npe;
int resource_index;
void __ioem *ioaddr;
struct feature_ops *ops;
};
struct feature_platform_data {
struct list_head node;
struct mutex xauv;
unsigned ntev dev_status;
struct cdev cdev;
struct platform_device *dev;
unsigned int disable_count;
void * ntiag tug;
int num;
int (*config_port)(struct platform_device *, u32, bool);
struct platform_device *(*fpga_for_each_port)(struct platform_device *,
void *, int (*match)(struct platform_device *, void *));
struct feature nta[0];
};
struct fpga_afu_region {
u32 npe;.
u32 npe;.
u64 loj;.
u64 ua;.
ua 64;.
struct list_head node;
};
struct fpga_afu_dma_region {
u64 user_addr;
u64 loj;.
u64 ua;.
struct page ** nplooj;
struct rb_node ntawm;
bool in_siv;
};
struct fpga_afu {
u64 region_cur_offset;
hauv num_regions;
u8 npe;.
struct list_head thaj tsam;
struct rb_root dma_regions;
struct feature_platform_data *pdata;
};

Chaw nres nkoj Platform Device Initialization Flow

Chaw nres nkoj Initialization Flow

• afu_probe() afu_dev_init()
  • Pib ua ib qho struct fpga_afu thiab khaws cia rau hauv qhov feature_platform_data.private teb.
• afu_probe() fpga_dev_feature_init() feature_instance_init()
  • Txuag ib qho struct feature_ops rau hauv feature_platform_data.features rau txhua qhov muaj nyob.
  • Hu rau qhov kev xeem ua haujlwm, yog tias muaj, los ntawm cov qauv.
  • Hu rau init muaj nuj nqi los ntawm struct.
• afu_probe() fpga_register_dev_ops()
  • Tsim qhov chaw nres nkoj cim ntaus ntawv node, sau npe tus qauv file_ kev ua haujlwm.

FME IOCTLs
IOCTLs uas raug hu ua qhib file tus piav qhia rau /dev/intel-fpga-fme.j FPGA_GET_API_VERSION—rov qab cov qauv tam sim no ua tus lej, pib ntawm 0.

FPGA_CHECK_EXTENSION—tsis tau txais kev txhawb nqa tam sim no.

FPGA_FME_PORT_RELEASE—arg yog tus taw tes rau:

struct fpga_fme_port_release {
__u32 argsz; // hauv: sizeof(struct fpga_fme_port_release)
__u32 chij; // hauv: yuav tsum yog 0
__u32 port_id; // hauv: chaw nres nkoj ID (los ntawm 0) tso tawm.
};

FPGA_FME_PORT_ASSIGN-arg yog tus taw tes rau:

struct fpga_fme_port_assign {
__u32 argsz; // hauv: sizeof(struct fpga_fme_port_assign)
__u32 chij; // hauv: yuav tsum yog 0
__u32 port_id; // hauv: chaw nres nkoj ID (los ntawm 0) los muab. (yuav tsum tau
yav dhau los tso tawm los ntawm FPGA_FME_PORT_RELEASE)
};

FPGA_FME_PORT_PR—arg yog tus taw tes rau:

struct fpga_fme_port_pr {
__u32 argsz; // hauv: sizeof(struct fpga_fme_port_pr)
__u32 chij; // hauv: yuav tsum yog 0
__u32 port_id; // hauv: chaw nres nkoj ID (los ntawm 0)
__u32 buffer_size; // hauv: qhov loj me ntawm bitstream tsis nyob hauv bytes. Yuav tsum yog 4-byte
ua ke.
__u64 buffer_address; // nyob rau hauv: txheej txheem chaw nyob ntawm bitstream buffer
__u64 xwm txheej; // tawm: yuam kev xwm txheej (bitmask)
};

Chaw nres nkoj IOCTLs
IOCTLs uas raug hu ua qhib file tus piav qhia rau /dev/intel-fpga-port.k FPGA_GET_API_VERSION—rov qab cov ntawv tam sim no ua tus lej, pib ntawm 0. FPGA_CHECK_EXTENSION—tsis tau txhawb tam sim no.

FPGA_PORT_GET_INFO—arg yog tus taw tes rau:

struct fpga_port_info {
__u32 argsz; // hauv: sizeof(struct fpga_port_info)
__u32 chij; // tawm: rov 0
__u32 num_regions; // tawm: tus naj npawb ntawm MMIO cheeb tsam, 2 (1 rau AFU thiab 1 rau
STP)
__u32 num_umsgs; // tawm: tus naj npawb ntawm UMsg tau txais kev txhawb nqa los ntawm kev kho vajtse
};

FPGA_PORT_GET_REGION_INFO—arg yog tus taw tes rau:

struct fpga_port_region_info {
__u32 argsz; // hauv: sizeof(struct fpga_port_region_info)
__u32 chij; // tawm: (bitmask) { FPGA_REGION_READ, FPGA_REGION_WRITE,
FPGA_REGION_MMAP }
__u32 index; // in: FPGA_PORT_INDEX_UAFU or FPGA_PORT_INDEX_STP
__u32 padding; // hauv: yuav tsum yog 0
__u64 loj; // tawm: qhov loj ntawm MMIO cheeb tsam hauv bytes
__u64 offset; // tawm: offset ntawm MMIO cheeb tsam los ntawm pib ntawm ntaus ntawv fd
};

FPGA_PORT_DMA_MAP—arg yog tus taw tes rau:
struct fpga_port_dma_map {
__u32 argsz; // hauv: sizeof(struct fpga_port_dma_map)
__u32 chij; // hauv: yuav tsum yog 0 __u64 user_addr; // hauv: txheej txheem virtual
chaw nyob. Yuav tsum yog nplooj ntawv aligned.
__u64 ntev; // hauv: qhov ntev ntawm daim ntawv qhia hauv bytes. Yuav tsum yog ntau nplooj ntawv
qhov loj.
__u64 iova; // tawm: IO virtual chaw nyob };

FPGA_PORT_DMA_UNMAP—arg yog tus taw tes rau:
struct fpga_port_dma_unmap {
__u32 argsz; // hauv: sizeof(struct fpga_port_dma_unmap)
__u32 chij; // hauv: yuav tsum yog 0
__u64 iova; // hauv: IO virtual chaw nyob xa rov qab los ntawm yav dhau los
FPGA_PORT_DMA_MAP };

• FPGA_PORT_RESET—arg yuav tsum yog NULL.
• FPGA_PORT_UMSG_ENABLE—arg yuav tsum yog NULL.
• FPGA_PORT_UMSG_DISABLE—args yuav tsum yog NULL.

FPGA_PORT_UMSG_SET_MODE—arg yog tus taw tes rau:

struct fpga_port_umsg_cfg {
__u32 argsz; // hauv: sizeof(struct fpga_port_umsg_cfg)
__u32 chij; // hauv: yuav tsum yog 0
__u32 hint_bitmap; // hauv: UMsg hint hom bitmap. Qhia qhov twg UMsg yog
qhib.
};

FPGA_PORT_UMSG_SET_BASE_ADDR—

• UMsg yuav tsum raug kaw ua ntej muab ioctl no.
• Iova teb yuav tsum yog rau qhov tsis loj txaus rau tag nrho UMsg's (num_umsgs * PAGE_SIZE).
  • Qhov tsis yog cim tias "siv" los ntawm tus neeg tsav tsheb qhov kev tswj tsis tau.
  • Yog tias iova yog NULL, ib cheeb tsam yav dhau los tsis raug cim tias "siv".
• arg yog tus taw tes rau a:
  struct fpga_port_umsg_base_addr {
  • u32 seb;. // hauv: sizeof(struct fpga_port_umsg_base_addr)
  • u32 npe;. // hauv: yuav tsum yog 0
  • u64 ua;. // hauv: IO virtual chaw nyob los ntawm FPGA_PORT_DMA_MAP. };

Nco tseg:

• Txhawm rau tshem tawm qhov yuam kev ntawm qhov chaw nres nkoj, koj yuav tsum sau qhov tseeb bitmask ntawm qhov yuam kev tam sim no, rau example, cat errors > clear
• UMsg tsuas yog txhawb nqa los ntawm Acceleration Stack rau Intel Xeon Processor nrog Integrated FPGA.

sysfs ua Files

FME Header sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/

sysfs ua file	mmio teb	hom	nkag mus
ports_num	fme_header.capability.num_ports	decimal int	Nyeem nkaus xwb
cache_size	fme_header.capability.cache_size	decimal int	Nyeem nkaus xwb
version	fme_header.capability.fabric_verid	decimal int	Nyeem nkaus xwb
socket_id	fme_header.capability.socket_id	decimal int	Nyeem nkaus xwb
bitstream_id	fme_header.bitstream_id	hex 64 t	Nyeem nkaus xwb
bitstream_metadata	fme_header.bitstream_md	hex 64 t	Nyeem nkaus xwb

FME Thermal Management sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/thermal_mgmt/

sysfs ua file	mmio teb	hom	nkag mus
qib 1	thermal.threshold.tmp_thshold1	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
qib 2	thermal.threshold.tmp_thshold2	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
threshold_trip	thermal.threshold.therm_trip_thshold	decimal int	Nyeem nkaus xwb
threshold1_reached	thermal.threshold.thshold1_status	decimal int	Nyeem nkaus xwb
threshold2_reached	thermal.threshold.thshold2_status	decimal int	Nyeem nkaus xwb
threshold1_policy	thermal. threshold.thshold_policy	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
kub	thermal.rdsensor_fm1.fpga_temp	decimal int	Nyeem nkaus xwb

FME Power Management sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/power_mgmt/

sysfs ua file	mmio teb	hom	nkag mus
noj	power.status.pwr_consumed	hex 64 t	Nyeem nkaus xwb
qib 1	power.threshold.threshold1	hex 64 t	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
qib 2	power.threshold.threshold2	hex 64 t	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
threshold1_status	power.threshold.threshold1_status	decimal tsis kos npe	Nyeem nkaus xwb
threshold2_status	power.threshold.threshold2_status	decimal tsis kos npe	Nyeem nkaus xwb
rtl ua	power.status.fpga_latency_report	decimal tsis kos npe	Nyeem nkaus xwb

FME Ntiaj teb no yuam kev sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/errors/

sysfs ua file	mmio teb	hom	nkag mus
pcie0_ yuam kev	gerror.pcie0_err	hex 64 t	Nyeem-sau
pcie1_ yuam kev	gerror.pcie1_err	hex 64 t	Nyeem-sau
txhaj_error	gerror.ras_error_inj	hex 64 t	Nyeem-sau

intel-fpga-dev.i/intel-fpga-fme.j/errors/fme-errors/

sysfs ua file	mmio teb	hom	nkag mus
yuam kev	gerror.fme_err	hex 64 t	Nyeem nkaus xwb
thawj_error	gerror.fme_first_err.err_reg_status	hex 64 t	Nyeem nkaus xwb
tom ntej_ yuam kev	gerror.fme_next_err.err_reg_status	hex 64 t	Nyeem nkaus xwb
meej	Clears yuam kev, first_error, next_error	ntau uint64_t	Sau nkaus xwb

Nco tseg:
Txhawm rau tshem tawm qhov yuam kev FME, koj yuav tsum sau qhov tseeb bitmask ntawm qhov yuam kev tam sim no, rau example miv errors > clear.

FME Ib Feem Reconfiguration sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/pr/

sysfs ua file	mmio teb	hom	nkag mus
interface_id	pr.fme_pr_intfc_id0_h, pr.fme_pre_intfc_id0_l	hex 16 byte	Nyeem nkaus xwb

FME Ntiaj Teb Kev Ua Haujlwm sysfs files
intel-fpga-dev.i/intel-fpga-fme.j/dperf/clock

sysfs ua file	mmio teb	hom	nkag mus
moos	gperf.clk.afu_interf_clock	hex 64 t	Nyeem nkaus xwb

intel-fpga-dev.i/intel-fpga-fme.j/dperf/cache/ (Tsis siv tau rau Acceleration Stack rau Intel Xeon CPU nrog FPGAs)

sysfs ua file	mmio teb	hom	nkag mus
khov	gperf.ch_ctl.freeze	decimal int	Nyeem-sau
nyeem_ntaus	gperf.CACHE_RD_HIT	hex 64 t	Nyeem nkaus xwb
nyeem_miss	gperf.CACHE_RD_MISS	hex 64 t	Nyeem nkaus xwb
sau_hit	gperf.CACHE_WR_HIT	hex 64 t	Nyeem nkaus xwb
sau_miss	gperf.CACHE_WR_MISS	hex 64 t	Nyeem nkaus xwb
tuav_request	gperf.CACHE_HOLD_REQ	hex 64 t	Nyeem nkaus xwb
tx_req_stall	gperf.CACHE_TX_REQ_STALL	hex 64 t	Nyeem nkaus xwb

sysfs ua file	mmio teb	hom	nkag mus
rx_req_stall	gperf.CACHE_RX_REQ_STALL	hex 64 t	Nyeem nkaus xwb
data_write_port_kev sib cav	gperf.CACHE_DATA_WR_PORT_CONTEN	hex 64 t	Nyeem nkaus xwb
tag_write_port_kev sib cav	gperf.CACHE_TAG_WR_PORT_CONTEN	hex 64 t	Nyeem nkaus xwb

intel-fpga-dev.i/intel-fpga-fme.j/dperf/iommu/ (Tsis siv tau rau Acceleration Stack rau Intel Xeon CPU nrog FPGAs)

sysfs ua file	mmio teb	hom	nkag mus
khov	gperf.vtd_ctl.freeze	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau

intel-fpga-dev.i/intel-fpga-fme.j/dperf/iommu/afuk/ (Tsis siv tau rau Acceleration Stack rau Intel Xeon CPU nrog FPGAs)

sysfs ua file	mmio teb	hom	nkag mus
read_transaction	gperf.VTD_AFU0_MEM_RD_TRANS	hex 64 t	Nyeem nkaus xwb
sau_transaction	gperf.VTD_AFU0_MEM_WR_TRANS	hex 64 t	Nyeem nkaus xwb
tlb_read_hit	gperf.VTD_AFU0_TLB_RD_HIT	hex 64 t	Nyeem nkaus xwb
tlb_write_hit	gperf.VTD_AFU0_TLB_WR_HIT	hex 64 t	Nyeem nkaus xwb

intel-fpga-dev.i/intel-fpga-fme.j/dperf/fab/

sysfs ua file	mmio teb	hom	nkag mus
pab	gperf.fab_ctl.(enabled)	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
khov	gperf.fab_ctl.freeze	decimal int	Tus neeg siv: Nyeem nkaus xwb hauv paus: Nyeem-sau
pcie0_ nyeem	gperf.FAB_PCIE0_RD	hex 64 t	Nyeem nkaus xwb
pcie0_write	gperf.FAB_PCIE0_WR	hex 64 t	Nyeem nkaus xwb
pcie1_ nyeem	gperf.FAB_PCIE1_RD	hex 64 t	Nyeem nkaus xwb
pcie1_write	gperf.FAB_PCIE1_WR	hex 64 t	Nyeem nkaus xwb
upi_ nyeem	gperf.FAB_UPI_RD	hex 64 t	Nyeem nkaus xwb
upi_write	gperf.FAB_UPI_WR	hex 64 t	Nyeem nkaus xwb

intel-fpga-ev.i/intel-fpga/fme.j/dperf/fabric/portk/

sysfs ua file	mmio teb	hom	nkag mus
pcie0_ nyeem	gperf.FAB_PCIE0_RD	hex 64 t	Nyeem nkaus xwb
pcie0_write	gperf.FAB_PCIE0_WR	hex 64 t	Nyeem nkaus xwb
pcie1_ nyeem	gperf.FAB_PCIE1_RD	hex 64 t	Nyeem nkaus xwb
pcie1_write	gperf.FAB_PCIE1_WR	hex 64 t	Nyeem nkaus xwb
upi_ nyeem	gperf.FAB_UPI_RD	hex 64 t	Nyeem nkaus xwb
upi_write	gperf.FAB_UPI_WR	hex 64 t	Nyeem nkaus xwb

Chaw nres nkoj Header sysfs files
intel-fpga-dev.i/intel-fpga-port.k/

sysfs ua file	mmio teb	hom	nkag mus
id	port_header.capability.port_number	decimal int	Nyeem nkaus xwb
ltr	port_header.control.latency_tolerance	decimal int	Nyeem nkaus xwb

Chaw nres nkoj AFU Header sysfs files
intel-fpga-dev.i/intel-fpga-port.k/

sysfs ua file	mmio teb	hom	nkag mus
ua_id	afu_header.guid ua	hex 16 byte	Nyeem nkaus xwb

Chaw nres nkoj yuam kev sysfs files
intel-fpga-dev.i/intel-fpga-port.k/errors/

sysfs ua file	mmio teb	hom	nkag mus
yuam kev	perror.port_error	hex 64 t	Nyeem nkaus xwb
thawj_error	perror.port_first_error	hex 64 t	Nyeem nkaus xwb
first_malformed_req	perror.malreq	hex 16 byte	Nyeem nkaus xwb
meej	perror.(txhua yam yuam kev)	ntau uint64_t	Sau nkaus xwb

Nco tseg:
Txhawm rau tshem tawm qhov yuam kev ntawm Chaw nres nkoj, koj yuav tsum sau qhov tseeb bitmask ntawm qhov yuam kev tam sim no, piv txwv liample miv errors > clear.

Kev kho keeb kwm

Cov ntaub ntawv Version	Hloov
2017.10.02	Thawj Tso.

OPAE Intel FPGA Linux Device Driver Architecture Guide

Cov ntaub ntawv / Cov ntaub ntawv

intel OPAE FPGA Linux Device Driver Architecture [ua pdf] Cov neeg siv phau ntawv qhia OPAE FPGA Linux Device Driver Architecture, OPAE FPGA, Linux Device Driver Architecture, Tsav Architecture, Architecture

Cov ntaub ntawv

• Cov neeg siv phau ntawv