Telemetry i Junos mo AI / ML Workloads
Tusitala: Shalini Mukherjee

Folasaga

Talu ai e mana'omia e feso'ota'iga fa'apipi'i AI ni feso'ota'iga e leai se gau ma e maualuga le gaosiga ma maualalo le taofiofia, o se elemene taua o le feso'ota'iga AI o le aoina lea o fa'amatalaga mata'ituina. O le Junos Telemetry e mafai ai ona mata'ituina fa'ailoga autu o fa'atinoga, e aofia ai faitoto'a ma fa'atatau mo le fa'atonutonuina o le fa'atosina ma le paleni o uta. gRPC sauniga e lagolagoina le tafega o faamatalaga telemetry. gRPC ose fa'aonaponei, tatala-puna, maualuga fa'atinoga fa'avae e fausia ile HTTP/2 felauaiga. E fa'amalosia ai le fa'agasologa fa'ale-a'oa'oga fa'ale-aganu'u ma e aofia ai fetu'utu'unai-metadata i ulutala talosaga. O le laasaga muamua i le telemetry o le iloa lea o faʻamaumauga e ao ona aoina. Ona mafai lea ona tatou suʻesuʻeina nei faʻamatalaga i faʻatulagaga eseese. O le taimi lava matou te aoina ai faʻamaumauga, e taua le tuʻuina atu i se faatulagaga e faigofie ona mataʻituina, fai filifiliga ma faʻaleleia le auaunaga o loʻo ofoina atu. I totonu o lenei pepa, matou te faʻaogaina le telemetry stack e aofia ai Telegraf, InfluxDB, ma Grafana. O lenei fa'aputuga telemetry e aoina fa'amaumauga e fa'aaoga ai se fa'ata'ita'iga tulei. O fa'ata'ita'iga toso fa'aleaganu'u e fa'atosina tele puna'oa, e mana'omia ai le fa'aogaina o le lima, ma e mafai ona aofia ai fa'amatalaga avanoa i fa'amaumauga latou te aoina. O fa'ata'ita'iga Push e fa'ato'ilaloina nei tapula'a e ala i le tu'uina atu o fa'amaumauga e aunoa ma se fa'atasi. Latou te faʻatamaoaigaina faʻamatalaga e ala i le faʻaogaina o tagata faʻaoga tags ma igoa. O le taimi lava e maua ai faʻamatalaga i se faʻasologa e sili atu ona mafai ona faitau, matou te teuina i totonu o se faʻamaumauga ma faʻaaogaina i se vaʻaiga vaʻaia web talosaga mo le su'esu'eina o feso'otaiga. Ata. 1 o lo'o fa'aali mai ia i tatou le fa'afuafuaina o lenei fa'aputuga mo le aoina lelei o fa'amaumauga, teuina, ma le va'aiga, mai masini feso'ota'iga o lo'o tuleia fa'amaumauga i le tagata aoina i fa'amaumauga o lo'o fa'aalia i luga o dashboards mo su'esu'ega.

TIG faaputuga

Na matou faʻaogaina se Ubuntu server e faʻapipiʻi uma polokalama e aofia ai le TIG stack.

Telegraf
Ina ia aoina faʻamatalaga, matou te faʻaogaina Telegraf i luga o le Ubuntu server o loʻo faʻaogaina le 22.04.2. O le Telegraf version o loʻo taʻavale i lenei demo o le 1.28.5.
Telegraf o se sui faʻapipiʻi faʻapipiʻiina mo le aoina ma le lipotia o metotia. E fa'aogaina le processor plugins e faʻatamaoaigaina ma faʻavasega faʻamaumauga. Le galuega faatino plugins e fa'aaogaina e lafo ai fa'amaumauga nei i faleoloa fa'amaumauga eseese. I totonu o lenei pepa matou te faʻaaogaina lua plugins: tasi mo leo fa'apipi'i matala ma le isi mo fa'atagata Juniper.
InfluxDB
Ina ia teuina faʻamaumauga i se faʻamaumauga faʻasologa o taimi, matou te faʻaogaina le InfluxDB. O le mea fa'apipi'i i Telegraf e tu'uina atu fa'amaumauga i le InfluxDB, lea e teu ai i se faiga sili ona lelei. O loʻo matou faʻaogaina le V1.8 aua e leai se CLI o loʻo iai mo V2 ma luga.
Grafana
E faʻaaogaina le Grafana e vaʻaia ai lenei faʻamatalaga. E toso e Grafana fa'amaumauga mai le InfluxDB ma fa'ataga ai tagata fa'aoga e fa'atupu fa'amaumauga mauoa ma fegalegaleai. O iinei, o loʻo matou faʻaogaina le version 10.2.2.

Fa'atonuga i luga ole Suiga

Mo le fa'atinoina o lenei fa'aputuga, e mana'omia muamua ona fa'atulaga le ki e pei ona fa'aalia i le Ata 2. Sa matou fa'aogaina le port 50051. So'o se uafu e mafai ona fa'aoga iinei. Ulufale i le QFX ki ma fa'aopoopo le fa'atulagaga lea.

Fa'aaliga: O lenei fa'atulagaga e mo labs/POCs ona o le fa'aupuga o lo'o tu'uina atu i tusitusiga manino. Faʻaaoga SSL e aloese mai ai.

Siosiomaga

Nginx
E manaʻomia lenei mea pe a le mafai ona e faʻaalia le uafu o loʻo talimalo ai Grafana. O le isi laasaga o le faʻapipiʻi o le nginx i luga o le Ubuntu server e avea ma sui sui sui. A maeʻa ona faʻapipiʻi le nginx, faʻaopoopo laina o loʻo faʻaalia i le Ata 4 i le "default" faila ma faʻanofo le faila mai /etc/nginx i /etc/nginx/sites-enabled.

Ia mautinoa ua fetuutuunai le pa puipui ina ia maua ai le avanoa atoatoa i le auaunaga nginx e pei ona faaalia i le Ata 5.

O le taimi lava e faʻapipiʻi ai le nginx ma faia suiga manaʻomia, e tatau ona mafai ona tatou maua le Grafana mai le a web browser e ala i le faʻaogaina o le tuatusi IP o le Ubuntu server lea e faʻapipiʻi uma ai le polokalama.
O loʻo i ai se faʻalavelave laʻititi i Grafana e le faʻatagaina oe e toe setiina le upu faʻamaonia. Fa'aaoga laasaga nei pe a e feagai ma lenei mataupu.
Laasaga e faia i luga o le Ubuntu server e seti ai le upu faʻamaonia i Grafana:

• Alu ile /var/lib/grafana/grafana.db
• Faʻapipiʻi le sqlite3
  ole sudo apt faʻapipiʻi le sqlite3
• Fa'atonu le fa'atonuga i luga o lau fa'amau
  o sqlite3 grafana.db
•  Sqlite command prompt tatala; fai le fesili lea:
  > tape mai le tagata fa'aoga lea e saini = 'admin'
• Toe amata le grafana ma lolomi le admin e avea ma username ma upu faataga. E fa'atonuina mo se fa'aupuga fou.

O le taimi lava e fa'apipi'i uma ai le polokalama, fai le faila fa'amau i Telegraf e fesoasoani e toso mai fa'amatalaga telemetry mai le ki ma tulei i le InfluxDB.

Openconfig Sensor Plugin

I luga o le Ubuntu server, fa'asa'o le faila /etc/telegraf/telegraf.conf e fa'aopoopo mea uma e mana'omia plugins ma sensors. Mo le fa'aogaina o masini, matou te fa'aogaina le gNMI plugin o lo'o fa'aalia i le Ata 6. Mo fa'amoemoega fa'ata'ita'iga, fa'aopoopo le igoa talimalo e pei o le "spine1", le numera o le taulaga "50051" lea e fa'aoga mo le gRPC, le igoa ole igoa ma le fa'aupuga ole ki, ma le numera. o sekone mo le toe viliina pe a faaletonu.
I le stanza o le saofaga, faʻaopoopo se igoa tulaga ese, "cpu" mo lenei mea faʻapitoa, le ala faʻalogo, ma le taimi vaʻa mo le puʻeina o nei faʻamatalaga mai le ki. Fa'aopoopo le mea fa'apipi'i tutusa inputs.gnmi ma inputs.gnmi.subscription mo masini fa'apipi'i matala uma. (Ata 6)

Fa'apipi'i fa'a-native Sensor

Ole Juniper telemetry interface plugin lea e fa'aogaina mo masini fa'aletino. I le faila telegraf.conf lava e tasi, fa'aopoopo i ai le fa'apipi'i masini fa'apipi'i inputs.jti_openconfig_telemetry lea e toetoe lava tutusa ma le openconfig. Fa'aoga se ID tagata fa'atau mo so'o se masini; iinei, matou te faʻaaogaina le "telegraf3". O le igoa tulaga ese o loʻo faʻaogaina iinei mo lenei masini o le "mem" (Ata 7).

O le mea mulimuli, fa'aopoopo se mea fa'apipi'i fa'apipi'i outputs.influxdb e lafo atu ai fa'amatalaga fa'alogo i le InfluxDB. O iinei, o le database ua faaigoaina "telegraf" ma le igoa ole igoa "influx" ma le upu "influxdb" (Ata 8).

A uma loa ona e fa'asa'o le faila telegraf.conf, toe amata le auaunaga telegraf. Ia, siaki i totonu o le InfluxDB CLI e fa'amautinoa pe ua faia fua mo masini tulaga ese uma. Fa'aigoa "influx" e ulufale i le InfluxDB CLI.

E pei ona vaaia i le Ata. 9, ulufale i le influxDB vave ma faʻaoga le database "telegraf". O igoa fa'apitoa uma e tu'uina atu i masini e lisiina o ni fua.
Ina ia vaʻai i le gaioiga o soʻo se fua e tasi, ia mautinoa e saʻo le faila telegraf ma o loʻo galue le masini, faʻaaoga le poloaiga "filifili * mai le cpu limit 1" e pei ona faʻaalia i le Ata 10.

Soo se taimi lava e fai ai suiga i le faila telegraf.conf, ia mautinoa e taofi le InfluxDB, toe amata Telegraf, ona amata lea o le InfluxDB.
Fa'amau i le Grafana mai le su'esu'ega ma fai fa'ailoga laupapa pe a uma ona fa'amautinoa o lo'o aoina sa'o fa'amaumauga.
Alu i Feso'ota'iga> InfuxDB> Fa'aopoopo fa'amaumauga fou.

1. Tu'u se igoa ile puna'oa lea. I lenei demo o le "su'ega-1".
2.  I lalo o le HTTP stanza, faʻaaoga le Ubuntu server IP ma le 8086 port.
   
3. I fa'amatalaga InfluxDB, fa'aoga le igoa fa'amaumauga e tasi, "telegraf," ma tu'u le igoa ole igoa ma le fa'aupuga ole server Ubuntu.
4. Kiliki Save & test. Ia mautinoa e te vaʻai i le savali, "manuia".
   
5. A mae'a ona fa'aopoopo le puna fa'amatalaga, alu ile Dashboards ma kiliki Fou. Sei o tatou faia ni nai lisi laupapa e taua mo AI / ML galuega mamafa i le faʻatonu faʻatonu.

Examples Of Sensor Graphs

O mea nei o exampo nisi fa'atau tetele e mana'omia mo le mata'ituina o feso'otaiga AI/ML.
Pasenetage fa'aogaina mo se atina'e ulufale et-0/0/0 i luga ole tui-1

• Filifili le puna o faʻamatalaga e pei ole suʻega-1.
• I le vaega FROM, filifili le fuaina o le "interface". O le igoa tulaga ese lea e fa'aoga mo lenei ala fa'alogo.
• I le vaega WHERE, filifili masini::tag, ma i totonu o le tag tau, filifili le igoa talimalo o le ki, o lona uiga, spine1.
• I le vaega FILIFILI, filifili le lala sensor e te manaʻo e mataʻituina; i le tulaga lea, filifili le "field(/interfaces/interface[if_name='et-0/0/0']/state/counters/if_in_1s_octets)". I le taimi nei i le vaega lava e tasi, kiliki i luga o le "+" ma faʻaopopo le numera numera (/50000000000 * 100). O lo'o matou fa'atatauina le pasenetage fa'aogaina se fa'aoga 400G.
• Ia mautinoa o le FORMAT o le "taimi-faasologa," ma faaigoa le kalafi i le vaega ALIAS.

Ole pito maualuga ole nofoia mo so'o se laina

• Filifili le puna o faʻamatalaga e pei ole suʻega-1.
• I le vaega FROM, filifili le fua o le "buffer."
• I le vaega WHERE, e tolu malae e fa'atumu. Filifili masini::tag, ma i totonu o le tag tau filifili le igoa talimalo o le ki (ie spine-1); MA filifili /cos/interfaces/interface/@name::tag ma filifili le atina'e (ie et- 0/0/0); MA filifili foi le laina, /cos/interfaces/interface/queues/queue/@queue::tag ma filifili le laina numera 4.
• I le vaega FILIFILI, filifili le lala sensor e te manaʻo e mataʻituina; i le tulaga lea, filifili le "field(/cos/interfaces/interface/queues/queue/PeakBuferOccupancy)."
• Ia mautinoa o le FORMAT o le "taimi-faasologa" ma faaigoa le kalafi i le vaega ALIAS.

E mafai ona e tu'ufa'atasia fa'amaumauga mo le tele o feso'ota'iga i luga o le kalafi tutusa e pei ona va'aia i le Ata 17 mo et-0/0/0, et-0/0/1, et-0/0/2 etc.

PFC ma le ECN o lona uiga o mea e maua mai

Mo le su'eina o le fua fa'atatau (o le ese'esega o le tau i totonu o le taimi), fa'aaoga le faiga o fesili mata'utia.
Ole su'esu'ega lea na matou fa'aogaina e su'e ai le uiga fa'atatau i le va o tau PFC e lua ile et-0/0/0 ole Spine-1 ile sec.
FILIFILI mea e maua mai(o lona uiga(“/interfaces/interface[if_name='et-0/0/0′]/state/pfc-counter/tx_pkts”), 1s) MAI “interface” WHERE (“mea faigaluega”::tag = 'Spine-1') MA le $timeFilter GROUP I LE taimi($vala)

FILIFILI mea e maua mai(o lona uiga(“/interfaces/interface[if_name='et-0/0/8′]/state/error-counters/ecn_ce_marked_pkts”), 1s) MAI “interface” WHERE (“mea faigaluega”::tag = 'Spine-1') MA le $timeFilter GROUP I LE taimi($vala)

O mea sese i totonu o punaoa o lona uiga o mea e maua mai

O le fesili mata'utia mo mea sese o lona uiga o mea e maua mai e:
FILIFILI mea e maua mai(o lona uiga(“/interfaces/interface[if_name='et-0/0/0′]/state/error-counters/if_in_resource_errors”), 1s) MAI “interface” WHERE (“mea faigaluega”::tag = 'Spine-1') MA le $timeFilter GROUP I LE taimi($vala)

Fa'asu'u si'usi'u uiga fa'atupu

Ole su'esu'ega mata'utia mo pa'u si'usi'u o lona uiga ole fa'atatau ole:
FILIFILI mea e maua mai(o lona uiga(“/cos/interfaces/interface/queues/queue/tailDropBytes”), 1s) MAI le “buffer” WHERE (“meafaigaluega”::tag = 'Lau-1' MA “/cos/interfaces/interface/@name”::tag = 'et-0/0/0' MA “/cos/interfaces/interface/queues/queue/@queue”::tag = '4') MA le $timeFilter VULEGA I LE taimi($__vala) fa'atumu(null)
 fa'aogaina o le PPU

• Filifili le puna o faʻamatalaga e pei ole suʻega-1.
• I le vaega FROM, filifili le fua o le "newcpu"
• I le WHERE, e tolu matā'ele e fa'atumu. Filifili masini::tag ma i totonu o le tag tau filifili le igoa talimalo o le ki (ie tui-1). MA i totonu / vaega / vaega / meatotino / meatotino / igoa:tag, ma filifili cpuutilization-total MA ile igoa::tag filifili RE0.
• I le vaega FILIFILI, filifili le lala sensor e te manaʻo e mataʻituina. I le tulaga lea, filifili le "field(state/value)".

Ole su'esu'ega mata'utia mo le su'eina ole fa'aleaga ole si'usi'u e pa'u mo le tele o suiga ile tele o feso'ota'iga ile bits/sec.
FILIFILI non_negative_derivative(mean(“/cos/interfaces/interface/queues/queue/tailDropBytes”), 1s)*8 MAI “buff” WHERE (masini::tag =~ /^Spine-[1-2]$/) ma (“/cos/interfaces/interface/@name”::tag =~ /et-0\/0\/[0-9]/ po o le “/cos/interfaces/interface/@name”::tag=~/et-0\/0\/1[0-5]/) MA LE $timeFilter VAEGA I LE taimi($__ vaeluaga), masini::tag fa'atumu(null)

O nisi nei o mea muamuaample tele o kalafi e mafai ona faia mo le mataʻituina o se fesoʻotaʻiga AI/ML.

Aotelega

O lenei pepa o loʻo faʻaalia ai le auala o le tosoina o faʻamatalaga telemetry ma vaʻaia e ala i le fatuina o kalafi. O lenei pepa o loʻo talanoa faʻapitoa e uiga i AI / ML sensors, e le gata i le natura ma le openconfig ae o le seti e mafai ona faʻaoga mo ituaiga uma o masini. Ua matou fa'aofi fo'i fofo mo le tele o fa'afitauli e ono feagai ma oe a'o fa'atūina le seti. O laasaga ma galuega faatino o loʻo faʻaalia i lenei pepa e faʻapitoa i faʻasologa o le TIG stack na taʻua muamua. E mafai ona suia e faʻatatau i le faʻaogaina o le polokalama, masini ma le Junos version.

Fa'asinomaga

Juniper Yang Data Model Explorer mo filifiliga faʻalogo uma
https://apps.juniper.net/ydm-explorer/
Openconfig forum mo masini fa'apipi'i
https://www.openconfig.net/projects/models/

Autasi ma Sales Headquarters
Juniper Networks, Inc.
1133 Auala Fou
Sunnyvale, CA 94089 ISA
Telefoni: 888. IUNIPER (888.586.4737)
po’o le +1.408.745.2000
Fax: +1.408.745.2100
www.juniper.net
APAC ma EMEA Headquarters
Juniper Networks International BV
Boeing Avenue 240
1119 PZ Schiphol-Rijk
Amsterdam, Netherlands
Telefoni: +31.207.125.700
Fax: +31.207.125.701
Puletaofia 2023 Juniper Networks. Inc. Ail aia tatau fa'asao. Juniper Networks, le logo Juniper Networks, Juniper, Junos, ma isi fa'ailoga o fa'ailoga fa'amaufa'ailoga a Juniper Networks. inc. ma/po o ana paaga i le Iunaite Setete ma isi atunuu. O isi igoa atonu o fa'ailoga fa'ailoga a latou tagata e ona. E leai se matafaioi a Juniper Networks mo soʻo se mea le saʻo i totonu o lenei pepa. Juniper Networks fa'asaoina le aia tatau e sui ai. fesuia'i. fesiita'i, po'o seisi toe teuteu lenei lomiga e aunoa ma se fa'aaliga.
Auina atu manatu i: design-center-comments@juniper.net V1.0/240807/ejm5-telemetry-junos-ai-ml

Pepa / Punaoa

Juniper NETWORKS Telemetry In Junos mo AI ML Workloads Software [pdf] Taiala mo Tagata Fa'aoga Telemetry I Junos mo AI ML Workloads Software, Junos mo AI ML Workloads Software, AI ML Workloads Software, Workloads Software, Software

Fa'asinomaga

• Tusi Taiala