Telemetry a Junos don AI/ML Workloads
Marubuci: Shalini Mukherjee

Gabatarwa

Kamar yadda tarin AI yana buƙatar cibiyoyin sadarwa marasa asara tare da babban kayan aiki da ƙarancin jinkiri, muhimmin abu na cibiyar sadarwar AI shine tarin bayanan sa ido. Junos Telemetry yana ba da damar saka idanu mai mahimmanci na mahimman alamun aiki, gami da ƙofa da ƙididdiga don sarrafa cunkoso da daidaita nauyi. Zaman gRPC yana goyan bayan watsa bayanan telemetry. gRPC zamani ne, buɗaɗɗen tushe, babban tsarin aiki wanda aka gina akan jigilar HTTP/2. Yana ba da damar iyawar yawo na gida biyu kuma ya haɗa da sassauƙan metadata na al'ada a cikin buƙatun kai. Mataki na farko a cikin telemetry shine sanin menene bayanan da za a tattara. Daga nan za mu iya yin nazarin wannan bayanai ta hanyoyi daban-daban. Da zarar mun tattara bayanan, yana da mahimmanci mu gabatar da su a cikin tsari mai sauƙin saka idanu, yanke shawara da inganta sabis ɗin da ake bayarwa. A cikin wannan takarda, muna amfani da tarin telemetry wanda ya ƙunshi Telegraf, InfluxDB, da Grafana. Wannan tarin telemetry yana tattara bayanai ta amfani da ƙirar turawa. Samfuran ja na al'ada suna da ƙarfin albarkatu, suna buƙatar sa hannun hannu, kuma suna iya haɗawa da gibin bayanai a cikin bayanan da suke tattarawa. Tsarin turawa sun shawo kan waɗannan iyakoki ta hanyar isar da bayanai ba tare da izini ba. Suna wadatar bayanan ta hanyar amfani da abokantaka mai amfani tags da sunaye. Da zarar bayanan sun kasance cikin sigar da za a iya karantawa, muna adana su a cikin ma'ajin bayanai kuma mu yi amfani da su a cikin hangen nesa web aikace-aikace don nazarin hanyar sadarwa. Hoto. 1 yana nuna mana yadda aka ƙera wannan tari don ingantaccen tattara bayanai, adanawa, da hangen nesa, daga na'urorin sadarwar da ke tura bayanai zuwa mai tarawa zuwa bayanan da ake nunawa akan dashboards don bincike.

Farashin TIG

Mun yi amfani da uwar garken Ubuntu don shigar da duk software ciki har da tarin TIG.

Telegraph
Don tattara bayanai, muna amfani da Telegraf akan uwar garken Ubuntu da ke gudana 22.04.2. Sigar Telegraf da ke gudana a cikin wannan demo shine 1.28.5.
Telegraf wakili ne mai sarrafa plugin don tattarawa da ba da rahoton awo. Yana amfani da processor plugins don haɓakawa da daidaita bayanan. Abubuwan da aka fitar plugins Ana amfani da su don aika wannan bayanan zuwa ɗakunan ajiya daban-daban. A cikin wannan takarda muna amfani da biyu plugins: ɗaya don na'urori masu auna firikwensin buɗe ido, ɗayan kuma don na'urori masu auna firikwensin Juniper.
InfluxDB
Don adana bayanan a cikin jerin bayanai na lokaci, muna amfani da InfluxDB. Kayan aikin fitarwa a cikin Telegraf yana aika bayanan zuwa InfluxDB, wanda ke adana shi cikin ingantaccen tsari. Muna amfani da V1.8 kamar yadda babu CLI yanzu don V2 da sama.
Grafana
Ana amfani da Grafana don ganin wannan bayanan. Grafana yana jan bayanan daga InfluxDB kuma yana bawa masu amfani damar ƙirƙirar dashboards masu wadata da ma'amala. Anan, muna gudanar da sigar 10.2.2.

Kanfigareshan Akan Sauyawa

Don aiwatar da wannan tari, da farko muna buƙatar saita maɓalli kamar yadda aka nuna a hoto na 2. Mun yi amfani da tashar jiragen ruwa 50051. Ana iya amfani da kowace tashar jiragen ruwa a nan. Shiga zuwa canjin QFX kuma ƙara saitin mai zuwa.

Lura: Wannan saitin na Labs/POCs ne kamar yadda ake watsa kalmar sirri a cikin rubutu mai haske. Yi amfani da SSL don guje wa wannan.

Muhalli

Nginx
Ana buƙatar wannan idan ba za ku iya fallasa tashar tashar da Grafana ke ɗaukar nauyinsa ba. Mataki na gaba shine shigar da nginx akan uwar garken Ubuntu don zama wakili na baya. Da zarar an shigar da nginx, ƙara layin da aka nuna a cikin Hoto 4 zuwa fayil ɗin "tsoho" kuma matsar da fayil ɗin daga /etc/nginx zuwa /etc/nginx/sites-enabled.

Tabbatar cewa an daidaita bangon wuta don ba da cikakkiyar dama ga sabis na nginx kamar yadda aka nuna a hoto 5.

Da zarar an shigar da nginx kuma an yi canje-canjen da ake buƙata, yakamata mu sami damar shiga Grafana daga a web browser ta amfani da adireshin IP na uwar garken Ubuntu inda aka shigar da duk software.
Akwai ƙaramin kuskure a cikin Grafana wanda baya barin ku sake saita kalmar sirri ta tsoho. Yi amfani da waɗannan matakan idan kun shiga cikin wannan batu.
Matakan da za a yi akan uwar garken Ubuntu don saita kalmar wucewa a Grafana:

• Je zuwa /var/lib/grafana/grafana.db
• Shigar sqlite3
  o sudo apt shigar sqlite3
• Guda wannan umarni akan tashar ku
  o sqlite3 grafana.db
•  Umurnin Sqlite yana buɗewa; gudanar da tambaya mai zuwa:
  > share daga mai amfani inda login = 'admin'
• Sake kunna grafana kuma rubuta admin azaman sunan mai amfani da kalmar wucewa. Yana haifar da sabon kalmar sirri.

Da zarar an shigar da duk software, ƙirƙiri fayil ɗin config a cikin Telegraf wanda zai taimaka cire bayanan telemetry daga maɓalli kuma tura shi zuwa InfluxDB.

Openconfig Sensor Plugin

A kan uwar garken Ubuntu, shirya fayil ɗin /etc/telegraf/telegraf.conf don ƙara duk abin da ake buƙata. plugins da na'urori masu auna sigina. Don na'urori masu auna buɗe ido, muna amfani da kayan aikin gNMI da aka nuna a Hoto na 6. Don dalilai na demo, ƙara sunan mai masauki a matsayin "spine1", lambar tashar jiragen ruwa "50051" da ake amfani da ita don gRPC, sunan mai amfani da kalmar wucewa ta sauya, da lambar. na daƙiƙa don sake bugawa idan an gaza.
A cikin tsarin biyan kuɗi, ƙara suna na musamman, “cpu” don wannan firikwensin na musamman, hanyar firikwensin, da tazarar lokacin ɗaukar wannan bayanai daga maɓalli. Ƙara abubuwan shigar plugin iri ɗaya.gnmi da shigarwar.gnmi.subscription don duk buɗaɗɗen firikwensin firikwensin. (Hoto na 6)

Filogin Sensor na asali

Wannan plugin ɗin juniper telemetry ne plugin ɗin da aka yi amfani da shi don firikwensin asali. A cikin fayil ɗin telegraf.conf iri ɗaya, ƙara abubuwan abubuwan firikwensin firikwensin na asali.jti_openconfig_telemetry inda filayen sun kusan iri ɗaya da buɗewa. Yi amfani da ID na abokin ciniki na musamman don kowane firikwensin; Anan, muna amfani da "telegraf3". Sunan na musamman da aka yi amfani da shi a nan don wannan firikwensin shine "mem" (Hoto 7).

Ƙarshe, ƙara abin fitarwa plugins.influxdb don aika wannan bayanan firikwensin zuwa InfluxDB. Anan, bayanan suna suna “telegraf” tare da sunan mai amfani azaman “influx” da kalmar sirri “influxdb” (Hoto 8).

Da zarar kun gyara fayil ɗin telegraf.conf, sake kunna sabis ɗin telegraf. Yanzu, bincika InfluxDB CLI don tabbatar da idan an ƙirƙiri ma'auni don duk na'urori masu auna firikwensin. Buga "influx" don shigar da InfluxDB CLI.

Kamar yadda aka gani a Figure. 9, shigar da hanzarin influxDB kuma yi amfani da bayanan "telegraf". Duk sunaye na musamman da aka ba na'urori masu auna firikwensin an jera su azaman ma'auni.
Don ganin fitowar kowane ma'auni, kawai don tabbatar da cewa fayil ɗin telegraf daidai ne kuma firikwensin yana aiki, yi amfani da umarnin "zaɓi * daga iyakar cpu 1" kamar yadda aka nuna a hoto 10.

Duk lokacin da aka yi canje-canje zuwa fayil ɗin telegraf.conf, tabbatar da dakatar da InfluxDB, sake kunna Telegraf, sannan fara InfluxDB.
Shiga Grafana daga mai binciken kuma ƙirƙirar dashboards bayan tabbatar da cewa ana tattara bayanan daidai.
Je zuwa Haɗi> InfuxDB> Ƙara sabon tushen bayanai.

1. Bada suna ga wannan tushen bayanan. A cikin wannan demo shine "test-1".
2.  Karkashin HTTP stanza, yi amfani da IP uwar garken Ubuntu da tashar jiragen ruwa 8086.
   
3. A cikin cikakkun bayanai na InfluxDB, yi amfani da sunan bayanai iri ɗaya, “telegraf,” kuma samar da sunan mai amfani da kalmar wucewa ta uwar garken Ubuntu.
4. Danna Ajiye & Gwaji. Tabbatar cewa kun ga sakon, "nasara".
   
5. Da zarar an ƙara tushen bayanan cikin nasara, je zuwa Dashboards kuma danna Sabo. Bari mu ƙirƙiri ƴan dashboards waɗanda ke da mahimmanci ga ayyukan AI/ML a cikin yanayin edita.

ExampLes Na Sensor Graphs

Wadannan su ne examples na wasu manyan ƙididdiga waɗanda ke da mahimmanci don sa ido kan hanyar sadarwa ta AI/ML.
Kashitage amfani don ingress interface et-0/0/0 akan kashin baya-1

• Zaɓi tushen bayanai azaman gwaji-1.
• A cikin sashin FROM, zaɓi ma'auni azaman "interface". Wannan shine musamman suna da aka yi amfani da shi don wannan hanyar firikwensin.
• A cikin sashin WHERE, zaɓi na'ura::tag, kuma a cikin tag darajar, zaɓi sunan mai masaukin maɓalli, wato, spine1.
• A cikin sashin SELECT, zaɓi reshen firikwensin da kake son saka idanu; a cikin wannan yanayin zaɓi "filin (/ musaya / mu'amala [if_name='et-0/0/0']/state/counters/if_in_1s_octets)”. Yanzu a cikin wannan sashe, danna "+" kuma ƙara wannan lissafin lissafin (/50000000000 * 100). Ainihin muna ƙididdige kashi ɗayatage amfani da wani 400G dubawa.
• Tabbatar cewa FORMAT shine "jerin lokaci," kuma suna suna jadawali a sashin ALIAS.

Matsakaicin wurin buffer ga kowane jerin gwano

• Zaɓi tushen bayanai azaman gwaji-1.
• A cikin sashin FROM, zaɓi ma'auni azaman "maɓalli."
• A cikin sashin INA, akwai fage guda uku da za a cika. Zaɓi na'ura::tag, kuma a cikin tag darajar zaɓi sunan mai masaukin mai sauyawa (watau spine-1); Kuma zaɓi /cos/musammam/interface/@name::tag kuma zaɓi abin dubawa (watau et- 0/0/0); Kuma zaɓi jerin gwano kuma, /cos/interfaces/interface/queues/queue/@queue::tag kuma zaɓi lambar layi 4.
• A cikin sashin SELECT, zaɓi reshen firikwensin da kake son saka idanu; a wannan yanayin zaɓi "filin (/cos/interfaces/interface/queues/queue/PeakBuffeerOccupancy)."
• Tabbatar cewa FORMAT shine "jerin lokaci" kuma suna suna jadawali a sashin ALIAS.

Kuna iya tattara bayanai don musaya masu yawa akan jadawali ɗaya kamar yadda aka gani a Hoto 17 don et-0/0/0, et-0/0/1, et-0/0/2 da sauransu.

PFC da ECN suna nufin asali

Don gano ma'anar abin da aka samo asali (bambancin ƙima a cikin kewayon lokaci), yi amfani da ɗanyen yanayin tambaya.
Wannan ita ce tambayar influx da muka yi amfani da ita don nemo ma'anar ma'anar tsakanin ƙimar PFC guda biyu akan et-0/0/0 na Spine-1 a cikin daƙiƙa guda.
Zaɓi abin da aka samo asali (ma'ana ("/ musaya / mu'amala [if_name='et-0/0/0′]/state/pfc-counter/tx_pkts”), 1s) DAGA “interface” INA (“na'ura”::tag = 'Spine-1') DA $Tsarin GROUP TA lokaci ($ tazara)

Zabi abin da aka samo asali (ma'ana ("/ musaya / mu'amala [if_name = 'et-0/0/8′]/state/error-counters/ecn_ce_marked_pkts”), 1s) DAGA “interface” INA (“na'ura”::tag = 'Spine-1') DA $Tsarin GROUP TA lokaci ($ tazara)

Kurakuran kayan shigar da bayanai suna nufin asali

Da ɗanyen tambayar kurakuran albarkatu na nufin abin da aka samo asali shine:
Zaɓi abin da aka samo asali (ma'ana ("/ musaya / mu'amala [if_name='et-0/0/0′]/state/error-counters/if_in_resource_errors”), 1s) DAGA “interface” INA (“na’ura”::tag = 'Spine-1') DA $Tsarin GROUP TA lokaci ($ tazara)

Digon wutsiya yana nufin asali

Ma'anar ɗanyen tambaya don sauke wutsiya ma'anar asali ita ce:
Zaɓi abin da aka samo asali (ma'ana ("/cos/musulunci/musamman musaya/layi/ jerin gwano/tailDropBytes")), 1s) DAGA "bufier" INA ("na'ura"::tag = 'Leaf-1' DA "/ cos / musaya / mu'amala / @ suna"::tag = 'et-0/0/0' DA "/cos/musulunci/interface/layi/ jerin gwano/@queue"::tag = '4') DA $timeTace GROUP BY lokaci($__tazara) cika (rasa)
 CPU amfani

• Zaɓi tushen bayanai azaman gwaji-1.
• A cikin sashin FROM, zaɓi ma'auni azaman "newcpu"
• A cikin INA, akwai fage guda uku da za a cika. Zaɓi na'ura::tag kuma a cikin tag ƙima zaɓi sunan mai masaukin sauyawa (watau spine-1). KUMA a cikin /bangaren/bangaren/kayayyaki/dukiya/suna:tag, kuma zaɓi cpuutilization-total AND in name::tag zaɓi RE0.
• A cikin sashin SELECT, zaɓi reshen firikwensin da kake son saka idanu. A wannan yanayin, zaɓi "filin (jihar/darajar)".

Da ɗanyen tambaya don nemo abin da ba maras kyau na wutsiya yana faɗuwa don sauyawa da yawa akan musaya masu yawa a cikin rago/sec.
SELECT non_negative_derivative(ma'ana ("/cos/musulunci/interface/queues/queue/tailDropBytes"),1s)*8 DAGA "bufier" INA (na'urar::tag =~ / ^ Spine-[1-2]$/) da ("/cos/interfaces/interface/@name"::tag =~ /et-0\/0\/[0-9]/ ko "/cos/interfaces/interface/@name"::tag=~/et-0\/0\/1[0-5]/) DA $timeTace GROUP BY lokaci($__tazara),na'ura::tag cika (babu)

Waɗannan su ne wasu daga cikin tsohonamples na jadawali waɗanda za a iya ƙirƙira don sa ido kan hanyar sadarwa ta AI/ML.

Takaitawa

Wannan takarda tana misalta hanyar jan bayanan telemetry da hangen nesa ta hanyar ƙirƙirar hotuna. Wannan takarda tana magana musamman game da na'urori masu auna firikwensin AI/ML, na asali da na buɗe ido amma ana iya amfani da saitin don kowane nau'in firikwensin. Mun kuma haɗa mafita don batutuwa da yawa waɗanda za ku iya fuskanta yayin ƙirƙirar saitin. Matakan da abubuwan da aka bayyana a cikin wannan takarda sun keɓanta ga nau'ikan tarin TIG da aka ambata a baya. Yana iya canzawa dangane da nau'in software, firikwensin da sigar Junos.

Magana

Juniper Yang Data Model Explorer don duk zaɓuɓɓukan firikwensin
https://apps.juniper.net/ydm-explorer/
Openconfig forum don openconfig na'urori masu auna firikwensin
https://www.openconfig.net/projects/models/

Hedikwatar Kamfanin da Talla
Abubuwan da aka bayar na Juniper Networks, Inc.
1133 Innovation Way
Sunnyvale, CA 94089 Amurka
Waya: 888. JUNIPER (888.586.4737)
ko +1.408.745.2000
Fax: +1.408.745.2100
www.juniper.net
APAC da EMEA Headquarter
Juniper Networks International BV
Hanyar Boeing 240
1119 PZ Schiphol-Rijk
Amsterdam, Netherlands
Waya: +31.207.125.700
Fax: +31.207.125.701
Haƙƙin mallaka 2023 Juniper Networks. Inc. Ail haƙƙin mallaka. Juniper Networks, alamar Juniper Networks, Juniper, Junos, da sauran alamun kasuwanci alamun kasuwanci ne masu rijista na Juniper Networks. inc. da/ko masu haɗin gwiwa a Amurka da wasu ƙasashe. Wasu sunaye na iya zama alamun kasuwanci na masu su. Juniper Networks ba ta da alhakin kowane kuskure a cikin wannan takaddar. Juniper Networks tana da haƙƙin canzawa. gyara. canja wuri, ko in ba haka ba a sake duba wannan ɗaba'ar ba tare da sanarwa ba.
Aika ra'ayi zuwa: design-center-comments@juniper.net V1.0/240807/ejm5-telemetry-junos-ai-ml

Takardu / Albarkatu

Juniper NETWORKS Telemetry A Junos don AI ML Aiki Software [pdf] Jagorar mai amfani Telemetry A cikin Junos don AI ML Kayan Aikin Aiki, Junos don AI ML Kayan Aiki Software, AI ML Aiki Software, Kayan Aiki Software, Software

Magana

• Manual mai amfani