L'écosystème Cisco Secure AI Factory avec Mistral.ai et NVIDIA
Patrice Nivaggioli
AI Engineer | Cisco EMEA
Track 4/BK 2
Agenda
- Introduction
- Cisco + NVIDIA: Bringing AI to the Enterprise
- Cisco + NVIDIA: Bringing Secure AI to the Enterprise
- Cisco + Mistral.ai: Bringing GenAI to the Enterprise
- Wrap-Up
Cisco AI Strategy
AI in Cisco
Networking, Security, Collaboration, Observability solutions
Examples:
- WIFI RRM in Catalyst Center
- SDWAN Path prediction
- Failure prediction with Thousand Eyes
- Malicious Activities with HyperShield
- Threat Detection in Splunk/XDR
AI with Cisco
API/SDKs, Agents, Agentic Frameworks, Open weights models
Examples:
- Object Detection with Meraki API
- Logs and Metrics AI Analysis with Splunk MLTK and DSDL
- AI Agents with Webex Contact Center fdrn.ai and agntcy.org
AI on Cisco
Cisco Secure AI Factory
- Accelerated Computing and Networking
- AI Datacenter Fabric
- AI Edge
- AI Observability and Security
Partnerships: Cisco, NVIDIA
AI use cases across industries
Employee Experience
Chatbots | AI assistants | Copilots
Customer Experience
Virtual agents | Specialized knowledge base
Business Process
Fraud Management | Regulatory Compliance | Energy Management | Smart Building | Supply Chain Planning | Industry/Utilities Digital Twin
AI Ops
Threats Detection/Prevention | Incident Response/Remediation | Automation | Root Cause Analysis | Network Digital Twins
Challenges with AI projects
delays time to value realization
Security vulnerabilities
AI models, frameworks, apps, and supporting infrastructure represent a new cyberattack surface
Network performance bottlenecks
Model training and inferencing generates a lot of traffic, slowing networks and delaying time-to-value
Complex AI infrastructure deployment
Lack of high-performance infrastructure with integrated compute, network, storage, and AI software can stall AI projects
Bringing AI to the Enterprise
Partnership: Cisco and NVIDIA
Cisco and NVIDIA expand partnership to accelerate AI adoption in the enterprise
Partnership focus
- Private data centers
- Backend AI networks
- Full-stack AI infrastructures
- Cisco's switching/control plane/security and infrastructure stack
How are we partnering?
- Cisco will be part of NVIDIA's Enterprise Reference Architecture
- Cisco will be part of NVIDIA's Cloud Partner Reference Architecture
- Deliver AI Factory with customer choice: Cisco Silicon One® or NVIDIA Spectrum-X Silicon
AI Datacenter Networks
Front End (N/S) (OOB) (Storage)
Scale Up (intra-node)
Back End (Scale-Out) (E/W)
LLM Training Parallelism and Algorithms
Pipeline parallelism: ncclSend, ncclReceive
Data parallelism: ncclAllReduce
Tensor Parallelism: ncclAllGather
And also: MoE (mixture of experts), ncclSend/nccIRecv (alltoall), FSDP (fully sharded data parallelism), ncclAllGather, and other variations.
Communication in a ring (or tree) is limited by the speed of the lowest link.
Collective Communication and RDMA
RDMA CPU Offload
RDMA Stack:
- RDMA Applications: Pytorch, TensorFlow, JAX
- RDMA Software Stack: NCCL, OpenMPI, GPFS
- IB Transport Protocol
- IB Network Layer
- Infiniband Link Layer
- Infiniband Management
- InfiniBand
- UDP
- IP
- Ethernet Link Layer
- Ethernet / IP Management
- RoCEv2
Backend Fabric for AI Training
- GPU Memory to GPU Memory RDMA
- Very High Bandwidth
- Very Low Latency
- Ultra Bursty Traffic (on/off)
- Low entropy flow and elephant flows
Technologies: Non blocking, Lossless, PFC, ECN, Packet Load Balancing
AI Ready Fabric: Design 1024 GPUs (8 x N9364E, 16 x N9364E)
Rail Optimized design
Bringing Secure AI to the Enterprise
Partnership: Cisco and NVIDIA
The new AI risk landscape
Consequences of unmanaged AI risk:
- Financial damage
- Litigation risk
- Reputational damage
- Compliance risk
- Security risk
- IP leakage
Example Scenario: A customer interacts with an AI chatbot for a car purchase, leading to a legally binding offer for $1.
Cisco Secure AI Factory with NVIDIA
- Security-first architecture
- High-performance, enterprise-proven networking, compute, storage, and AI software stack
- Pre-validated, with flexible deployment options
Components: AI/GenAI models, Enterprise data, NVIDIA AI Enterprise, Cisco UCS, Cisco Nexus Switches, NetApp, Pure Storage, VAST Data.
Outputs: Knowledgebase Copilots, Content & Code Generation, Virtual Agent & Chatbots, Detection & Predictions.
Security-first architecture enables safe Enterprise AI
Securing the Applications
Cisco AI Defense: Robust testing and runtime security of LLMs and generative AI applications.
Securing the Workloads
Cisco Hypershield: Protection against adversary lateral movement and proactive vulnerability mitigation without the need for patching, all from a single management interface.
Securing the Infrastructure
Integration with NVIDIA Bluefield-3's DOCA AppShield for intrusion detection in AI-focused virtual machines and containers. (Future)
Cisco Hybrid Mesh Firewall: Unified security management and consistent and pervasive policy across multiple enforcement points. (Future)
To include management of NVIDIA BlueField®-3 DPUs for enabling AI Cluster perimeter firewalls. (Future)
Cisco Secure AI Factory stack details
AI/GenAI Pipeline:
- NVIDIA: AI Enterprise, NeMo, NIMS, Blueprints
- Compute: Cisco UCS® w/ NVIDIA Accelerated Computing, NVIDIA BlueField®-3 DPUs, Managed w/ Cisco Intersight®
- Networking: Cisco Nexus 9300 w/ Silicon One® Switches w/ NVIDIA Spectrum-X*, Managed w/ Cisco Nexus Dashboard
- Storage: NetApp | Pure Storage | VAST Data
- Security: Cisco AI Defense, Cisco Hybrid Mesh Firewall, Isovalent, Secure Firewall, Hypershield
Outputs: Knowledgebase Copilots, Content & Code Generation, Virtual Agent & Chatbots, Detection & Predictions.
Modular system deployment and scaling with AI PODS
AI POD: An atomic unit of Secure AI Factory with NVIDIA, including NVIDIA AI Enterprise, Cisco UCS, and NetApp/Pure Storage/VAST Data.
Cisco Secure AI Factory with NVIDIA: Scalable architecture using AI PODs, incorporating Compute, Networking, and Security components.
What we build at Mistral AI
Frontier models
- Open and enterprise models released every month
- SOTA across language, code, vision
- Deeply customizable and deployable anywhere
Enterprise platforms
- Portable inference engine across datacenter, cloud, edge
- Enterprise interfaces for builders and users
- All the tooling required to start seeing ROI
Enterprise solutions
- Use case discovery and execution
- Custom model training
- Deployment, optimization, and scaling
Partners: Snowflake, CMA CGM, BNP PARIBAS, orange™, CISCO, STELLANTIS, CURSOR, Harve
Cisco and Mistral to accelerate GenAI adoption in the enterprise
Main focus areas
- Private enterprise architecture
- End-to-end customizability
- Enterprise Context
- Enterprise AI strategy
How are we collaborating?
- Enterprise Grade GenAI Platform
- Custom model training and enterprise interfaces development
- Deployment, Optimization and Scaling
La Plateforme
Applied AI and deployment services:
- AI tooling: Agents, Fine-tuning, Embedding, Function/tool calling, Distillation, Safety, Monitoring
- Library of frontier LLMs: SOTA language models, Small and edge models, Code models, Multimodal models, Custom models
- AI infrastructure management: Inference container, Routing/caching, Load balancing, API gateway, Security and resilience
Cisco Secure AI Factory stack with Mistral.ai
AI/GenAI Pipeline:
- Mistral.ai: Frontier Models & Enterprise Platform, Tools, Agents, OCR
- NVIDIA: AI Enterprise, NeMo, NIMS, Blueprints
- Compute: Cisco UCS® w/ NVIDIA Accelerated Computing, NVIDIA BlueField®-3 DPUs, Managed w/ Cisco Intersight®
- Networking: Cisco Nexus 9300 w/ Silicon One® Switches w/ NVIDIA Spectrum-X*, Managed w/ Cisco Nexus Dashboard
- Storage: NetApp | Pure Storage | VAST Data
- Security: Cisco AI Defense, Cisco Hybrid Mesh Firewall, Isovalent, Secure Firewall, Hypershield
Outputs: Knowledgebase Copilots, Content & Code Generation, Virtual Agent & Chatbots, Detection & Predictions.
Wrap-Up
- Cisco + NVIDIA: AI Networking Fabric
- Cisco + NVIDIA: Cisco Secure AI Factory
- Cisco + Mistral.ai: Enterprise Grade GenAI Platform
Toute l'équipe Datacenter France est à votre disposition sur notre stand.
