Module 08: Hands on With Microsoft Foundry Local - Complete Developer Toolkit

Overview

Microsoft Foundry Local represents the next generation of edge AI development, providing developers with powerful tools to build, deploy, and scale AI applications locally while maintaining seamless integration with Azure AI Foundry. This module provides comprehensive coverage of Foundry Local from installation to advanced agent development.

Microsoft Foundry Local 代表着边缘 AI 开发的下一代形态：它为开发者提供了在本地构建、部署和扩展 AI 应用的强大工具，同时与 Azure AI Foundry 保持无缝集成。本模块全面覆盖 Foundry Local，从安装到进阶 agent 开发。

Key Technologies:

Microsoft Foundry Local CLI and SDK
Azure AI Foundry integration
On-device model inference
Local model caching and optimization
Agent-based architectures

关键技术：Microsoft Foundry Local CLI 与 SDK；Azure AI Foundry 集成；端侧模型推理；本地模型缓存与优化；基于 agent 的架构。

Learning Objectives

By completing this module, you will:

完成本模块后，你将能够：

Master Foundry Local: Install, configure, and optimize for Windows 11 development
Deploy Diverse Models: Run phi, qwen, deepseek, and GPT models locally with CLI commands
Build Production Solutions: Create AI applications with advanced prompt engineering and data integration
Leverage Open-Source Ecosystem: Integrate Hugging Face models and community contributions
Develop AI Agents: Build intelligent agents with grounding and orchestration capabilities
Implement Enterprise Patterns: Create modular, scalable AI solutions for production deployment

掌握 Foundry Local：为 Windows 11 开发完成安装、配置与优化；部署多种模型：用 CLI 命令在本地运行 phi、qwen、deepseek 和 GPT 模型；构建生产方案：用进阶提示词工程与数据集成打造 AI 应用；利用开源生态：集成 Hugging Face 模型与社区贡献；开发 AI agent：构建带 grounding 与编排能力的智能 agent；落地企业模式：打造模块化、可扩展、可上生产的 AI 方案。

Session Structure

1: Getting Started with Foundry Local

Focus: Installation, CLI setup, model deployment, and hardware optimization

Key Topics: Complete installation • CLI commands • Model caching • Hardware acceleration • Multi-model deployment

关键主题：完整安装 · CLI 命令 · 模型缓存 · 硬件加速 · 多模型部署

Sample: REST Chat Quickstart • OpenAI SDK Integration • Model Discovery & Benchmarking

示例：REST 聊天快速上手 · OpenAI SDK 集成 · 模型发现与基准测试

Duration: 2-3 hours | Level: Beginner

时长：2–3 小时 | 难度：入门

2: Build AI Solutions with Azure AI Foundry

Focus: Advanced prompt engineering, data integration, and cloud connectivity

Key Topics: Prompt engineering • Data integration • Azure workflows • Performance optimization • Monitoring

关键主题：提示词工程 · 数据集成 · Azure 工作流 · 性能优化 · 监控

Sample: Chainlit RAG Application

示例：Chainlit RAG 应用

Duration: 2-3 hours | Level: Intermediate

时长：2–3 小时 | 难度：中级

3: Open-Source Models Foundry Local

Focus: Hugging Face integration, BYOM strategies, and community models

Key Topics: HuggingFace integration • Bring-your-own-model • Model Mondays insights • Community contributions • Model selection

关键主题：HuggingFace 集成 · 自带模型 · Model Mondays 洞察 · 社区贡献 · 模型选型

Sample: Multi-Agent Orchestration

示例：多 agent 编排

Duration: 2-3 hours | Level: Intermediate

时长：2–3 小时 | 难度：中级

4: Explore Cutting-Edge Models

Focus: LLMs vs SLMs, EdgeAI implementation, and advanced demos

Key Topics: Model comparison • Edge vs cloud inference • Phi + ONNX Runtime • Chainlit RAG app • WebGPU optimization

关键主题：模型对比 · 边缘推理 vs 云端推理 · Phi + ONNX Runtime · Chainlit RAG 应用 · WebGPU 优化

Sample: Models-as-Tools Router

示例：模型即工具的路由器

Duration: 3-4 hours | Level: Advanced

时长：3–4 小时 | 难度：进阶

5: Build AI-Powered Agents Fast

Focus: Agent architectures, system prompts, grounding, and orchestration

Key Topics: Agent design patterns • System prompt engineering • Grounding techniques • Multi-agent systems • Production deployment

关键主题：agent 设计模式 · 系统提示词工程 · grounding 技术 · 多 agent 系统 · 生产部署

Sample: Multi-Agent Orchestration • Advanced Multi-Agent System

示例：多 agent 编排 · 进阶多 agent 系统

Duration: 3-4 hours | Level: Advanced

时长：3–4 小时 | 难度：进阶

6: Foundry Local - Models as Tools

Focus: Modular AI solutions, enterprise scaling, and production patterns

Key Topics: Models as tools • On-device deployment • SDK/API integration • Enterprise architectures • Scaling strategies

关键主题：模型即工具 · 端侧部署 · SDK/API 集成 · 企业架构 · 扩展策略

Sample: Models-as-Tools Router • Foundry Tools Framework

示例：模型即工具的路由器 · Foundry Tools 框架

Duration: 3-4 hours | Level: Expert

时长：3–4 小时 | 难度：专家

7: Direct API Integration Patterns

Focus: Pure REST API integration without SDK dependencies for maximum control

Key Topics: HTTP client implementation • Custom authentication • Model health monitoring • Streaming responses • Production error handling

关键主题：HTTP 客户端实现 · 自定义认证 · 模型健康监控 · 流式响应 · 生产级错误处理

Sample: Direct API Client

示例：直接调用 API 的客户端

Duration: 2-3 hours | Level: Intermediate

时长：2–3 小时 | 难度：中级

8: Windows 11 Native Chat Application

Focus: Building modern native chat applications with Foundry Local integration

Key Topics: Electron development • Fluent Design System • Native Windows integration • Real-time streaming • Chat interface design

关键主题：Electron 开发 · Fluent Design System · Windows 原生集成 · 实时流式输出 · 聊天界面设计

Sample: Windows 11 Chat Application

示例：Windows 11 聊天应用

Duration: 3-4 hours | Level: Advanced

时长：3–4 小时 | 难度：进阶

9: Advanced Multi-Agent Orchestration

Focus: Sophisticated agent coordination, specialized task delegation, and collaborative AI workflows

Key Topics: Intelligent agent coordination • Function calling patterns • Cross-agent communication • Workflow orchestration • Quality assurance mechanisms

关键主题：智能 agent 协调 · 函数调用模式 · 跨 agent 通信 · 工作流编排 · 质量保障机制

Sample: Advanced Multi-Agent System

示例：进阶多 agent 系统

Duration: 4-5 hours | Level: Expert

时长：4–5 小时 | 难度：专家

10: Foundry Local as Tools Framework

Focus: Tool-first architecture for integrating Foundry Local into existing applications and frameworks

Key Topics: LangChain integration • Semantic Kernel functions • REST API frameworks • CLI tools • Jupyter integration • Production deployment patterns

关键主题：LangChain 集成 · Semantic Kernel 函数 · REST API 框架 · CLI 工具 · Jupyter 集成 · 生产部署模式

Sample: Foundry Tools Framework

示例：Foundry Tools 框架

Duration: 4-5 hours | Level: Expert

时长：4–5 小时 | 难度：专家

Prerequisites

System Requirements

Operating System: Windows 11 (22H2 or later)
Memory: 16GB RAM (32GB recommended for larger models)
Storage: 50GB free space for model caching
Hardware: NPU-enabled device recommended (Copilot+ PC), GPU optional
Network: High-speed internet for initial model downloads

Development Environment

Visual Studio Code with AI Toolkit extension
Python 3.10+ and pip
Git for version control
PowerShell or Command Prompt
Azure CLI (optional for cloud integration)

Knowledge Prerequisites

Basic understanding of AI/ML concepts
Command line familiarity
Python programming basics
REST API concepts
Basic knowledge of prompting and model inference

Module Timeline

Total Estimated Time: 30-38 hours

预计总时长：30–38 小时

Session	Focus Area	Samples	Time	Complexity
1	Setup & Basics	01, 02, 03	2-3 hours	Beginner
2	AI Solutions	04	2-3 hours	Intermediate
3	Open Source	05	2-3 hours	Intermediate
4	Advanced Models	06	3-4 hours	Advanced
5	AI Agents	05, 09	3-4 hours	Advanced
6	Enterprise Tools	06, 10	3-4 hours	Expert
7	Direct API Integration	07	2-3 hours	Intermediate
8	Windows 11 Chat App	08	3-4 hours	Advanced
9	Advanced Multi-Agent	09	4-5 hours	Expert
10	Tools Framework	10	4-5 hours	Expert

Key Resources

Official Documentation:

Microsoft Foundry Local GitHub - Source code and official samples
Azure AI Foundry Documentation - Complete setup and usage guide
Model Mondays Series - Weekly model highlights and tutorials

官方文档：Microsoft Foundry Local GitHub——源码与官方示例；Azure AI Foundry 文档——完整的安装与使用指南；Model Mondays 系列——每周模型亮点与教程。

Community & Support:

Foundry Local Discussions - Community Q&A and feature requests
Microsoft AI Developer Community - Latest news and best practices

社区与支持：Foundry Local Discussions——社区问答与功能需求；Microsoft AI 开发者社区——最新动态与最佳实践。

Learning Outcomes

Upon completing this module, you will be equipped to:

完成本模块后，你将具备以下能力：

Technical Mastery

Deploy and Manage: Foundry Local installations across development and production environments
Integrate Models: Seamlessly work with diverse model families from Microsoft, Hugging Face, and community sources
Build Applications: Create production-ready AI applications with advanced features and optimizations
Develop Agents: Implement sophisticated AI agents with grounding, reasoning, and tool integration

Strategic Understanding

Architecture Decisions: Make informed choices between local vs cloud deployment
Performance Optimization: Optimize inference performance across different hardware configurations
Enterprise Scaling: Design applications that scale from local prototypes to enterprise deployments
Privacy and Security: Implement privacy-preserving AI solutions with local inference

Innovation Capabilities

Rapid Prototyping: Quickly build and test AI application concepts across all 10 sample patterns
Community Integration: Leverage open-source models and contribute to the ecosystem
Advanced Patterns: Implement cutting-edge AI patterns including RAG, agents, and tool integration
Framework Mastery: Expert-level integration with LangChain, Semantic Kernel, Chainlit, and Electron
Production Deployment: Deploy scalable AI solutions from local prototypes to enterprise systems
Future-Ready Development: Build applications ready for emerging AI technologies and patterns

Getting Started

Environment Setup: Ensure Windows 11 with recommended hardware (see Prerequisites)
Install Foundry Local: Follow Session 1 for complete installation and configuration
Run Sample 01: Start with basic REST API integration to verify setup
Progress Through Samples: Complete samples 01-10 for comprehensive mastery

1）环境准备：确认 Windows 11 与推荐硬件（见前置条件）；2）安装 Foundry Local：按第 1 节完成安装与配置；3）运行示例 01：从基础 REST API 集成开始，验证环境是否正常；4）逐步推进示例：完成示例 01 到 10，达到全面掌握。

Success Metrics

Track your progress through all 10 comprehensive samples:

用全部 10 个综合示例跟踪你的进度：

Foundation Level (Samples 01-03)

[ ] Successfully install and configure Foundry Local
[ ] Complete REST API integration (Sample 01)
[ ] Implement OpenAI SDK compatibility (Sample 02)
[ ] Perform model discovery and benchmarking (Sample 03)

Application Level (Samples 04-06)

[ ] Deploy and run at least 4 different model families
[ ] Build a functional RAG chat application (Sample 04)
[ ] Create multi-agent orchestration system (Sample 05)
[ ] Implement intelligent model routing (Sample 06)

Advanced Integration Level (Samples 07-10)

[ ] Build production-ready API client (Sample 07)
[ ] Develop Windows 11 native chat application (Sample 08)
[ ] Implement advanced multi-agent system (Sample 09)
[ ] Create comprehensive tools framework (Sample 10)

Mastery Indicators

[ ] Successfully run all 10 samples without errors
[ ] Customize at least 3 samples for specific use cases
[ ] Deploy 2+ samples in production-like environments
[ ] Contribute improvements or extensions to sample code
[ ] Integrate Foundry Local patterns into personal/professional projects

Quick Start Guide - All 10 Samples

Environment Setup (Required for All Samples)

powershell

# 1. Clone and navigate to Module08
cd Module08

# 2. Create Python virtual environment
py -m venv .venv
.\.venv\Scripts\activate

# 3. Install base dependencies
pip install -r requirements.txt

# 4. Install Foundry Local (if not already installed)
winget install Microsoft.FoundryLocal

# 5. Verify Foundry Local installation
foundry --version
foundry model list

Core Foundation Samples (01-06)

Sample 01: REST Chat Quickstart

示例 01：REST 聊天快速上手

powershell

# Start Foundry Local service
foundry model run phi-4-mini

# Run REST chat demo
python samples/01/chat_quickstart.py

Sample 02: OpenAI SDK Integration

示例 02：OpenAI SDK 集成

powershell

# Ensure model is running
foundry status

# Run SDK demo
python samples/02/sdk_quickstart.py

Sample 03: Model Discovery & Benchmarking

示例 03：模型发现与基准测试

powershell

# Run comprehensive model testing
samples/03/list_and_bench.cmd

# Or run individual components
foundry model list --available
foundry model download qwen2.5-0.5b
foundry model benchmark phi-4-mini

Sample 04: Chainlit RAG Application

示例 04：Chainlit RAG 应用

powershell

# Install Chainlit dependencies
pip install chainlit langchain chromadb

# Start RAG chat application
chainlit run samples/04/app.py -w
# Opens browser at http://localhost:8000

Sample 05: Multi-Agent Orchestration

示例 05：多 agent 编排

powershell

# Run agent coordinator demo
python -m samples.05.agents.coordinator

# Run specific agent examples
python samples/05/examples/specialists_demo.py

Sample 06: Models-as-Tools Router

示例 06：模型即工具的路由器

powershell

# Configure environment
set BASE_URL=http://localhost:8000
set GENERAL_MODEL=phi-4-mini
set CODE_MODEL=qwen2.5-7b

# Run intelligent router
python samples/06/router.py "Analyze this Python code for performance issues"

Advanced Integration Samples (07-10)

Sample 07: Direct API Client

示例 07：直接调用 API 的客户端

powershell

# Navigate to sample directory
cd samples/07

# Install additional dependencies
pip install -r requirements.txt

# Run basic API examples
python examples/basic_usage.py

# Try streaming responses
python examples/streaming.py

# Test production patterns
python examples/production.py

Sample 08: Windows 11 Chat Application

示例 08：Windows 11 聊天应用

powershell

# Navigate to sample directory
cd samples/08

# Install Node.js dependencies
npm install

# Start Electron application
npm start

# Or build for production
npm run build

Sample 09: Advanced Multi-Agent System

示例 09：进阶多 agent 系统

powershell

# Navigate to sample directory
cd samples/09

# Install agent system dependencies
pip install -r requirements.txt

# Run basic coordination example
python examples/basic_coordination.py

# Try complex workflow
python examples/complex_workflow.py

# Interactive agent demo
python examples/interactive_demo.py

Sample 10: Foundry Tools Framework

示例 10：Foundry Tools 框架

powershell

# Navigate to sample directory
cd samples/10

# Install framework dependencies
pip install -r requirements.txt

# Run basic tools demo
python examples/basic_tools.py

# Start REST API server
python examples/rest_api_server.py
# API available at http://localhost:8080

# Try CLI application
python examples/cli_application.py --help

# Launch Jupyter notebook
jupyter notebook examples/jupyter_notebook.ipynb

# Test LangChain integration
python examples/langchain_demo.py

Troubleshooting Common Issues

Foundry Local Connection Errors

Foundry Local 连接错误

powershell

# Check service status
foundry status

# Restart if needed
foundry restart

# Verify endpoint accessibility
curl http://localhost:5273/v1/models

Model Loading Issues

模型加载问题

powershell

# Check available models
foundry model list --cached

# Download missing models
foundry model download phi-4-mini
foundry model download qwen2.5-0.5b

# Force reload if needed
foundry model unload --all
foundry model run phi-4-mini

Dependency Issues

依赖问题

powershell

# Upgrade pip and reinstall
python -m pip install --upgrade pip
pip install -r requirements.txt --force-reinstall

# For Node.js samples
npm cache clean --force
npm install

Summary

This module represents the cutting edge of edge AI development, combining Microsoft's enterprise-grade tools with the flexibility and innovation of the open-source ecosystem. By mastering Foundry Local through all 10 comprehensive samples, you'll be positioned at the forefront of AI application development.

本模块代表着边缘 AI 开发的最前沿：把微软企业级工具与开源生态的灵活性和创新结合起来。通过全部 10 个综合示例掌握 Foundry Local，你就能站到 AI 应用开发的最前线。

Complete Learning Path:

Foundation (Samples 01-03): API integration and model management
Applications (Samples 04-06): RAG, agents, and intelligent routing
Advanced (Samples 07-10): Production frameworks and enterprise integration

完整学习路径：基础（示例 01–03）——API 集成与模型管理；应用（示例 04–06）——RAG、agent 与智能路由；进阶（示例 07–10）——生产框架与企业集成。

For Azure OpenAI integration (Session 2), see the individual sample README files for required environment variables and API version settings.

关于 Azure OpenAI 集成（第 2 节），所需的环境变量与 API 版本设置请参见各个示例自己的 README。

Module 08 Samples: Foundry Local Development Guide

04

Workshop Notebooks

02.Workflow-MultiAgent

Podcast Application

🎙️ AI 播客工作室工作坊

Module 08: Hands on With Microsoft Foundry Local - Complete Developer Toolkit ​

Overview ​

Learning Objectives ​

Session Structure ​

Prerequisites ​

System Requirements ​

Development Environment ​

Knowledge Prerequisites ​

Module Timeline ​

Key Resources ​

Learning Outcomes ​

Technical Mastery ​

Strategic Understanding ​

Innovation Capabilities ​

Getting Started ​

Success Metrics ​

Foundation Level (Samples 01-03) ​

Application Level (Samples 04-06) ​

Advanced Integration Level (Samples 07-10) ​

Mastery Indicators ​

Quick Start Guide - All 10 Samples ​

Environment Setup (Required for All Samples) ​

Core Foundation Samples (01-06) ​

Advanced Integration Samples (07-10) ​

Troubleshooting Common Issues ​

Summary ​

Module 08: Hands on With Microsoft Foundry Local - Complete Developer Toolkit

Overview

Learning Objectives

Session Structure

Prerequisites

System Requirements

Development Environment

Knowledge Prerequisites

Module Timeline

Key Resources

Learning Outcomes

Technical Mastery

Strategic Understanding

Innovation Capabilities

Getting Started

Success Metrics

Foundation Level (Samples 01-03)

Application Level (Samples 04-06)

Advanced Integration Level (Samples 07-10)

Mastery Indicators

Quick Start Guide - All 10 Samples

Environment Setup (Required for All Samples)

Core Foundation Samples (01-06)

Advanced Integration Samples (07-10)

Troubleshooting Common Issues

Summary