What is Azure AI Foundry?

Azure AI Foundry is Microsoft’s unified enterprise-AI platform, designed to let organisations build, manage and deploy AI applications and agents at scale. It combines a catalog of models, tools for evaluation, monitoring, governance, and an SDK/REST interface. 

Key features include:

• A single Azure resource (kind = AIServices) to manage agents, models, and deployments. 
• Unified SDK client libraries for languages like Python, C#, JavaScript. 
• Enterprise-grade controls: role-based access (RBAC), network isolation, observability. 

In a hybrid workflow, Azure AI Foundry acts as the “cloud side” – where you deploy your AI model end-points for scalable production, while still enabling local or edge invocation when desired.

What is Foundry Local?

Foundry Local is the on-device or edge variant of the Foundry ecosystem: it allows you to run AI models locally on your device (e.g., desktop, edge appliance) without needing to route all inference to the cloud. 

Notable characteristics:

• Works offline (once model is downloaded) and offers inference without cloud dependency. 
• Integrates via REST/SDK and CLI, making it accessible for local use cases. 
• Designed for privacy-sensitive or latency-critical applications: keep data on-device, reduce round-trip delay.

Thus, Foundry Local becomes the “edge” component in the hybrid workflow and can work seamlessly with the cloud counterpart.

Why a Hybrid Workflow Matters

A hybrid AI workflow — where both local (edge) and cloud inference are available — offers strategic advantages:

• Latency & performance: Local inference improves responsiveness when network or cloud latency is unacceptable.
• Data sovereignty / privacy: Sensitive data can be processed locally without leaving device or premise.
• Cost control: Use local hardware for high-volume inference; fall back to cloud when needed.
• Scalability & fallback: Use cloud for peak load or large models; local for baseline or offline mode.
• Resilience: When connectivity is poor or offline, the local component keeps functioning.

Together, using Foundry Local and Azure AI Foundry cloud allows you to tailor where inference happens — based on priority of latency, cost, privacy and scale.

Architecture of a Hybrid Workflow

Here’s a typical architecture pattern:

1. Device/Edge: Foundry Local installed, local model loaded. App invokes local endpoint via REST or SDK.
2. Cloud: Azure AI Foundry resource provides REST/SDK endpoint for cloud model.
3. Routing layer: Within your application, logic chooses local vs cloud (feature flag, latency check, fallback).
4. Data flow: User sends request → local service: if model available and meets constraints → use local; else route to cloud endpoint → response returned.
5. Telemetry & monitoring: Logs from both local and cloud inference feed into central monitoring/observability platform (e.g., Azure Monitor) for unified insight.
6. Governance & security: Azure AI Foundry provides enterprise controls; local component must abide by encryption, model versioning and update controls.
7. Model catalogue & lifecycle: Cloud and local may share model versions; updated models may be deployed to cloud first, then pushed/packaged for local devices.

Microsoft documentation verifies this device-to-cloud workflow: Foundry Local is described as “on-device AI inference … and scale to Azure AI Foundry as your needs grow.” 

Getting Started: Setup Cloud and Local Environments

Cloud Setup – Azure AI Foundry

1. Ensure you have an Azure subscription.
2. Create an Azure AI Foundry resource: see “Create an AI Foundry resource” guide. 
3. After creation, navigate to your Foundry project, obtain the endpoint and credentials. You’ll use these in your SDK/REST calls. 
4. (Optional) Configure networking, RBAC, private link as needed (see “What’s new in Azure AI Foundry” for network security perimeter preview). 

az group create --name ai-services-rg --location westus2
az cognitiveservices account create \
  --name my-foundry-account \
  --resource-group ai-services-rg \
  --kind AIServices \
  --sku S0 \
  --location westus2

Local Setup – Foundry Local

1. Ensure your system meets hardware/OS prerequisites: e.g. Windows 10/11 x64, 8 GB+ RAM, 3 GB disk, optional GPU/NPU support. 
2. Install Foundry Local:
   • Windows: winget install Microsoft.FoundryLocal
   • macOS: brew tap microsoft/foundrylocal && brew install foundrylocal 
3. Download and run a model locally:
   • foundry model run qwen2.5-0.5b
4. Confirm the REST/SDK endpoint is running and accessible from your application logic.
   With both environments ready, you can now build your hybrid workflow.

Implementing Local Inference with Foundry Local

Here’s a quick example in Python of calling the REST endpoint from Foundry Local. Assume you have a local service running.

import requests

endpoint = "http://127.0.0.1:5000/v1/chat/completions"
model_name = "qwen2.5-0.5b"

payload = {
    "model": model_name,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the weather today?"}
    ],
    "max_tokens": 100,
    "temperature": 0.7
}

response = requests.post(endpoint, json=payload)
print(response.json()["choices"][0]["message"]["content"])

This aligns with the CLI/REST interface described for Foundry Local. 

You can also integrate via the Foundry Local SDK or use OpenAI-compatible libraries pointing to the local endpoint.

Implementing Cloud Inference with Azure AI Foundry

Using the Azure AI Foundry SDK for Python:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

endpoint = "https://<your-foundry-endpoint>"
credential = DefaultAzureCredential()
project_client = AIProjectClient(endpoint=endpoint, credential=credential)

# Use the models client
chat_client = project_client.get_azure_openai_client(api_version="2024-12-01-preview")

response = chat_client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {"role": "system", "content": "You are an assistant."},
        {"role": "user", "content": "How do I build a hybrid AI workflow?"}
    ],
    max_tokens=150,
    temperature=0.7
)
print(response.choices[0].message.content)

This uses the official SDK as described in Microsoft documentation. 

You may also choose to call the REST API directly using the /v1/chat/completions endpoint defined in the API spec. 

Switching Between Local & Cloud in App Code

In a hybrid architecture you’ll likely implement logic such as:

def get_response(user_input):
    if local_model_available() and meets_latency_privacy():
        return call_local_endpoint(user_input)
    else:
        return call_cloud_endpoint(user_input)

Key considerations:

• Feature detection: Check if the local model is loaded/healthy.
• Routing logic: Prioritise local but fallback to cloud when local resources exhausted.
• Unified API contract: Use same message format for both local and cloud for ease of development.
• Telemetry: Tag responses with source (local vs cloud) for monitoring.
• Versioning: Ensure model versions align; if cloud model is updated, consider pushing artifact to local.

By abstracting the endpoint logic, you enable seamless switching between on-device and cloud without changing business logic.

Best Practices for Hybrid Deployment

• Governance & security: Use the enterprise governance features of Azure AI Foundry (RBAC, network isolation, data encryption) and ensure local devices follow compliance standards. 
• Observability: Collect logs from both local and cloud; integrate into Azure Monitor or other SIEM for unified view.
• Cost management: Use cost planning guidance for Azure AI Foundry (see quotas & limits) and consider hardware TCO for local component. 
• Model lifecycle: Version-control models and ensure local deployments are updated in sync with cloud when needed.
• Fallback logic: Ensure cloud endpoint acts as fallback when local fails (resource pressure, hardware issue).
• Offline readiness: In scenarios where network is unavailable, ensure local models can serve trimmed use-cases gracefully.
• Edge device classification: For local inference, ensure hardware meets requirements (e.g., GPU/NPU) per Foundry Local guide. 
• Responsible AI & trust: Use the “Trustworthy AI for Azure AI Foundry” guidance to align with ethics and safety. 

Example: Build a Chat App That Starts Local, Falls Back to Cloud

Step 1: Local Setup

• Install Foundry Local and download qwen2.5-0.5b model. 
• Launch local service.

Step 2: Cloud Setup

• Create Azure AI Foundry resource and set up SDK access. 

Step 3: Build Application Code (Python simplified)

import requests
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Cloud client
cloud_endpoint = "https://<your-foundry-endpoint>"
cred = DefaultAzureCredential()
cloud_client = AIProjectClient(endpoint=cloud_endpoint, credential=cred).get_azure_openai_client(api_version="2024-12-01-preview")

def call_cloud(user_msg):
    resp = cloud_client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role":"user","content":user_msg}],
        max_tokens=150
    )
    return resp.choices[0].message.content

def call_local(user_msg):
    local_endpoint = "http://127.0.0.1:5000/v1/chat/completions"
    payload = {
        "model": "qwen2.5-0.5b",
        "messages": [{"role":"user","content":user_msg}],
        "max_tokens":150
    }
    resp = requests.post(local_endpoint, json=payload)
    return resp.json()["choices"][0]["message"]["content"]

def get_response(user_msg):
    try:
        # Check if local service running
        return call_local(user_msg)
    except Exception as e:
        print("Local failed, falling back to cloud:", e)
        return call_cloud(user_msg)

if __name__=="__main__":
    user_input = input("You: ")
    bot_response = get_response(user_input)
    print("Bot:", bot_response)

Step 4: Deploy & Monitor

• When used on a device with no internet or heavy latency, the local endpoint serves responses.
• When local is not available or complex reasoning required, fallback to cloud model.
• Monitor usage: tag responses with source=local or source=cloud, log latency and error rates.
• Update model versions: when cloud model updated, schedule pushing new version to local device.

Monitoring, Logging & Observability Across Local + Cloud

• Use Azure Monitor / Log Analytics for your cloud endpoint (within Azure AI Foundry).
• For local service: set up local logging (file or local telemetry pipeline) and send summary metrics (e.g., via Azure Application Insights if permitted).
• Key metrics to track: inference latency, error count, source (local/cloud), model version used, fallback rate (how often local failed and cloud used).
• For governance: use the “Trustworthy AI for Azure AI Foundry” guidance for audit-trailed agent behavior. 

Cost, Quotas and Scalability Considerations

• Cloud: Azure AI Foundry uses quotas/limits (e.g., tokens per minute, requests per minute). 
• Reservation model: For cloud deployments you can purchase Provisioned Throughput Reservations for cost savings. 
• Local: Hardware cost amortisation, power/maintenance, and model size/hardware compatibility must be factored.
• Hybrid strategy: Use local for baseline or latency-critical tasks; offload heavier tasks to cloud to optimise cost and scale.

Use Cases Ideal for Hybrid Approach

• Field-service devices or kiosks with intermittent connectivity.
• On-premises enterprise applications with stringent data privacy (e.g., healthcare, legal).
• Real-time applications (gaming, AR/VR) where latency matters.
• High-volume inference workloads where local hardware amortises better than high cloud token cost.
• Hybrid product suites: local inference for base logic, cloud for heavy reasoning/fallback.

Limitations & Things to Watch

• Foundry Local is currently in Preview; features may evolve. 
• Hardware constraints: not all models may run on your device; memory/VRAM limits apply.
• Model catalogue and update cadence may lag cloud availability.
• You must build the routing logic and telemetry system yourself to manage hybrid.
• Network, security and update management for local devices is your responsibility.
• Cloud quotas may still apply, and local fallback can increase complexity of governance.

Conclusion & Next Steps

In summary: by leveraging both Foundry Local and Azure AI Foundry, you can create a hybrid AI workflow that combines the best of edge/on-device inference (latency, privacy) with cloud scalability and model freshness. You can start small locally, build your application logic and routing, then scale to the cloud when needed — all while using consistent APIs, SDKs and governance controls.

Next steps for you:

1. Set up your Azure AI Foundry resource and deploy a cloud model.
2. Install Foundry Local on a device and run your first model.
3. Build the routing logic in your application to switch between local/cloud.
4. Add telemetry, monitoring and fallback logic.
5. Evaluate cost and latency trade-offs and iterate.

When you’re ready, you can dig deeper into advanced topics: multi-agent orchestration, RAG with vector search, edge clusters, offline fallback modes, and model update pipelines.

Frequently Asked Questions (FAQs)

Q1: Can I use the same model version in Foundry Local and Azure AI Foundry so behaviour is identical?

A1: Yes — ideally you deploy the same model version (or close variant) in both local and cloud so application behaviour is consistent. Foundry Local is designed to “scale to Azure AI Foundry as your needs grow.” 

Q2: Do I need an Azure subscription to run Foundry Local?

A2: No, you do not. Foundry Local runs on your device without requiring an Azure subscription. 

Q3: How do I secure my hybrid workflow and protect sensitive data?

A3: Use Azure AI Foundry’s enterprise governance (RBAC, network controls) and ensure local devices adhere to encryption, update and security policies. See the “Trustworthy AI” guidance. 

Q4: What happens if the local model fails or runs out of memory?

A4: That’s why fallback to the cloud is key. The hybrid routing logic should detect local failure conditions and route to cloud endpoint.

Q5: Can I monitor both local and cloud inference uniformly?

A5: Yes – you should instrument telemetry in both and integrate into a unified analytics platform (e.g., Azure Monitor, Log Analytics). Logs should include source (local/cloud), latency, errors and model version.

Q6: How does cost compare between local and cloud?

A6: Local costs include hardware, power, maintenance. Cloud costs include token/inference billing and quotas. Hybrid allows you to optimise by using local for baseline and cloud for peak/heavy tasks. Cloud quotas are documented. 

Happy Sharing…