In this blog, we’ll walk through the process of creating your first AI agent using Azure AI Agent Service. We’ll focus on creating a basic agent setup that uses Managed Identity for authentication. This blog will help you get started quickly, so you can focus on writing code and seeing the results.

Blog Summary:

In this blog, we will explore overview of Azure AI Agent Service, what is AI Agent and the process of creating AI agents using Azure AI Agent Service.

The blog will provide a demo overview, guiding you through the steps to create an AI agent using Managed Identity for authentication. We will also include a detailed code walkthrough to help you understand the process of creating and running an AI agent.

By the end of this blog, you will have a comprehensive understanding of how to leverage Azure AI Agent Service to build powerful AI-driven applications with minimal code.

Azure AI Agent Service:

Azure AI Agent Service simplifies the process of building, deploying, and scaling AI agents. It abstracts away much of the complexity, allowing you to create powerful AI-driven applications with minimal code. With features like managed identity, robust security, and seamless integration with other Azure services, it’s an excellent choice for developers looking to leverage AI in their applications.

What is Aure AI Agent Service?

Azure AI Agent Service is a fully managed service designed to empower developers to securely build, deploy, and scale high-quality, and extensible AI agents without needing to manage the underlying compute and storage resources. What originally took hundreds of lines of code to support client side function calling can now be done in just a few lines of code with Azure AI Agent Service.

Want to explore on What is Azure AI Agent Service: What is Azure AI Agent Service? – Azure AI services | Microsoft Learn

Want to see the announcement from Microsoft: Azure AI Agent Service: Revolutionizing AI Agent Development and Deployment

What is an AI Agent?

An AI agent is a software program that can autonomously perform tasks on behalf of a user or another system. It interacts with its environment, collects data, and uses that data to make decisions and take actions to achieve specific goals. For example, an AI agent could be used in a customer service application to answer queries, process requests, and even perform transactions without human intervention.

Why Use Azure AI Agent Service?

Azure AI Agent Service is a fully managed service that simplifies the process of building, deploying, and scaling AI agents. Here are some key benefits:

• Ease of Use: It reduces the amount of code needed to create and manage AI agents, making development faster and more efficient.
• Scalability: The service handles the underlying infrastructure, so you don’t have to worry about scaling your agents as demand grows.
• Security: Azure provides robust security features to protect your data and ensure compliance with industry standards.
• Integration: It seamlessly integrates with other Azure services and tools, allowing you to build comprehensive solutions.

Comparing Azure Agents and Azure OpenAI Assistants

Both Azure AI Agents and Azure OpenAI Assistants enable you to build intelligent applications, but they have some differences:

Models supported by Azure AI Agent Service:

The Azure AI Agent Service also supports the following models from the Azure AI Foundry model catalog.

• Azure OpenAI (gpt-4o, gpt-4o-mini, gpt-35-turbo)
• Llama 3.1-70B-instruct
• Mistral-large-2407
• Cohere command R+

Demo:

We will be using Quickstart from Microsoft Learn to create a new agent and guide you through upcoming blogs on how to proceed further with learnings. Quickstart – Create a new Azure AI Agent Service project – Azure AI services | Microsoft Learn

Demo Overview:

We will help you create your first AI agent using Azure AI Agent Service. We’ll focus on deploying a basic agent setup that uses Managed Identity for authentication.

This demo will show us the creation and execution of an AI agent that processes a request to generate a bar chart.

Prerequisites:

• An Azure subscription – Create one for free.
• Python 3.8 or later
• Azure AI Developer RBAC role 
• Install the Azure CLI and the machine learning extension
• Access to Azure AI Foundry https://ai.azure.com/

RBAC roles configuration:

Create a customRole.json file:

{
  "Name": "CustomMLAgentRole",
  "IsCustom": true,
  "Description": "Custom role with permissions to manage agents in Azure ML workspace",
  "Actions": [
    "Microsoft.MachineLearningServices/workspaces/read",
    "Microsoft.MachineLearningServices/workspaces/write",
    "Microsoft.MachineLearningServices/workspaces/listKeys/action",
    "Microsoft.MachineLearningServices/workspaces/computes/read",
    "Microsoft.MachineLearningServices/workspaces/computes/write",
    "Microsoft.MachineLearningServices/workspaces/computes/delete"
  ],
  "NotActions": [],
  "AssignableScopes": [
    "/subscriptions/<enter your subscription here>/resourceGroups/rgBasicAgent/providers/Microsoft.MachineLearningServices/workspaces/ai-basic-project-tqgb"
  ]
}

Install ML Extension:

Refer this guide to install the ML extension and verify. Install and set up the CLI (v2)

Lab (step by step)

Set up your Azure AI Hub and Agent project

Creating an Azure AI Hub to set up your app environment and Azure resources.

Go to Azure AI Foundry: https://ai.azure.com/

Click on Create Project.

Creating an Azure AI project under your Hub creates an endpoint for your app to call, and sets up app services to access resources in your tenant.

Choose Basic or Standard Agent Setup

Basic Setup: Agents use multitenant search and storage resources fully managed by Microsoft. You don’t have visibility or control over these underlying Azure resources.

Standard Setup: Agents use customer-owned, single-tenant search and storage resources. With this setup, you have full control and visibility over these resources, but you incur costs based on your usage.

We will choose here, Basic agent set up and deploy this basic agent set up.

Click on Deploy to azure (Basic Template).

Project structure looks like below.

To understand more about this project structure, refer: Getting started with Azure AI Studio

Click on Deploy to azure (Basic Template).

You can modify the names of Instance details used in this template, review and create.

Check the model availability from here: Supported models in Azure AI Agent Service – Azure AI services | Microsoft Learn

These are the important parameters on Model selection! 

You will see below the services created.

Configure and run an agent:

Below is the breakdown of the components involved in configuring and running an Azure AI Agent, along with their descriptions and usage at different stages of the program flow:

Configure Azure AI Foundry SDK in VS Code:

Open visual Studio code and new terminal and run below commands.

pip install azure-ai-projects
pip install azure-identity

azure-ai-projects and azure-identity packages are part of the Azure AI Foundry SDK. The Azure AI Foundry SDK is a comprehensive toolchain designed to simplify the development of AI applications on Azure. It enables developers to access popular models, combine data and AI services, and build AI-powered applications efficiently.

To know more: How to get started with Azure AI Foundry SDK – Azure AI Foundry | Microsoft Learn

Get the Discovery Url and connection string:

Or using the command below, get the discovery Url and update your connectionstring.

az ml workspace show -n {project_name} –resource-group {resource_group_name} –query discovery_url

Update the code

import os
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import CodeInterpreterTool
from azure.identity import DefaultAzureCredential
from typing import Any
from pathlib import Path

# Create an Azure AI Client from a connection string, copied from your Azure AI Foundry project.
# At the moment, it should be in the format "<HostName>;<AzureSubscriptionId>;<ResourceGroup>;<ProjectName>"
# HostName can be found by navigating to your discovery_url and removing the leading "https://" and trailing "/discovery"
# To find your discovery_url, run the CLI command: az ml workspace show -n {project_name} --resource-group {resource_group_name} --query discovery_url
# Project Connection example: eastus.api.azureml.ms;12345678-abcd-1234-9fc6-62780b3d3e05;my-resource-group;my-project-name
# Customer needs to login to Azure subscription via Azure CLI and set the environment variables

# Set environment variable
os.environ['PROJECT_CONNECTION_STRING'] = 'eastus.api.azureml.ms;<update your subscription here>;rgBasicAgent;ai-basic-project-tqgb'

project_client = AIProjectClient.from_connection_string(
    credential=DefaultAzureCredential(), conn_str=os.environ["PROJECT_CONNECTION_STRING"]
)

# Ensure the environment variable is set
connection_string = os.getenv("PROJECT_CONNECTION_STRING")
if connection_string is None:
    raise ValueError("The environment variable PROJECT_CONNECTION_STRING is not set.")

with project_client:
    # Create an instance of the CodeInterpreterTool
    code_interpreter = CodeInterpreterTool()

    # The CodeInterpreterTool needs to be included in creation of the agent
    agent = project_client.agents.create_agent(
        model="gpt-4o-mini",
        name="my-agent",
        instructions="You are helpful agent",
        tools=code_interpreter.definitions,
        tool_resources=code_interpreter.resources,
    )
    print(f"Created agent, agent ID: {agent.id}")

    # Create a thread
    thread = project_client.agents.create_thread()
    print(f"Created thread, thread ID: {thread.id}")

    # Create a message
    message = project_client.agents.create_message(
        thread_id=thread.id,
        role="user",
        content="Could you please create a bar chart for the operating profit using the following data and provide the file to me? Company A: $1.2 million, Company B: $2.5 million, Company C: $3.0 million, Company D: $1.8 million",
    )
    print(f"Created message, message ID: {message.id}")

    # Run the agent
    run = project_client.agents.create_and_process_run(thread_id=thread.id, assistant_id=agent.id)
    print(f"Run finished with status: {run.status}")

    if run.status == "failed":
        # Check if you got "Rate limit is exceeded.", then you want to get more quota
        print(f"Run failed: {run.last_error}")

    # Get messages from the thread
    messages = project_client.agents.list_messages(thread_id=thread.id)
    print(f"Messages: {messages}")

    # Get the last message from the sender
    last_msg = messages.get_last_text_message_by_sender("assistant")
    if last_msg:
        print(f"Last Message: {last_msg.text.value}")

    # Generate an image file for the bar chart
    for image_content in messages.image_contents:
        print(f"Image File ID: {image_content.image_file.file_id}")
        file_name = f"{image_content.image_file.file_id}_image_file.png"
        project_client.agents.save_file(file_id=image_content.image_file.file_id, file_name=file_name)
        print(f"Saved image file to: {Path.cwd() / file_name}")

    # Print the file path(s) from the messages
    for file_path_annotation in messages.file_path_annotations:
        print(f"File Paths:")
        print(f"Type: {file_path_annotation.type}")
        print(f"Text: {file_path_annotation.text}")
        print(f"File ID: {file_path_annotation.file_path.file_id}")
        print(f"Start Index: {file_path_annotation.start_index}")
        print(f"End Index: {file_path_annotation.end_index}")
        project_client.agents.save_file(file_id=file_path_annotation.file_path.file_id, file_name=Path(file_path_annotation.text).name)

    # Delete the agent once done
    #project_client.agents.delete_agent(agent.id)
    print("Deleted agent")

Code walkthrough:

Explanation of the Code and Process Flow

This code demonstrates how to create and run an AI agent using Azure AI Agent Service. Here’s a step-by-step explanation of the process flow:

1. Import Necessary Libraries:
   • The code imports essential libraries, including os, AIProjectClient, CodeInterpreterTool, DefaultAzureCredential, Any, and Path.
2. Set Environment Variable:
   • The environment variable PROJECT_CONNECTION_STRING is set to store the connection string for the Azure AI Foundry project.
3. Create Azure AI Client:
   • An instance of AIProjectClient is created using the connection string and DefaultAzureCredential for authentication.
   • DefaultAzureCredential simplifies authentication by allowing apps to use different methods based on their environment. This enables seamless promotion from local development to production without code changes. It automatically detects and uses the appropriate authentication method, eliminating the need for manual configuration or conditional logic. This approach is preferred for its efficiency and ease of use. Refer for more details: How to authenticate .NET applications with Azure services – .NET | Microsoft Learn
4. Check Environment Variable:
   • The code checks if the PROJECT_CONNECTION_STRING environment variable is set. If not, it raises a ValueError.
5. Create Code Interpreter Tool:
   • An instance of CodeInterpreterTool is created. This tool will be used by the agent to interpret and execute code.
6. Create Agent:
   • An AI agent is created using the create_agent method. The agent is configured with a model, name, instructions, and tools (including the CodeInterpreterTool).
7. Create Thread:
   • A conversation thread is created using the create_thread method. This thread will store all messages exchanged during the session.
8. Create Message:
   • A message is created and added to the thread. The message contains a request to create a bar chart for the operating profit of several companies.
9. Run the Agent:
   • The agent is run using the create_and_process_run method. The agent processes the thread’s messages and performs the requested tasks.
10. Check Run Status:
    • The code checks the status of the run. If the run failed, it prints the error message.
11. Get Messages from Thread:
    • The messages from the thread are retrieved using the list_messages method. The messages are printed to the console.
12. Get Last Message from Assistant:
    • The last message from the assistant is retrieved and printed to the console.
13. Generate Image File for Bar Chart:
    • The code iterates through the image contents in the messages and saves the image files to the current working directory.
14. Print File Paths from Messages:
    • The code iterates through the file path annotations in the messages and prints the file paths to the console. The files are saved to the specified paths.
15. Delete Agent (Optional):
    • The agent is deleted using the delete_agent method (commented out in the code).

Run the program:

Run the program: python BasicAgent.py

Output

Lets see below screenshot on the output generated by this code.

This output shows that

• the AI agent successfully processed the user’s request
• generated a bar chart,
• and provided a link to download the chart.

Behind the scenes:

Let’s understand what is happening in the background.

The output indicates the successful creation and execution of an AI agent that processes a request to generate a bar chart.

Here’s a step-by-step explanation of what happened:

1. Agent Creation:
   • An AI agent was created with the ID asst_wPnumV9GurkbxHvHJCnnkvAc.
2. Thread Creation:
   • A thread was created with the ID thread_OEUUPZNp4HjjLq9GEihgnhZp.
3. Message Creation:
   • A message was created with the ID msg_T0MTP1WHLrpJW5ySVdK7qpqJ.
4. Run Status:
   • The run finished with the status RunStatus.COMPLETED.
5. Messages:
   • The agent processed a user request to create a bar chart for operating profits.
   • The user message (msg_T0MTP1WHLrpJW5ySVdK7qpqJ) requested the creation of a bar chart with specific data.
   • The agent responded with a message (msg_3m8rfG8cWxLgFx3oVAXIDkyq) indicating that the bar chart was created and provided a download link.
6. Last Message:
   • The last message from the agent stated: “I have created the bar chart for the operating profits. You can download it using the link below: Download Operating Profit Chart”.
7. File Paths:
   • The output includes details about the file path annotation:
     • Type: file_path
     • Text: sandbox:/mnt/data/operating_profit_chart.png
     • File ID: assistant-I2CW55fgnZ5gHQDD0pQ4aBLY
     • Start Index: 133
     • End Index: 177
8. Agent Deletion:
   • The agent was deleted after completing the task.

AI Agent’s Role

• Agent Creation: The AI agent is created with specific instructions and tools, including the CodeInterpreterTool. This setup defines the agent’s capabilities and how it will interact with the user.
• Message Processing: When the agent is run, it processes the messages in the thread. In this case, the agent interprets the user’s request to create a bar chart for the operating profit of several companies.
• Task Execution: The agent uses its tools (e.g., CodeInterpreterTool) to perform the requested task. It generates the bar chart based on the provided data and appends the result to the thread.
• Response Generation: The agent generates responses, including text messages and image files, which are added to the thread. These responses are then retrieved and displayed by the program.
• Run Steps: The agent’s actions during the run are logged as run steps. These steps provide a detailed record of how the agent processed the messages and performed the tasks.

Conclusion:

By following these steps, you’ve successfully created and deployed your first AI agent using Azure AI Agent Service. This setup allows you to leverage the power of AI with minimal code, enabling you to focus on building innovative solutions.

Stay tuned for more advanced topics and use cases in future posts!

What’s next:

If you are looking for more reading on this topic, refer below.

Azure AI Agents Sample Code: This GitHub repository provides sample code for the Azure AI Agent service. It includes a Python application with a Flask frontend to demonstrate the Azure AI Agent service. The application retrieves weather information using the OpenAPI 3.x specification and integrates with the Azure AI Foundry SDK.

Repository: Azure AI Agents

Azure AI Samples: This repository contains a variety of Azure AI examples, including end-to-end samples and smaller code snippets for common developer tasks. It includes scenarios for building multi-agent applications, integrating with Azure services, and more.

Repository: Azure AI Samples

References:

https://learn.microsoft.com/en-us/azure/ai-services/agents/quickstart?pivots=programming-language-csharp

GitHub – guygregory/Azure-AI-Agents: Sample code for the Azure AI Agent …

Azure AI Samples – GitHub