Extracting Various Visual Features from Images with Azure AI Vision Image Analysis Service.
The Azure AI Vision Image Analysis service is a powerful set of tools for extracting various visual features from images. This service allows users to analyze and classify images using state-of-the-art computer vision algorithms.
In this post, we will explore how to extract different visual features from images using the Azure Image Analysis SDK. The SDK provides a rich and user-friendly interface for performing image analysis tasks.
Overview:
The Azure AI Vision Image Analysis service provides a powerful set of tools for extracting various visual features from images. For example, it can determine whether an image contains adult content, find specific brands or objects, or find human faces.
The latest version of Image Analysis, 4.0, which is now in general availability, has new features like synchronous OCR and people detection.
Use Cases:
Here are some use cases where you might want to leverage this service:
Content Moderation
You can use Image Analysis to automatically detect and filter out adult content, ensuring that your platform or application remains safe and appropriate for all users.
Brand Recognition
If you’re building an application that needs to identify specific brands or logos within images (for marketing, analytics, or other purposes), Image Analysis can help you achieve this.
Object Detection
When you need to locate and identify specific objects within images (such as finding cars, animals, or other items), Image Analysis can be a valuable tool.
People Detection
The latest version of Image Analysis (4.0) includes people detection capabilities. This can be useful for scenarios like counting the number of people in a crowd or analyzing social distancing compliance.
Image Captioning
If you want to generate descriptive captions for images, consider using Image Analysis. Note that image captioning is available in version 3.2 and is supported in specific Azure regions.
Custom Models
With version 4.0 (preview), you can create and train custom models for image classification or object detection. This allows you to tailor the service to your specific use case1.
Remember to choose the appropriate version of Image Analysis based on your requirements. Version 4.0 offers better models and additional features, while version 3.2 provides a wider range of capabilities. If your use case is not yet supported by 4.0, consider using 3.2
Image Analysis SDK:
The Image Analysis SDK (preview) provides a convenient way to access the Image Analysis service using version 4.0 of the REST APIs
There are few limitations of this SDK as mentioned below and further details are updated here:
Image Analysis SDK Overview – Azure AI services | Microsoft Learn
Demo:
There are few samples to get started: check here
Let’s get started to get the caption for an image using Azure SDK.
Prerequisites:
Azure Subscription
Visual Studio IDE
Step by step:
Create Computer Vision:
Go to the Azure portal and create a Computer Vision.
Set up application:
Open Visual Studio, and under Get started select Create a new project. Set the template filters to C#/All Platforms/Console.
Select Console App (command-line application that can run on .NET on Windows, Linux and macOS) and choose Next.
Update the project name to ImageAnalysisQuickstart and choose Next. Select .NET 6.0 or above, and choose Create to create the project.
Install the client SDK
dotnet add package Azure.AI.Vision.ImageAnalysis –prerelease
Create environment variables:
Get the Key and endpoint from Azure portal.
setx VISION_KEY your-key
setx VISION_ENDPOINT your-endpoint
Output:
Another example:
That’s it!
In the demo code above, we load an image from url, retrieve its caption using the Azure Vision API, and print the caption to the console.
By following these steps, you’ll have a solid understanding of how to analyze images using the Azure SDK and C#.
Code Walkthrough:
Authenticate the client.
In order to interact with Azure Image Analysis, you’ll need to create an instance of the ImageAnalysisClient class. To configure a client for use with Azure Image Analysis, provide a valid endpoint URI to an Azure Computer Vision resource along with a corresponding key credential authorized to use the Azure Computer Vision resource.
using Azure;
using Azure.AI.Vision.ImageAnalysis;
using System;
using System.IO;
string endpoint = Environment.GetEnvironmentVariable("VISION_ENDPOINT");
string key = Environment.GetEnvironmentVariable("VISION_KEY");
// Create an Image Analysis client.
ImageAnalysisClient client = new ImageAnalysisClient(new Uri(endpoint), new AzureKeyCredential(key));Generate an image caption for an image file
Upload or reuse any existing image.
Generate an image caption for an image URL
The synchronous Analyze method call returns an ImageAnalysisResult object, which contains the generated caption and its confidence score in the range [0, 1].
By default, the caption may contain gender terms (for example: “man”, “woman”, “boy”, “girl”). You have the option to request gender-neutral terms (for example: “person”, “child”) by setting genderNeutralCaption = True when calling Analyze.
using Azure;
using Azure.AI.Vision.ImageAnalysis;
using System;
public class Program
{
ImageAnalysisClient client = new ImageAnalysisClient(
new Uri(endpoint),
new AzureKeyCredential(key));
ImageAnalysisResult result = client.Analyze(
new Uri("https://aka.ms/azsdk/image-analysis/sample.jpg"),
VisualFeatures.Caption,
new ImageAnalysisOptions { GenderNeutralCaption = true });
Console.WriteLine($"Image analysis results:");
Console.WriteLine($" Metadata: Model: {result.ModelVersion} Image dimensions: {result.Metadata.Width} x {result.Metadata.Height}");
Console.WriteLine($" Caption:");
Console.WriteLine($" '{result.Caption.Text}', Confidence {result.Caption.Confidence:F4}");
Console.WriteLine(" Read:");
foreach (DetectedTextBlock block in result.Read.Blocks)
foreach (DetectedTextLine line in block.Lines)
{
Console.WriteLine($" Line: '{line.Text}', " +
$"Bounding Polygon: [{string.Join(" ", line.BoundingPolygon)}]");
foreach (DetectedTextWord word in line.Words)
{
Console.WriteLine($" Word: '{word.Text}', " +
$"Confidence {word.Confidence.ToString("#.####")}, " +
$"Bounding Polygon: [{string.Join(" ", word.BoundingPolygon)}]");
}
}
}
static void Main()
{
try
{
AnalyzeImage();
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
}Complete Code can be found here:Code link
Samples:
In case you want to explore more samples, go through this link:
| ID | Image Analysis – Feature | Description |
| 1 | Get a Dense Caption for an image. | The Azure AI Vision ImageAnalysis Dense Caption feature generates detailed captions for up to 10 different regions in an image, including one for the whole image. It provides a more comprehensive understanding of the image content without the need to view the image itself. This library offers an easy way to extract dense captions from images using the Azure Computer Vision service. |
| 2 | Tags | The Azure AI Vision ImageAnalysis Tags feature extracts content tags for thousands of recognizable objects, living beings, scenery, and actions that appear in images. It can provide additional context about the image content and can be used for searching, filtering, and organizing images based on their content. This library offers an easy way to extract tags from images using the Azure Computer Vision service. |
| 3 | Objects | The Azure AI Vision ImageAnalysis Objects feature enables the detection of objects within images. It focuses on detecting physical objects in an image and returning their locations. This library offers an easy way to extract object information from images using the Azure Computer Vision service. |
| 4 | People Detection | The Azure AI Vision ImageAnalysis People Detection feature detects people appearing in images. The Azure Computer Vision service provides AI algorithms for processing images and returning bounding box coordinates for each detected person, along with a confidence score. This library offers an easy way to extract people detection information from images using the Azure Computer Vision service. |
| 5 | Read An Image | This sample demonstrates how to read text from an image. To get started you’ll need a URL for a Computer Vision endpoint |
| 6 | Analyze an Image with All Visual Features | This sample demonstrates how to analyze an image using all visual features provided by Azure AI Vision ImageAnalysis. This includes the features Caption, Dense Captions, Tags, Objects, SmartCrops, People, and Read. |
Conclusion
In this blog, we explored how to use the Azure SDK to access Azure Vision and Analyze Images. By utilizing this SDK, developers have the ability to tailor their solutions based on specific requirements.
In conclusion, the Azure SDK is a powerful enabler for accessing Azure Vision and Analyze Image. It allows developers to easily incorporate image analysis capabilities into their solutions and customize the solution to specific requirements. By leveraging the SDK’s features and capabilities, developers can build robust and efficient image analysis applications.
Don’t miss out on this opportunity to enhance your AI skills and explore the full potential of the Azure Image Analysis service.
Happy coding!
References:
https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/sdk/overview-sdk
