The chat functionality in the Azure AI Studio playground allows users to engage with AI-driven conversations, both hearing and speaking with AI models.

This interactive environment provides a convenient way to test, evaluate, and experiment with various chat models, making it a valuable tool for research and development in the field of AI.

 Overview:

Do you want to give your app the ability to hear and speak by pairing Azure OpenAI Service with Azure AI Speech to enable richer interactions?

If so, then let’s explore the features provided by Microsoft AI and jump right into a demo to see how everything works.

We will use Azure OpenAI Service and Azure AI Speech to:

• Speak to the assistant via speech to text.
• Hear the assistant’s response via text to speech.

The speech to text and text to speech features can be used together or separately in the Azure AI Studio playground. We will see how to use the playground to test your chat model before deploying it.

Before demo, lets understands few concepts:

Azure AI Studio:

Reter this post for AI Studio Getting started with Azure AI Studio

Azure AI Speech

This service enables developers to build speech-to-text and text-to-speech applications using advanced speech recognition and language models. It offers features like real-time transcription, speech synthesis, and sentiment analysis.

Getting started with Azure AI Services

 Azure Open AI:

Refer this post to know more about Getting started with Azure Open AI Services

By pairing Azure OpenAI Service with Azure AI Speech, you can provide your app with enhanced voice capabilities, enabling richer interactions and a more engaging user experience.

Demo:

Prerequisite:

• An Azure subscription
• Access to Azure OpenAI in the desired Azure subscription.
• An Azure AI hub resource with a chat model deployed.
• An Azure AI project in Azure AI Studio.

High Level steps:

Below table demonstrates how level steps needed for this demonstration.

Step by Step:

Let’s first cover the Pre-requites part.

• Access granted to Azure OpenAI in the desired Azure subscription.

Request Access to Azure OpenAI Service (microsoft.com)

• An Azure AI hub resource with a chat model deployed.
• Sign in to Azure AI Studio.
• Go to your project or create a new project in Azure AI Studio.

Create a new AI Project

Once a project is created, you can access the Tools, Components, and AI project settings assets in the left navigation panel.

For a project that uses an Azure AI hub with support for Azure OpenAI, you see the Playground navigation option under Tools.

Configure Playground:

Select Create and Real-time endpoint:

Create Speech to Text Service & Text to Speech Service:

You need to create below two Azure AI Speech services.

• Create Azure AI Speech to Text
• Create Azure AI Text to Speech

Follow below steps:

Select Network: All Networks for this demonstration.

Identity: Off

You will below detail for Speech service.

Similarly, create another Speech service (required for Text to Speech).

Playground settings-> Enable Speech to Text Service:

Enable Speech to text and text to speech services.

Now, you have configured all the settings. Time now to Evaluate!

Let’s Go to the Playground, together 😊

In this chat session, you use both speech to text and text to speech. You use the speech to text feature to speak to the assistant, and the text to speech feature to hear the assistant’s response.

Select the microphone button and speak to the assistant.

For example, you can say “Do you know where I can get an Xbox”.

Or try with some other topic, “Recommend me with some recent Netflix web series”.

I tried with below:

Behind the scene:

We used Azure AI Speech to converse with an Azure OpenAI Service chat model. The text recognized by the Speech service is sent to Azure OpenAI. The text response from Azure OpenAI is then synthesized by the Speech service.

Speak into the microphone to start a conversation with Azure OpenAI.

• The Speech service recognizes your speech and converts it into text (speech to text).
• Your request as text is sent to Azure OpenAI.
• The Speech service text to speech feature synthesizes the response from Azure OpenAI to the default speaker.

Let’s also see from Developers point of view, until now we have used the Azure AI studio to create the chat model with these components, and there will be next steps to deploy this chat model to your web apps (with and without your data). We will cover this in an upcoming post.

Also, let’s have a look at the code behind used for this!

Code behind:

You can select the View Code button to view and copy the sample code, which includes configuration for Azure OpenAI and Speech services. You can use the sample code to enable speech to text and text to speech in your application.

Complete Python code is here:

Conclusion:

Hurray! 🎉

You have successfully configured a chat model using the Model, gpt-35-turbo-16k, and utilized Azure AI Studio for the configuration.

In this blog post, we covered the process of setting up a chat model using Azure AI Studio and the gpt-35-turbo-16k model. By following the instructions provided, you were able to configure your chat model and integrate it with Azure AI services.

In addition to configuring the chat model, you also used Azure AI services such as Text to Speech and Speech to Text to enhance your chat experience. These services allow you to convert text to speech and vice versa, allowing for more interactive conversations with your chat model.

Now, we will move on to the follow-up post, where we will deploy this chat model to your web app and explore more advanced features.

Stay tuned for more exciting updates! 🚀

References:

https://learn.microsoft.com/en-us/azure/ai-studio/how-to/create-projects

https://learn.microsoft.com/en-us/azure/ai-studio/quickstarts/hear-speak-playground

https://learn.microsoft.com/en-us/azure/ai-studio/tutorials/deploy-chat-web-app

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/overview

https://github.com/Azure-Samples/Cognitive-Speech-TTS/tree/master/SpokenChat