Can we use the azure gpt key?

[YongLD]

As the title said, How can I use the azure gpt key in DataDreamer?

[AjayP13]

Hi @YongLD, yes, you can, see the documentation for OpenAI here. It supports taking in parameters like "base_url" which can be used to connect to the Azure OpenAI service:

https://datadreamer.dev/docs/latest/datadreamer.llms.html#datadreamer.llms.OpenAI

So you can set base_url="https://YOUR_RESOURCE_NAME.openai.azure.com" and api_version="YOUR_API_VERSION" and set api_key="YOUR_AZURE_API_KEY".

[seanbenhur]

Hi @AjayP13 , can you provide a sample code?, I am trying to use Azure OAI and have provided my resource name in the model_name but I am getting the below error

[ 🤖 DataDreamer 💤 ] Step 'Generate Attributes' is running. ⏳
[ 🤖 DataDreamer 💤 ] [ 🧠 OpenAI (gpt4)] Finished call to 'datadreamer.llms.openai.OpenAI.retry_wrapper.<locals>._retry_wrapper' after 0.525(s), this was the 1st time calling it.
[ 🤖 DataDreamer 💤 ] [ 🧠 OpenAI (gpt4)] Retrying datadreamer.llms.openai.OpenAI.retry_wrapper.<locals>._retry_wrapper in 3.0 seconds as it raised NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}.
[ 🤖 DataDreamer 💤 ] [ 🧠 OpenAI (gpt4)] Finished call to 'datadreamer.llms.openai.OpenAI.retry_wrapper.<locals>._retry_wrapper' after 3.624(s), this was the 2nd time calling it.
[ 🤖 DataDreamer 💤 ] [ 🧠 OpenAI (gpt4)] Retrying datadreamer.llms.openai.OpenAI.retry_wrapper.<locals>._retry_wrapper in 3.0 seconds as it raised NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}.

[AjayP13]

Hi @seanbenhur, I actually don't use Azure OpenAI, so I've never tested this directly since I don't have a Azure OpenAI key, but you can see on this line:

DataDreamer/src/llms/openai.py

Line 196 in ad3dd9c

else openai.AzureOpenAI(**other_kwargs, **self.kwargs)

how we pass base_url, api_version, and api_key to the underlying AzureOpenAI class from the openai Python library.

If you don't mind debugging this for me and seeing if you can get it to work or seeing if there's a bug I can fix it. And if you get it to work, it would be great if you could report back on what was the issue and a code snippet for others. If you want to reach me you can email me at ajayp@seas.upenn.edu and I can help assist you as well trying to figure this out further if you can't get it to work / we could debug over a video call if needed.

[seanbenhur]

Sure, I will work on it

[jonasherfort]

@AjayP13 The problem seems to be that in the code Azure OpenAI uses azure_ad_token if you're using Azure Active Directory (AAD) tokens. If you're using API keys, you should continue using api_key, not azure_ad_token.

[AjayP13]

Hi @jonasherfort, thanks for identifying the issue. I will work on fixing this soon, in the mean time you can override the OpenAI class to fix the issue to unblock yourself if you are comfortable doing overrides.