Skip to content

azure-ai-inference : LangChain AzureAIChatCompletionsModel Does Not Support OpenAI-Style JSON Schema Format #44201

@RashmiJK

Description

@RashmiJK
  • Issue description
    When using langchain_azure_ai.AzureAIChatCompletionsModel with structured output (e.g., create_agent(..., response_format=ProviderStrategy()...), the integration fails with:
    ValueError: Unsupported response_format {'type': 'json_schema', 'json_schema': {...}}
    This occurs in azure/ai/inference/_patch.py

  • Steps to Reproduce

    from langchain_azure_ai.chat_models import AzureAIChatCompletionsModel
    from pydantic import BaseModel, Field
    import os
    
    llm = AzureAIChatCompletionsModel(
        endpoint=os.getenv("GITHUB_INFERENCE_ENDPOINT"),
        credential=os.getenv("GITHUB_TOKEN"),
        model="gpt-4o",
    )
    
    class ContactInfo(BaseModel):
        name: str = Field(description="The name of the person")
        email: str = Field(description="The email of the person")
        phone: str = Field(description="The phone number of the person")
    
    agent = create_agent(
        model = gllm_4_1,
        tools = [search_tool],
        response_format=ProviderStrategy(ContactInfo)
    )
    
    agent.invoke({"messages": [HumanMessage(content="Extract contact info from: John Doe, john@example.com, (555) 123-4567")]})
    
    
  • Alternatives considered
    Use langchain-openai instead, which treats Azure-compatible endpoints as OpenAI endpoints.

Metadata

Metadata

Assignees

No one assigned

    Labels

    AI Model InferenceIssues related to the client library for Azure AI Model Inference (\sdk\ai\azure-ai-inference)Service AttentionWorkflow: This issue is responsible by Azure service team.customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK teamquestionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions