ReleaseBytes
python Pydantic AI Releases · · 2.4.0

pydantic-ai v2.4.0: New Evaluators and Bug Fixes

sdkaipython-agentic
feature patch

Pydantic-ai version 2.4.0 introduces several new features, including enhanced evaluation capabilities with GEval and new span-based evaluators for agentic systems. It also addresses various bug fixes across different modules, improving stability and functionality. These updates benefit developers working with AI agents and LLM applications, offering more robust evaluation tools and refined behavior. The release includes dependency updates and contributions from new community members.

  • →Add GEval evaluator and standard quality metric rubrics for LLMJudge by @dmontagu in https://github.com/pydantic/pydantic-ai/pull/5129
  • →fix(xai): update XSearchTool handle limit by @gyx09212214-prog in https://github.com/pydantic/pydantic-ai/pull/5979
  • →Handle None metadata in Gradio weather agent example by @hramezani in https://github.com/pydantic/pydantic-ai/pull/6214
  • →fix(mistral): preserve extended usage fields from API response by @felipebridge in https://github.com/pydantic/pydantic-ai/pull/5693
  • →Avoid mutable default argument in process_tool_calls by @kratos0718 in https://github.com/pydantic/pydantic-ai/pull/6189
Features (3) ›

  • Split preserve_file_data into allow_uploaded_files (inbound security) and an AG-UI representation opt-in by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6232

  • Add GEval evaluator and standard quality metric rubrics for LLMJudge by @dmontagu in https://github.com/pydantic/pydantic-ai/pull/5129

  • Add agentic span-based evaluators (ToolCorrectness, TrajectoryMatch, ArgumentCorrectness, MaxToolCalls, MaxModelRequests) by @dmontagu in https://github.com/pydantic/pydantic-ai/pull/5130

Fixes (10) ›
  • fix(xai): update XSearchTool handle limit by @gyx09212214-prog in https://github.com/pydantic/pydantic-ai/pull/5979
  • Handle None metadata in Gradio weather agent example by @hramezani in https://github.com/pydantic/pydantic-ai/pull/6214
  • fix(mistral): preserve extended usage fields from API response by @felipebridge in https://github.com/pydantic/pydantic-ai/pull/5693
  • Avoid mutable default argument in process_tool_calls by @kratos0718 in https://github.com/pydantic/pydantic-ai/pull/6189
  • feat forward WebSearchTool.user_location to xAI web search by @manan-tech in https://github.com/pydantic/pydantic-ai/pull/6181

  • Clean up the pending task and streams on KeyboardInterrupt in synchronous run_until_complete wrappers by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6198

  • fix(tool-search): drop redundant description from ToolSearchMatch by @serozhenka in https://github.com/pydantic/pydantic-ai/pull/5654
  • fix(temporal): pass through genai_prices and httpx2 for workflow cost() by @syf2211 in https://github.com/pydantic/pydantic-ai/pull/6219
  • fix(evals): reject non-positive max_concurrency by @VectorPeak in https://github.com/pydantic/pydantic-ai/pull/6228
  • Lazy-start run_stream_events() background task on first event iteration by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6234
Read the original announcement →

https://github.com/pydantic/pydantic-ai/releases/tag/v2.4.0

Related releases