ReleaseBytes
databricks Databricks Blog ·

Run, Debug, and Scale Databricks Workloads from Local IDEs

blogaidatabricksgaengineer
feature

Databricks has enhanced its IDE experience, allowing developers to connect VS Code, Cursor, or the CLI directly to Databricks compute via a new SSH tunnel. This enables interactive running and debugging of Python and SQL workloads on real cluster infrastructure, including Serverless and AI Runtime. The updates aim to minimize workspace switches, keep project dependencies in sync, and leverage coding agents with full workspace context, improving local development for complex data and ML pipelines.

  • →Local IDE Integration for Databricks Workloads via SSH Tunnel
  • →Unified Environment and Coding Agent Support
  • →Simplified Connection and Management
  • →Unity Catalog and AI Gateway Integration
  • →Expanded Environment Customization
Features (1) ›
  • Local IDE Integration for Databricks Workloads via SSH Tunnel

    Developers can now connect their local IDEs or CLI to Databricks compute, including Serverless, AI Runtime, and dedicated clusters, using a new SSH tunnel. This enables interactive execution and debugging of Python and SQL workloads directly from the local environment.

Enhancements (3) ›
  • Unified Environment and Coding Agent Support

    The update ensures a single environment across the IDE and Databricks workspace, keeping files and project dependencies consistently synchronized with Databricks Runtime. It also allows coding agents like Cursor, Copilot, and Claude Code to operate with full workspace context within the SSH tunnel.

  • Unity Catalog and AI Gateway Integration

    The new experience allows developers to explore Unity Catalog data assets directly from their IDE, eliminating the need to switch to the workspace. Unity AI Gateway will also be automatically configured for SSH tunnel users, providing governed access and spend tracking for AI services.

  • Expanded Environment Customization

    Future updates will enable configuration of non-Python dependencies and custom Docker images at SSH tunnel startup. The existing IDE Extension will also be integrated, allowing deployment and management of Declarative Automation Bundles from a unified UI within the IDE.

Notes (1) ›
  • Simplified Connection and Management

    Connecting to the SSH tunnel is streamlined with simple Databricks CLI commands, supporting various cluster types and accelerator configurations. Additional features include managing project dependencies with a base environment flag and monitoring usage and costs via serverless usage policies.

Read the original announcement →

https://www.databricks.com/blog/run-debug-and-scale-databricks-workloads-your-local-ide

Related releases