Loading...
Loading...
Found 4 Skills
Use when building Apache Spark applications, distributed data processing pipelines, or optimizing big data workloads. Invoke for DataFrame API, Spark SQL, RDD operations, performance tuning, streaming analytics.
Create Databricks AI/BI dashboards. CRITICAL: You MUST test ALL SQL queries via execute_sql BEFORE deploying. Follow guidelines strictly.
Diagnose, compare, and optimize Apache Spark applications and SQL queries using Spark History Server data. Use this skill whenever the user wants to understand why a Spark app is slow, compare two benchmark runs or TPC-DS results, find performance bottlenecks (skew, GC pressure, shuffle spill, straggler tasks), get tuning recommendations, or optimize Spark/Gluten configurations. Also trigger when the user mentions 'diagnose', 'compare runs', 'why is this query slow', 'tune my Spark job', 'benchmark comparison', 'performance regression', or asks about executor skew, shuffle overhead, AQE effectiveness, or Gluten offloading issues.
Databricks SQL query optimizer: analyzes a slow SQL query, rewrites it for speed using SQL-level optimizations only, validates byte-for-byte result equivalence, and benchmarks both versions with statistical significance testing. Use this skill whenever the user wants to optimize, speed up, tune, or benchmark a SQL query on Databricks. Trigger on: "/databricks-sql-autotuner", "optimize this SQL", "make this query faster", "tune my Databricks query", "benchmark SQL on Databricks", "speed up this spark SQL", "SQL performance on Databricks", "EXPLAIN this query", "why is my query slow on Databricks", "SQL query optimization Databricks", or whenever a user pastes a SQL query and mentions performance, slowness, or runtime.