Loading...
Loading...
Found 92 Skills
Comprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, implementing real-time data processing, handling stateful operations, or optimizing streaming performance.
Use this skill when building data pipelines, ETL/ELT workflows, or data transformation layers. Triggers on Airflow DAG design, dbt model creation, Spark job optimization, streaming vs batch architecture decisions, data ingestion, data quality checks, pipeline orchestration, incremental loads, CDC (change data capture), schema evolution, and data warehouse modeling. Acts as a senior data engineer advisor for building reliable, scalable data infrastructure.
Develop Microsoft Fabric Spark/data engineering workflows with intelligent routing to specialized resources. Provides core workspace/lakehouse management and routes to: data engineering patterns, development workflow, or infrastructure orchestration. Use when the user wants to: (1) manage Fabric workspaces and resources, (2) develop notebooks and PySpark applications, (3) design data pipelines and orchestration, (4) provision infrastructure as code. Triggers: "develop notebook", "data engineering", "workspace setup", "pipeline design", "infrastructure provisioning", "Delta Lake patterns", "Spark development", "lakehouse configuration", "organize lakehouse tables", "create Livy session", "notebook deployment".
Sales rep / account manager persona for Spark. Client relationship tracking, pipeline review, follow-up cadence, and deal context.
Free 9-week data engineering course covering Docker, Terraform, Kestra, BigQuery, dbt, Spark, and Kafka with hands-on projects
Create notarized macOS app releases with Sparkle auto-updates, DMG installers, and GitHub releases. Use when releasing macOS apps, creating DMG files, notarizing apps, or setting up Sparkle updates. Handles version updates, code signing, notarization, and distribution.
Scala 3.4+ development specialist covering Akka, Cats Effect, ZIO, and Spark patterns. Use when building distributed systems, big data pipelines, or functional programming applications.
Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation
Query a running Apache Spark History Server from Copilot CLI. Use this whenever the user wants to inspect SHS applications, jobs, stages, executors, SQL executions, environment details, or event logs, especially when they mention Spark History Server, SHS, event log history, benchmark runs, or application IDs.
V8 JIT Compilation, TurboFan, Maglev, Sparkplug. Load this when needing to understand V8's compilation pipeline, JIT optimization, or JITless mode.
Meeting manager persona for Spark. Meeting preparation, transcript review, follow-up drafts, and scheduling.
Implements Syncfusion Flutter Spark Charts (SfSparkLineChart, SfSparkAreaChart, SfSparkBarChart, SfSparkWinLossChart) for compact, lightweight data visualization. Use when working with micro charts, sparklines, KPI indicators, or inline trend charts in Flutter dashboards. This skill covers chart configuration, data binding, markers, tooltips, and trackball for all four spark chart types.