data-science-eda

Original：🇺🇸 English

Translated

Exploratory Data Analysis (EDA): profiling, visualization, correlation analysis, and data quality checks. Use when understanding dataset structure, distributions, relationships, or preparing for feature engineering and modeling.

1installs

Sourcelegout/data-platform-agent-skills

Added on2026-02-21

NPX Install

npx skill4agent add legout/data-platform-agent-skills data-science-eda

SKILL.md Content

View Translation Comparison →

Exploratory Data Analysis (EDA)

Use this skill for understanding datasets before modeling: profiling distributions, detecting anomalies, identifying relationships, and assessing data quality.

When to use this skill

New dataset — need orientation on structure, types, distributions
Before feature engineering — understand variable relationships
Data quality investigation — find anomalies, missing patterns, outliers
Model preparation — validate assumptions about data

Core EDA workflow

Profile structure
- Schema, types, cardinality
- Missing value patterns
Analyze distributions
- Numerical: histograms, boxplots, skewness
- Categorical: frequencies, rare categories
Explore relationships
- Correlation matrix (numerical)
- Cross-tabulations (categorical)
- Target-variable relationships
Identify issues
- Outliers, duplicates, inconsistencies
- Class imbalance (classification)
- Temporal patterns (time series)

Quick tool selection

Task	Default choice	Notes
Automated profiling	ydata-profiling / pandas-profiling	Fast comprehensive reports
Interactive exploration	ipywidgets + plotly	Drill-down capability
Statistical tests	scipy.stats	Normality, correlations
Large datasets	Polars + lazy	Memory-efficient

Core implementation rules

1) Start with automated profiling

python

import polars as pl
from ydata_profiling import ProfileReport

df = pl.read_parquet("data.parquet")
profile = ProfileReport(df.to_pandas(), title="Data Profile")
profile.to_file("profile_report.html")

2) Focus on actionable insights

Document outliers worth investigating (not all outliers are problems)
Flag features with high cardinality or rare categories
Note strong correlations that may cause multicollinearity

3) Visualize for communication

Distribution plots for key variables
Correlation heatmap
Missing value patterns
Target relationship plots

4) Validate assumptions

Check for expected ranges/business rules
Verify temporal consistency
Confirm key relationships match domain knowledge

Common anti-patterns

❌ Skipping EDA and jumping to modeling
❌ Treating all outliers as errors
❌ Ignoring missing value mechanisms (MCAR/MAR/MNAR)
❌ Over-plotting large datasets without sampling
❌ Not documenting findings for team

Progressive disclosure

```
../references/automated-profiling.md
```
— ydata-profiling, Sweetviz, D-Tale
```
../references/visualization-patterns.md
```
— Matplotlib, Seaborn, Plotly patterns
```
../references/statistical-tests.md
```
— Scipy statistical tests guide
```
../references/large-dataset-eda.md
```
— Sampling, Polars, Dask approaches

Related skills

```
@data-science-feature-engineering
```
— Next step after EDA
```
@data-science-model-evaluation
```
— Validate modeling assumptions
```
@data-engineering-quality
```
— Data validation frameworks