Perform EDA and diagnose data quality

Given a tabular dataset loaded into a pandas DataFrame, write code to perform exploratory data analysis end-to-end: inspect and report column data types, measure feature scales (min, max, mean, std) and identify potential outliers, check and summarize missing values (counts and percentages), compute basic summary statistics, and plot a correlation heatmap to highlight strongly related features. Conclude with a short written summary of quality issues and next steps to address them.

Column-level summary statistics and missing data profile

You are given a table customer_metrics containing basic numeric features for a small set of customers. Using SQL, produce a column-level exploratory data analysis summary for the numeric feature columns (age, annual_income, monthly_spend). For each of these columns, return a single row with:

• column_name
• non_null_count
• missing_count
• missing_pct (percentage of rows where the value is NULL, rounded to 2 decimal places)
• min_value
• max_value
• avg_value (rounded to 2 decimal places)

Only include rows for the three feature columns (age, annual_income, monthly_spend), not for customer_id.

Table customer_metrics

column	type	constraints
customer_id	INT	PRIMARY KEY, NOT NULL
age	INT
annual_income	DECIMAL(10,2)
monthly_spend	DECIMAL(10,2)

Sample data:

age	annual_income	customer_id	monthly_spend
20	100000	1	200
25	90000	2	250
30	80000	3	300
35	70000	4	350
40	60000	5	400
NULL	55000	6	NULL
50	NULL	7	500
60	120000	8	600

Expected output

One row per numeric feature column (age, annual_income, monthly_spend) with counts of non-null and missing values, percentage missing, and basic scale statistics (min, max, average).

avg_value	column_name	max_value	min_value	missing_count	missing_pct	non_null_count
37.14	age	60	20	1	12.5	7
82142.86	annual_income	120000	55000	1	12.5	7
371.43	monthly_spend	600	200	1	12.5	7

Hints

1. Use COUNT(column) to count non-null values and COUNT(*) to count all rows.
2. Stack per-column summaries using UNION ALL, and use ROUND on averages and percentages for readability.

Compute pairwise correlations between numeric features

Using the same customer_metrics table, compute pairwise Pearson correlation coefficients between the numeric feature columns age, annual_income, and monthly_spend. Assume your SQL dialect provides a CORR(x, y) aggregate function that ignores NULL values.

Return one row per unordered column pair with:

• col_x (first column name)
• col_y (second column name)
• corr_value (Pearson correlation between the two columns)

Filter the result to only include pairs where the absolute value of the correlation is at least 0.9, and order the output by descending absolute correlation and then by col_x, col_y.

Table customer_metrics

column	type	constraints
customer_id	INT	PRIMARY KEY, NOT NULL
age	INT
annual_income	DECIMAL(10,2)
monthly_spend	DECIMAL(10,2)

Sample data:

age	annual_income	customer_id	monthly_spend
20	100000	1	200
25	90000	2	250
30	80000	3	300
35	70000	4	350
40	60000	5	400
NULL	55000	6	NULL
50	NULL	7	500
60	120000	8	600

Expected output

One row per numeric column pair with strong linear relationship (absolute correlation at least 0.9). With the sample data, age and monthly_spend are perfectly linearly related, so only that pair appears.

col_x	col_y	corr_value
age	monthly_spend	1

Hints

1. Manually enumerate the column pairs you want to correlate and combine them with UNION ALL.
2. Use the CORR aggregate on each pair and then filter on ABS(corr_value) to keep only strongly related features.