data-transformers

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Data Transformers

数据转换器

Centralized transformation logic for consistent data shaping across API routes.

用于在各API路由间实现统一数据格式的中心化转换逻辑。

When to Use This Skill

何时使用该工具

Data transformation is scattered across routes
Need consistent output formats across endpoints
Want testable, reusable transformation functions
Building dashboards with aggregated data

数据转换逻辑分散在各个路由中
需要在各端点间保持统一的输出格式
想要可测试、可复用的转换函数
构建包含聚合数据的仪表板

Core Concepts

核心概念

Centralize all transformation logic in one place:

Aggregators (category totals, counts)
Rankers (top-N by score)
Trend calculators (comparing periods)
Sanitizers (validate and clean data)

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  Raw Data   │────▶│ Transformers │────▶│  API Output │
└─────────────┘     └──────────────┘     └─────────────┘

将所有转换逻辑集中在一处：

聚合器（分类总计、计数）
排名器（按分数取前N项）
趋势计算器（对比不同时间段）
清理器（验证并清理数据）

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  Raw Data   │────▶│ Transformers │────▶│  API Output │
└─────────────┘     └──────────────┘     └─────────────┘

Implementation

实现

TypeScript

typescript

// lib/transformers.ts

// ============================================
// Category Aggregation
// ============================================

interface CategoryTotals {
  [category: string]: number;
}

function aggregateCategories(
  items: Array<{ category: string; count?: number }>
): CategoryTotals {
  const totals: CategoryTotals = {};

  for (const item of items) {
    const category = item.category?.toUpperCase() || 'OTHER';
    totals[category] = (totals[category] || 0) + (item.count ?? 1);
  }

  return totals;
}

function categoriesToBreakdown(
  totals: CategoryTotals,
  previousTotals?: CategoryTotals
): Array<{ category: string; count: number; percentage: number; trend: string }> {
  const total = Object.values(totals).reduce((sum, count) => sum + count, 0);
  
  return Object.entries(totals)
    .map(([category, count]) => {
      let trend: 'increasing' | 'stable' | 'decreasing' = 'stable';
      
      if (previousTotals) {
        const prevCount = previousTotals[category] ?? 0;
        const change = count - prevCount;
        if (change > prevCount * 0.1) trend = 'increasing';
        else if (change < -prevCount * 0.1) trend = 'decreasing';
      }

      return {
        category,
        count,
        percentage: total > 0 ? count / total : 0,
        trend,
      };
    })
    .sort((a, b) => b.count - a.count);
}

// ============================================
// Ranking
// ============================================

interface Rankable {
  score: number;
  count: number;
}

function rankItems<T extends Rankable>(
  items: T[], 
  limit = 5
): (T & { rank: number })[] {
  return items
    .sort((a, b) => {
      if (b.score !== a.score) return b.score - a.score;
      return b.count - a.count;
    })
    .slice(0, limit)
    .map((item, index) => ({ ...item, rank: index + 1 }));
}

// ============================================
// Trend Calculation
// ============================================

type SimpleTrend = 'increasing' | 'stable' | 'decreasing';

function calculateTrend(current: number, previous: number): SimpleTrend {
  if (previous === 0) return 'stable';
  const change = (current - previous) / previous;
  
  if (change > 0.1) return 'increasing';
  if (change < -0.1) return 'decreasing';
  return 'stable';
}

function calculateRollingAverage(values: number[], window = 7): number {
  if (values.length === 0) return 0;
  const slice = values.slice(-window);
  return slice.reduce((sum, v) => sum + v, 0) / slice.length;
}

function calculatePercentChange(current: number, previous: number): number {
  if (previous === 0) return current > 0 ? 100 : 0;
  return ((current - previous) / previous) * 100;
}

// ============================================
// Data Sanitization
// ============================================

interface Hotspot {
  country: string;
  countryCode: string;
  lat: number;
  lon: number;
  riskScore: number;
  eventCount: number;
}

function sanitizeHotspot(raw: Partial<Hotspot>): Hotspot | null {
  if (!raw.country || !raw.countryCode) return null;
  
  return {
    country: raw.country,
    countryCode: raw.countryCode,
    lat: raw.lat ?? 0,
    lon: raw.lon ?? 0,
    riskScore: Math.min(100, Math.max(0, raw.riskScore ?? 0)),
    eventCount: Math.max(0, raw.eventCount ?? 0),
  };
}

function filterValidHotspots(hotspots: Partial<Hotspot>[]): Hotspot[] {
  return hotspots
    .map(sanitizeHotspot)
    .filter((h): h is Hotspot => h !== null);
}

// ============================================
// String Utilities
// ============================================

function truncate(str: string, maxLen: number): string {
  if (!str) return '';
  return str.length > maxLen ? str.slice(0, maxLen - 3) + '...' : str;
}

function slugify(str: string): string {
  return str
    .toLowerCase()
    .replace(/[^\w\s-]/g, '')
    .replace(/\s+/g, '-')
    .replace(/-+/g, '-')
    .trim();
}

// ============================================
// Date Utilities
// ============================================

function formatRelativeTime(date: Date): string {
  const now = new Date();
  const diffMs = now.getTime() - date.getTime();
  const diffMins = Math.floor(diffMs / 60000);
  const diffHours = Math.floor(diffMs / 3600000);
  const diffDays = Math.floor(diffMs / 86400000);

  if (diffMins < 1) return 'just now';
  if (diffMins < 60) return `${diffMins}m ago`;
  if (diffHours < 24) return `${diffHours}h ago`;
  if (diffDays < 7) return `${diffDays}d ago`;
  return date.toLocaleDateString();
}

export {
  aggregateCategories,
  categoriesToBreakdown,
  rankItems,
  calculateTrend,
  calculateRollingAverage,
  calculatePercentChange,
  sanitizeHotspot,
  filterValidHotspots,
  truncate,
  slugify,
  formatRelativeTime,
};

typescript

// lib/transformers.ts

// ============================================
// Category Aggregation
// ============================================

interface CategoryTotals {
  [category: string]: number;
}

function aggregateCategories(
  items: Array<{ category: string; count?: number }>
): CategoryTotals {
  const totals: CategoryTotals = {};

  for (const item of items) {
    const category = item.category?.toUpperCase() || 'OTHER';
    totals[category] = (totals[category] || 0) + (item.count ?? 1);
  }

  return totals;
}

function categoriesToBreakdown(
  totals: CategoryTotals,
  previousTotals?: CategoryTotals
): Array<{ category: string; count: number; percentage: number; trend: string }> {
  const total = Object.values(totals).reduce((sum, count) => sum + count, 0);
  
  return Object.entries(totals)
    .map(([category, count]) => {
      let trend: 'increasing' | 'stable' | 'decreasing' = 'stable';
      
      if (previousTotals) {
        const prevCount = previousTotals[category] ?? 0;
        const change = count - prevCount;
        if (change > prevCount * 0.1) trend = 'increasing';
        else if (change < -prevCount * 0.1) trend = 'decreasing';
      }

      return {
        category,
        count,
        percentage: total > 0 ? count / total : 0,
        trend,
      };
    })
    .sort((a, b) => b.count - a.count);
}

// ============================================
// Ranking
// ============================================

interface Rankable {
  score: number;
  count: number;
}

function rankItems<T extends Rankable>(
  items: T[], 
  limit = 5
): (T & { rank: number })[] {
  return items
    .sort((a, b) => {
      if (b.score !== a.score) return b.score - a.score;
      return b.count - a.count;
    })
    .slice(0, limit)
    .map((item, index) => ({ ...item, rank: index + 1 }));
}

// ============================================
// Trend Calculation
// ============================================

type SimpleTrend = 'increasing' | 'stable' | 'decreasing';

function calculateTrend(current: number, previous: number): SimpleTrend {
  if (previous === 0) return 'stable';
  const change = (current - previous) / previous;
  
  if (change > 0.1) return 'increasing';
  if (change < -0.1) return 'decreasing';
  return 'stable';
}

function calculateRollingAverage(values: number[], window = 7): number {
  if (values.length === 0) return 0;
  const slice = values.slice(-window);
  return slice.reduce((sum, v) => sum + v, 0) / slice.length;
}

function calculatePercentChange(current: number, previous: number): number {
  if (previous === 0) return current > 0 ? 100 : 0;
  return ((current - previous) / previous) * 100;
}

// ============================================
// Data Sanitization
// ============================================

interface Hotspot {
  country: string;
  countryCode: string;
  lat: number;
  lon: number;
  riskScore: number;
  eventCount: number;
}

function sanitizeHotspot(raw: Partial<Hotspot>): Hotspot | null {
  if (!raw.country || !raw.countryCode) return null;
  
  return {
    country: raw.country,
    countryCode: raw.countryCode,
    lat: raw.lat ?? 0,
    lon: raw.lon ?? 0,
    riskScore: Math.min(100, Math.max(0, raw.riskScore ?? 0)),
    eventCount: Math.max(0, raw.eventCount ?? 0),
  };
}

function filterValidHotspots(hotspots: Partial<Hotspot>[]): Hotspot[] {
  return hotspots
    .map(sanitizeHotspot)
    .filter((h): h is Hotspot => h !== null);
}

// ============================================
// String Utilities
// ============================================

function truncate(str: string, maxLen: number): string {
  if (!str) return '';
  return str.length > maxLen ? str.slice(0, maxLen - 3) + '...' : str;
}

function slugify(str: string): string {
  return str
    .toLowerCase()
    .replace(/[^\w\s-]/g, '')
    .replace(/\s+/g, '-')
    .replace(/-+/g, '-')
    .trim();
}

// ============================================
// Date Utilities
// ============================================

function formatRelativeTime(date: Date): string {
  const now = new Date();
  const diffMs = now.getTime() - date.getTime();
  const diffMins = Math.floor(diffMs / 60000);
  const diffHours = Math.floor(diffMs / 3600000);
  const diffDays = Math.floor(diffMs / 86400000);

  if (diffMins < 1) return 'just now';
  if (diffMins < 60) return `${diffMins}m ago`;
  if (diffHours < 24) return `${diffHours}h ago`;
  if (diffDays < 7) return `${diffDays}d ago`;
  return date.toLocaleDateString();
}

export {
  aggregateCategories,
  categoriesToBreakdown,
  rankItems,
  calculateTrend,
  calculateRollingAverage,
  calculatePercentChange,
  sanitizeHotspot,
  filterValidHotspots,
  truncate,
  slugify,
  formatRelativeTime,
};

Usage Examples

使用示例

API Route

API路由

typescript

// api/dashboard/route.ts
import { 
  aggregateCategories, 
  rankItems, 
  filterValidHotspots 
} from '@/lib/transformers';

export async function GET() {
  const rawData = await fetchFromDatabase();
  
  return Response.json({
    categories: aggregateCategories(rawData.predictions),
    topHotspots: rankItems(filterValidHotspots(rawData.hotspots), 5),
    trend: calculateTrend(rawData.todayCount, rawData.yesterdayCount),
  });
}

typescript

// api/dashboard/route.ts
import { 
  aggregateCategories, 
  rankItems, 
  filterValidHotspots 
} from '@/lib/transformers';

export async function GET() {
  const rawData = await fetchFromDatabase();
  
  return Response.json({
    categories: aggregateCategories(rawData.predictions),
    topHotspots: rankItems(filterValidHotspots(rawData.hotspots), 5),
    trend: calculateTrend(rawData.todayCount, rawData.yesterdayCount),
  });
}

Dashboard Component

仪表板组件

typescript

const breakdown = categoriesToBreakdown(
  currentTotals,
  previousTotals
);

// Returns:
// [
//   { category: 'MILITARY', count: 150, percentage: 0.45, trend: 'increasing' },
//   { category: 'POLITICAL', count: 100, percentage: 0.30, trend: 'stable' },
//   ...
// ]

typescript

const breakdown = categoriesToBreakdown(
  currentTotals,
  previousTotals
);

// Returns:
// [
//   { category: 'MILITARY', count: 150, percentage: 0.45, trend: 'increasing' },
//   { category: 'POLITICAL', count: 100, percentage: 0.30, trend: 'stable' },
//   ...
// ]

Best Practices

最佳实践

One file for all transformers - easy to find and test
Pure functions - no side effects, predictable output
Handle edge cases - empty arrays, missing fields, null values
Type safety - use TypeScript generics where appropriate
Export from types package - share across frontend and backend

所有转换器放在一个文件中——便于查找和测试
使用纯函数——无副作用，输出可预测
处理边缘情况——空数组、缺失字段、空值
类型安全——适当使用TypeScript泛型
从类型包导出——在前端和后端间共享

Common Mistakes

常见误区

Scattering transformation logic across routes
Not handling edge cases (empty arrays, null values)
Mutating input data instead of returning new objects
Missing type guards for nullable returns
Not testing transformers in isolation

在各路由中分散转换逻辑
未处理边缘情况（空数组、空值）
直接修改输入数据而非返回新对象
可为空返回值缺少类型守卫
未单独测试转换器

data-transformers

Original

Translation

Data Transformers

数据转换器

When to Use This Skill

何时使用该工具

Core Concepts

核心概念

Implementation

实现

TypeScript

TypeScript

Usage Examples

使用示例

API Route

API路由

Dashboard Component

仪表板组件

Best Practices

最佳实践

Common Mistakes

常见误区

Related Patterns

相关模式