marketing-pipeline-ai-content

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Marketing Pipeline AI Content Automation

营销流水线AI内容自动化

Skill by ara.so — Marketing Skills collection.

This skill enables AI coding agents to work with the Ultimate AI Content Pipeline - a TypeScript-based content automation system that handles research, scriptwriting, and video generation using AI (Claude 3, OpenAI) and Remotion for video rendering.

由ara.so提供的技能 — 营销技能合集。

该技能支持AI编码Agent对接终极AI内容流水线——这是一个基于TypeScript的内容自动化系统，可借助AI（Claude 3、OpenAI）和Remotion完成调研、脚本撰写与视频渲染。

What This Project Does

本项目功能

The Marketing Pipeline is an end-to-end content automation system that:

Auto-crawls research from sources like TechCrunch, a16z, Twitter/X, and LinkedIn
Generates content in multiple formats (toplists, POV, case studies, how-tos) using Claude/OpenAI
Creates multilingual content (English & Vietnamese) with customizable tone
Renders videos automatically using Remotion for social media platforms
Provides a Next.js interface for managing the entire pipeline

营销流水线是一套端到端的内容自动化系统，具备以下能力：

自动爬取调研内容：从TechCrunch、a16z、Twitter/X和LinkedIn等来源获取信息
多格式内容生成：借助Claude/OpenAI生成榜单、观点文、案例研究、教程等多种格式内容
多语言内容支持：生成英文和越南语内容，支持自定义语气风格
自动渲染视频：使用Remotion为社交媒体平台生成视频
提供Next.js管理界面：用于管控整个流水线流程

Installation

安装步骤

bash

undefined

bash

undefined

Clone the repository

克隆仓库

git clone https://github.com/pennydinh/marketing-pineline-share.git cd marketing-pineline-share

Install dependencies

安装依赖

npm install

or

或

yarn install

or

或

pnpm install

undefined

pnpm install

undefined

Configuration

配置说明

Create a

.env.local

file in the root directory:

env

undefined

在根目录创建

.env.local

文件：

env

undefined

AI Services

AI服务配置

ANTHROPIC_API_KEY=your_claude_api_key OPENAI_API_KEY=your_openai_api_key

Research APIs

调研API配置

RAPIDAPI_KEY=your_rapidapi_key

Content Settings

内容设置

DEFAULT_LANGUAGE=en TONE=professional

undefined

DEFAULT_LANGUAGE=en TONE=professional

undefined

Project Structure

项目结构

marketing-pipeline/
├── src/
│   ├── app/              # Next.js app directory
│   ├── components/       # React components
│   ├── lib/
│   │   ├── research/     # Research crawling modules
│   │   ├── ai/           # AI generation (Claude/OpenAI)
│   │   ├── render/       # Remotion video rendering
│   │   └── utils/        # Helper functions
│   └── types/            # TypeScript type definitions
├── public/               # Static assets
└── remotion/            # Remotion compositions

marketing-pipeline/
├── src/
│   ├── app/              # Next.js应用目录
│   ├── components/       # React组件
│   ├── lib/
│   │   ├── research/     # 调研爬取模块
│   │   ├── ai/           # AI生成模块（Claude/OpenAI）
│   │   ├── render/       # Remotion视频渲染模块
│   │   └── utils/        # 辅助工具函数
│   └── types/            # TypeScript类型定义
├── public/               # 静态资源
└── remotion/            # Remotion合成内容

Core API Usage

核心API使用示例

1. Research Content Crawling

1. 调研内容爬取

typescript

import { crawlResearch } from '@/lib/research/crawler';

interface ResearchOptions {
  keyword: string;
  sources: ('techcrunch' | 'a16z' | 'twitter' | 'linkedin')[];
  timeframe: '24h' | '7d' | '30d';
}

async function gatherResearch(options: ResearchOptions) {
  const research = await crawlResearch({
    keyword: options.keyword,
    sources: options.sources,
    timeframe: options.timeframe,
  });
  
  return research; // Returns { articles: [], insights: [], data: [] }
}

// Example usage
const data = await gatherResearch({
  keyword: 'AI automation',
  sources: ['techcrunch', 'twitter'],
  timeframe: '24h',
});

typescript

import { crawlResearch } from '@/lib/research/crawler';

interface ResearchOptions {
  keyword: string;
  sources: ('techcrunch' | 'a16z' | 'twitter' | 'linkedin')[];
  timeframe: '24h' | '7d' | '30d';
}

async function gatherResearch(options: ResearchOptions) {
  const research = await crawlResearch({
    keyword: options.keyword,
    sources: options.sources,
    timeframe: options.timeframe,
  });
  
  return research; // 返回 { articles: [], insights: [], data: [] }
}

// 使用示例
const data = await gatherResearch({
  keyword: 'AI automation',
  sources: ['techcrunch', 'twitter'],
  timeframe: '24h',
});

2. AI Content Generation

2. AI内容生成

typescript

import { generateContent } from '@/lib/ai/generator';
import { Anthropic } from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

interface ContentConfig {
  format: 'toplist' | 'pov' | 'case-study' | 'how-to';
  tone: 'professional' | 'friendly' | 'humorous';
  language: 'en' | 'vi';
  research: any;
}

async function createContent(config: ContentConfig) {
  const prompt = `
Based on this research: ${JSON.stringify(config.research)}

Create a ${config.format} article in ${config.language} with a ${config.tone} tone.
Include data-backed insights and real examples.
  `;

  const message = await anthropic.messages.create({
    model: 'claude-3-5-sonnet-20241022',
    max_tokens: 4096,
    messages: [{
      role: 'user',
      content: prompt,
    }],
  });

  return message.content[0].text;
}

typescript

import { generateContent } from '@/lib/ai/generator';
import { Anthropic } from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

interface ContentConfig {
  format: 'toplist' | 'pov' | 'case-study' | 'how-to';
  tone: 'professional' | 'friendly' | 'humorous';
  language: 'en' | 'vi';
  research: any;
}

async function createContent(config: ContentConfig) {
  const prompt = `
Based on this research: ${JSON.stringify(config.research)}

Create a ${config.format} article in ${config.language} with a ${config.tone} tone.
Include data-backed insights and real examples.
  `;

  const message = await anthropic.messages.create({
    model: 'claude-3-5-sonnet-20241022',
    max_tokens: 4096,
    messages: [{
      role: 'user',
      content: prompt,
    }],
  });

  return message.content[0].text;
}

3. OpenAI Alternative

3. OpenAI替代方案

typescript

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function generateWithOpenAI(prompt: string) {
  const completion = await openai.chat.completions.create({
    model: 'gpt-4-turbo-preview',
    messages: [
      {
        role: 'system',
        content: 'You are an expert content creator specializing in marketing and social media.',
      },
      {
        role: 'user',
        content: prompt,
      },
    ],
    temperature: 0.7,
    max_tokens: 3000,
  });

  return completion.choices[0].message.content;
}

typescript

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function generateWithOpenAI(prompt: string) {
  const completion = await openai.chat.completions.create({
    model: 'gpt-4-turbo-preview',
    messages: [
      {
        role: 'system',
        content: 'You are an expert content creator specializing in marketing and social media.',
      },
      {
        role: 'user',
        content: prompt,
      },
    ],
    temperature: 0.7,
    max_tokens: 3000,
  });

  return completion.choices[0].message.content;
}

4. Video Rendering with Remotion

4. Remotion视频渲染

typescript

import { bundle } from '@remotion/bundler';
import { renderMedia, selectComposition } from '@remotion/renderer';
import path from 'path';

interface VideoConfig {
  title: string;
  content: string;
  duration: number;
  format: 'reels' | 'tiktok' | 'shorts';
}

async function renderContentVideo(config: VideoConfig) {
  const bundleLocation = await bundle({
    entryPoint: path.join(process.cwd(), 'remotion/index.ts'),
    webpackOverride: (config) => config,
  });

  const composition = await selectComposition({
    serveUrl: bundleLocation,
    id: 'ContentVideo',
    inputProps: {
      title: config.title,
      content: config.content,
    },
  });

  const dimensions = {
    reels: { width: 1080, height: 1920 },
    tiktok: { width: 1080, height: 1920 },
    shorts: { width: 1080, height: 1920 },
  };

  await renderMedia({
    composition,
    serveUrl: bundleLocation,
    codec: 'h264',
    outputLocation: `out/${config.title}.mp4`,
    ...dimensions[config.format],
  });
}

typescript

import { bundle } from '@remotion/bundler';
import { renderMedia, selectComposition } from '@remotion/renderer';
import path from 'path';

interface VideoConfig {
  title: string;
  content: string;
  duration: number;
  format: 'reels' | 'tiktok' | 'shorts';
}

async function renderContentVideo(config: VideoConfig) {
  const bundleLocation = await bundle({
    entryPoint: path.join(process.cwd(), 'remotion/index.ts'),
    webpackOverride: (config) => config,
  });

  const composition = await selectComposition({
    serveUrl: bundleLocation,
    id: 'ContentVideo',
    inputProps: {
      title: config.title,
      content: config.content,
    },
  });

  const dimensions = {
    reels: { width: 1080, height: 1920 },
    tiktok: { width: 1080, height: 1920 },
    shorts: { width: 1080, height: 1920 },
  };

  await renderMedia({
    composition,
    serveUrl: bundleLocation,
    codec: 'h264',
    outputLocation: `out/${config.title}.mp4`,
    ...dimensions[config.format],
  });
}

Common Patterns

常见使用模式

Complete Content Pipeline

完整内容流水线

typescript

import { crawlResearch } from '@/lib/research/crawler';
import { generateContent } from '@/lib/ai/generator';
import { renderContentVideo } from '@/lib/render/video';

async function runContentPipeline(keyword: string) {
  try {
    // Step 1: Research
    console.log('Starting research...');
    const research = await crawlResearch({
      keyword,
      sources: ['techcrunch', 'twitter'],
      timeframe: '24h',
    });

    // Step 2: Generate content
    console.log('Generating content...');
    const content = await createContent({
      format: 'toplist',
      tone: 'professional',
      language: 'en',
      research,
    });

    // Step 3: Render video
    console.log('Rendering video...');
    await renderContentVideo({
      title: keyword,
      content,
      duration: 30,
      format: 'reels',
    });

    return {
      success: true,
      content,
      videoPath: `out/${keyword}.mp4`,
    };
  } catch (error) {
    console.error('Pipeline error:', error);
    throw error;
  }
}

typescript

import { crawlResearch } from '@/lib/research/crawler';
import { generateContent } from '@/lib/ai/generator';
import { renderContentVideo } from '@/lib/render/video';

async function runContentPipeline(keyword: string) {
  try {
    // 步骤1：调研
    console.log('Starting research...');
    const research = await crawlResearch({
      keyword,
      sources: ['techcrunch', 'twitter'],
      timeframe: '24h',
    });

    // 步骤2：生成内容
    console.log('Generating content...');
    const content = await createContent({
      format: 'toplist',
      tone: 'professional',
      language: 'en',
      research,
    });

    // 步骤3：渲染视频
    console.log('Rendering video...');
    await renderContentVideo({
      title: keyword,
      content,
      duration: 30,
      format: 'reels',
    });

    return {
      success: true,
      content,
      videoPath: `out/${keyword}.mp4`,
    };
  } catch (error) {
    console.error('Pipeline error:', error);
    throw error;
  }
}

Bilingual Content Generation

双语内容生成

typescript

async function generateBilingualContent(research: any) {
  const [englishContent, vietnameseContent] = await Promise.all([
    createContent({
      format: 'pov',
      tone: 'professional',
      language: 'en',
      research,
    }),
    createContent({
      format: 'pov',
      tone: 'professional',
      language: 'vi',
      research,
    }),
  ]);

  return {
    en: englishContent,
    vi: vietnameseContent,
  };
}

typescript

async function generateBilingualContent(research: any) {
  const [englishContent, vietnameseContent] = await Promise.all([
    createContent({
      format: 'pov',
      tone: 'professional',
      language: 'en',
      research,
    }),
    createContent({
      format: 'pov',
      tone: 'professional',
      language: 'vi',
      research,
    }),
  ]);

  return {
    en: englishContent,
    vi: vietnameseContent,
  };
}

Batch Content Creation

批量内容创建

typescript

async function createMultipleFormats(keyword: string) {
  const research = await crawlResearch({
    keyword,
    sources: ['techcrunch', 'a16z'],
    timeframe: '7d',
  });

  const formats: Array<'toplist' | 'pov' | 'case-study' | 'how-to'> = [
    'toplist',
    'pov',
    'case-study',
    'how-to',
  ];

  const contents = await Promise.all(
    formats.map((format) =>
      createContent({
        format,
        tone: 'professional',
        language: 'en',
        research,
      })
    )
  );

  return formats.reduce((acc, format, index) => {
    acc[format] = contents[index];
    return acc;
  }, {} as Record<string, string>);
}

typescript

async function createMultipleFormats(keyword: string) {
  const research = await crawlResearch({
    keyword,
    sources: ['techcrunch', 'a16z'],
    timeframe: '7d',
  });

  const formats: Array<'toplist' | 'pov' | 'case-study' | 'how-to'> = [
    'toplist',
    'pov',
    'case-study',
    'how-to',
  ];

  const contents = await Promise.all(
    formats.map((format) =>
      createContent({
        format,
        tone: 'professional',
        language: 'en',
        research,
      })
    )
  );

  return formats.reduce((acc, format, index) => {
    acc[format] = contents[index];
    return acc;
  }, {} as Record<string, string>);
}

Running the Application

运行应用

Development Server

开发服务器

bash

npm run dev

bash

npm run dev

or

或

yarn dev

or

或

pnpm dev


Visit `http://localhost:3000` to access the Next.js interface.

pnpm dev


访问`http://localhost:3000`进入Next.js管理界面。

Build for Production

生产环境构建

bash

npm run build
npm start

bash

npm run build
npm start

Render Videos Only

仅渲染视频

bash

undefined

bash

undefined

If the project has a dedicated video rendering script

如果项目有独立的视频渲染脚本

npm run render

undefined

npm run render

undefined

API Routes (Next.js)

API路由（Next.js）

typescript

// app/api/generate/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { runContentPipeline } from '@/lib/pipeline';

export async function POST(request: NextRequest) {
  const { keyword, format, language } = await request.json();

  try {
    const result = await runContentPipeline(keyword);
    
    return NextResponse.json({
      success: true,
      data: result,
    });
  } catch (error) {
    return NextResponse.json(
      { success: false, error: error.message },
      { status: 500 }
    );
  }
}

typescript

// app/api/generate/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { runContentPipeline } from '@/lib/pipeline';

export async function POST(request: NextRequest) {
  const { keyword, format, language } = await request.json();

  try {
    const result = await runContentPipeline(keyword);
    
    return NextResponse.json({
      success: true,
      data: result,
    });
  } catch (error) {
    return NextResponse.json(
      { success: false, error: error.message },
      { status: 500 }
    );
  }
}

Troubleshooting

故障排查

API Rate Limits

API速率限制

typescript

// Implement retry logic with exponential backoff
async function retryWithBackoff<T>(
  fn: () => Promise<T>,
  maxRetries = 3
): Promise<T> {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      await new Promise(resolve => setTimeout(resolve, Math.pow(2, i) * 1000));
    }
  }
  throw new Error('Max retries exceeded');
}

typescript

// 实现指数退避重试逻辑
async function retryWithBackoff<T>(
  fn: () => Promise<T>,
  maxRetries = 3
): Promise<T> {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      await new Promise(resolve => setTimeout(resolve, Math.pow(2, i) * 1000));
    }
  }
  throw new Error('Max retries exceeded');
}

Video Rendering Memory Issues

视频渲染内存问题

typescript

// Use smaller compositions or split rendering
const composition = await selectComposition({
  serveUrl: bundleLocation,
  id: 'ContentVideo',
  inputProps: {
    title: config.title,
    content: config.content.slice(0, 500), // Limit content length
  },
});

typescript

// 使用更小的合成内容或拆分渲染
const composition = await selectComposition({
  serveUrl: bundleLocation,
  id: 'ContentVideo',
  inputProps: {
    title: config.title,
    content: config.content.slice(0, 500), // 限制内容长度
  },
});

Claude/OpenAI Token Limits

Claude/OpenAI令牌限制

typescript

function truncateContent(text: string, maxTokens = 3000): string {
  // Rough estimate: 1 token ≈ 4 characters
  const maxChars = maxTokens * 4;
  return text.length > maxChars ? text.slice(0, maxChars) : text;
}

typescript

function truncateContent(text: string, maxTokens = 3000): string {
  // 粗略估算：1令牌≈4个字符
  const maxChars = maxTokens * 4;
  return text.length > maxChars ? text.slice(0, maxChars) : text;
}

Environment Variables Reference

环境变量参考

Variable	Required	Description
`ANTHROPIC_API_KEY`	Yes	Claude API key from Anthropic
`OPENAI_API_KEY`	Optional	OpenAI API key (alternative to Claude)
`RAPIDAPI_KEY`	Yes	RapidAPI key for research crawling
`DEFAULT_LANGUAGE`	No	Default content language (en/vi)
`TONE`	No	Default content tone

变量	是否必填	说明
`ANTHROPIC_API_KEY`	是	来自Anthropic的Claude API密钥
`OPENAI_API_KEY`	否	OpenAI API密钥（Claude的替代方案）
`RAPIDAPI_KEY`	是	用于调研爬取的RapidAPI密钥
`DEFAULT_LANGUAGE`	否	默认内容语言（en/vi）
`TONE`	否	默认内容语气风格

Best Practices

最佳实践

Always validate research data before passing to AI generators
Cache research results to avoid redundant API calls
Use environment-specific configs for development vs production
Monitor API usage to stay within rate limits
Test video renders locally before batch processing
Implement proper error logging for production debugging

在将调研数据传入AI生成器前务必验证数据有效性
缓存调研结果，避免重复调用API
为开发和生产环境使用不同的配置
监控API使用情况，避免超出速率限制
批量处理前先在本地测试视频渲染效果
为生产环境实现完善的错误日志记录，便于调试