otel-logging-patterns

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

OpenTelemetry Logging Patterns

OpenTelemetry 日志模式

Table of Contents

目录

Purpose

用途

This skill provides production-grade OpenTelemetry logging patterns for Python applications. It covers:
  • OTEL Logging Architecture: Provider and processor configuration
  • Trace Correlation: Automatic trace context injection into logs
  • Structured Logging: Integration with structlog for context preservation
  • Log Exporters: OTLP, Jaeger, console, and file exporters
  • Error Handling: Comprehensive exception and error logging
  • Testing Patterns: Unit and integration tests for logging infrastructure
  • Performance: Optimization tips to minimize logging overhead
This enables production observability where logs, traces, and metrics are correlated through trace IDs for efficient debugging and monitoring.
本方案为Python应用提供生产级别的OpenTelemetry日志实现模式,涵盖以下内容:
  • OTEL日志架构:日志提供者与处理器配置
  • 追踪关联:自动将追踪上下文注入日志
  • 结构化日志:与structlog集成以保留上下文信息
  • 日志导出器:OTLP、Jaeger、控制台及文件导出器
  • 错误处理:全面的异常与错误日志记录
  • 测试模式:日志基础设施的单元测试与集成测试
  • 性能优化:减少日志开销的优化技巧
通过Trace ID关联日志、追踪与指标数据,实现生产环境的可观测性,从而提升调试与监控效率。

Quick Start

快速开始

For this project, use the authoritative OTEL logging module. Get started in 3 simple steps:
  1. Initialize OTEL at application startup (once):
python
from app.core.monitoring.otel_logger import initialize_otel_logger
针对本项目,使用官方OTEL日志模块,通过3个简单步骤快速上手:
  1. 在应用启动时初始化OTEL(仅需执行一次):
python
from app.core.monitoring.otel_logger import initialize_otel_logger

Call once at app startup

应用启动时调用一次

initialize_otel_logger( log_level="INFO", enable_console=True, enable_otlp=True, otlp_endpoint="localhost:4317" )

2. **Get logger and tracer in each module**:

```python
from app.core.monitoring.otel_logger import logger, get_tracer
initialize_otel_logger( log_level="INFO", enable_console=True, enable_otlp=True, otlp_endpoint="localhost:4317" )

2. **在每个模块中获取日志器与追踪器**:

```python
from app.core.monitoring.otel_logger import logger, get_tracer

Module-level initialization

模块级初始化

logger = logger(name) tracer = get_tracer(name)

3. **Use trace_span for operations**:

```python
from app.core.monitoring.otel_logger import trace_span, logger

logger = logger(__name__)
logger = logger(name) tracer = get_tracer(name)

3. **使用trace_span封装操作**:

```python
from app.core.monitoring.otel_logger import trace_span, logger

logger = logger(__name__)

Logs automatically include trace_id and span_id

日志将自动包含trace_id和span_id

with trace_span("process_order", order_id="12345") as span: logger.info("processing_order", order_id="12345") # Do work... logger.info("order_processed", result_count=5)

That's it! All logs automatically include trace context, and spans are created with attributes.

**Key Benefits**:
- ✅ Single entry point: `app/core/monitoring/otel_logger.py`
- ✅ No direct imports of structlog or opentelemetry needed
- ✅ Automatic trace context propagation
- ✅ Automatic exception handling in spans
- ✅ Works with async and sync functions
with trace_span("process_order", order_id="12345") as span: logger.info("processing_order", order_id="12345") # 执行业务操作... logger.info("order_processed", result_count=5)

完成以上步骤即可!所有日志将自动包含追踪上下文,并且Span会附带属性信息。

**核心优势**:
- ✅ 统一入口:`app/core/monitoring/otel_logger.py`
- ✅ 无需直接导入structlog或opentelemetry
- ✅ 自动追踪上下文传播
- ✅ Span中自动处理异常
- ✅ 支持异步与同步函数

Instructions

操作步骤

Step 1: Configure OTEL Logging Provider

步骤1:配置OTEL日志提供者

Set up the core OpenTelemetry logging infrastructure with provider and exporter configuration.
Basic setup (console exporter for development):
python
undefined
搭建核心OpenTelemetry日志基础设施,配置日志提供者与导出器。
基础配置(开发环境使用控制台导出器):
python
undefined

app/shared/otel_config.py

app/shared/otel_config.py

from opentelemetry import logs from opentelemetry.sdk.logs import LoggerProvider from opentelemetry.sdk.logs.export import ConsoleLogExporter, SimpleLogRecordExporter from opentelemetry.sdk.resources import Resource
def setup_console_logging(service_name: str) -> LoggerProvider: """Set up OTEL logging with console exporter.""" resource = Resource.create({"service.name": service_name}) logger_provider = LoggerProvider(resource=resource)
exporter = ConsoleLogExporter()
processor = SimpleLogRecordExporter(exporter)
logger_provider.add_log_record_processor(processor)
logs.set_logger_provider(logger_provider)

return logger_provider

**For production OTLP/Jaeger setup**, see [references/advanced-patterns.md](references/advanced-patterns.md#otelconfig-class)
from opentelemetry import logs from opentelemetry.sdk.logs import LoggerProvider from opentelemetry.sdk.logs.export import ConsoleLogExporter, SimpleLogRecordExporter from opentelemetry.sdk.resources import Resource
def setup_console_logging(service_name: str) -> LoggerProvider: """设置带有控制台导出器的OTEL日志系统。""" resource = Resource.create({"service.name": service_name}) logger_provider = LoggerProvider(resource=resource)
exporter = ConsoleLogExporter()
processor = SimpleLogRecordExporter(exporter)
logger_provider.add_log_record_processor(processor)
logs.set_logger_provider(logger_provider)

return logger_provider

**生产环境OTLP/Jaeger配置**,请参考[references/advanced-patterns.md](references/advanced-patterns.md#otelconfig-class)

Step 2: Integrate Structured Logging Library

步骤2:集成结构化日志库

Set up structlog with OTEL trace context integration:
python
undefined
设置structlog并与OTEL追踪上下文集成:
python
undefined

app/shared/logging_setup.py

app/shared/logging_setup.py

import structlog from opentelemetry.instrumentation.logging import LoggingInstrumentor
def setup_structlog() -> None: """Configure structlog with OTEL trace context integration.""" # Enable OTEL logging instrumentation LoggingInstrumentor().instrument()
# Configure structlog
structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.ExceptionRenderer(),
        structlog.processors.JSONRenderer(),
    ],
    logger_factory=structlog.logging.LoggerFactory(),
    cache_logger_on_first_use=True,
)
def get_logger(name: str): """Get a logger instance with trace context support.""" return structlog.logger(name)

**For advanced structlog configuration**, see [references/advanced-patterns.md](references/advanced-patterns.md#structlog-configuration)
import structlog from opentelemetry.instrumentation.logging import LoggingInstrumentor
def setup_structlog() -> None: """配置structlog并集成OTEL追踪上下文。""" # 启用OTEL日志埋点 LoggingInstrumentor().instrument()
# 配置structlog
structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.ExceptionRenderer(),
        structlog.processors.JSONRenderer(),
    ],
    logger_factory=structlog.logging.LoggerFactory(),
    cache_logger_on_first_use=True,
)
def get_logger(name: str): """获取支持追踪上下文的日志器实例。""" return structlog.logger(name)

**进阶structlog配置**,请参考[references/advanced-patterns.md](references/advanced-patterns.md#structlog-configuration)

Step 3: Add Trace Context Propagation

步骤3:添加追踪上下文传播

Ensure trace context flows through logs automatically:
python
undefined
确保追踪上下文自动在日志中传递:
python
undefined

app/shared/observability.py

app/shared/observability.py

from contextvars import ContextVar from opentelemetry import trace
from contextvars import ContextVar from opentelemetry import trace

Context variables for request tracking

用于请求追踪的上下文变量

request_id_var: ContextVar[str | None] = ContextVar("request_id", default=None) user_id_var: ContextVar[str | None] = ContextVar("user_id", default=None)
class ObservabilityContext: """Manage observability context (trace IDs, request IDs, user IDs)."""
@staticmethod
def set_request_id(request_id: str) -> None:
    """Set request ID in context."""
    request_id_var.set(request_id)

@staticmethod
def get_tracer(name: str):
    """Get tracer instance."""
    return trace.get_tracer(name)

@staticmethod
def set_span_attribute(key: str, value: any) -> None:
    """Set attribute on current span."""
    span = trace.get_current_span()
    if span.is_recording():
        span.set_attribute(key, value)

**For complete ObservabilityContext class**, see [references/advanced-patterns.md](references/advanced-patterns.md#observability-context)
request_id_var: ContextVar[str | None] = ContextVar("request_id", default=None) user_id_var: ContextVar[str | None] = ContextVar("user_id", default=None)
class ObservabilityContext: """管理可观测性上下文(Trace ID、请求ID、用户ID)。"""
@staticmethod
def set_request_id(request_id: str) -> None:
    """在上下文中设置请求ID。"""
    request_id_var.set(request_id)

@staticmethod
def get_tracer(name: str):
    """获取追踪器实例。"""
    return trace.get_tracer(name)

@staticmethod
def set_span_attribute(key: str, value: any) -> None:
    """为当前Span设置属性。"""
    span = trace.get_current_span()
    if span.is_recording():
        span.set_attribute(key, value)

**完整ObservabilityContext类**,请参考[references/advanced-patterns.md](references/advanced-patterns.md#observability-context)

Step 4: Set Up Log Exporters

步骤4:设置日志导出器

Configure different exporters for different environments:
python
undefined
针对不同环境配置不同的导出器:
python
undefined

app/config.py

app/config.py

from pydantic import Field from pydantic_settings import BaseSettings
class Config(BaseSettings): """Configuration for OTEL logging."""
otel_enabled: bool = Field(default=True)
otel_exporter_type: str = Field(default="console")  # 'otlp', 'jaeger', or 'console'
otel_otlp_endpoint: str = Field(default="localhost:4317")
otel_jaeger_host: str = Field(default="localhost")
otel_jaeger_port: int = Field(default=6831)

Then use config in main:

```python
from pydantic import Field from pydantic_settings import BaseSettings
class Config(BaseSettings): """OTEL日志系统配置。"""
otel_enabled: bool = Field(default=True)
otel_exporter_type: str = Field(default="console")  # 'otlp', 'jaeger', 或 'console'
otel_otlp_endpoint: str = Field(default="localhost:4317")
otel_jaeger_host: str = Field(default="localhost")
otel_jaeger_port: int = Field(default=6831)

然后在主程序中使用配置:

```python

main.py

main.py

from app.config import Config from app.shared.logging_setup import setup_structlog, get_logger from app.shared.otel_config import OTELConfig
async def main() -> None: """Main entry point.""" config = Config() setup_structlog()
if config.otel_enabled:
    otel_config = OTELConfig(
        service_name="my-service",
        exporter_type=config.otel_exporter_type,
        otlp_endpoint=config.otel_otlp_endpoint,
    )
    otel_config.setup_logging()
    otel_config.setup_tracing()

logger = get_logger(__name__)
logger.info("service_started")

**For complete exporter configuration**, see [references/advanced-patterns.md](references/advanced-patterns.md#exporter-configuration)
from app.config import Config from app.shared.logging_setup import setup_structlog, get_logger from app.shared.otel_config import OTELConfig
async def main() -> None: """主入口函数。""" config = Config() setup_structlog()
if config.otel_enabled:
    otel_config = OTELConfig(
        service_name="my-service",
        exporter_type=config.otel_exporter_type,
        otlp_endpoint=config.otel_otlp_endpoint,
    )
    otel_config.setup_logging()
    otel_config.setup_tracing()

logger = get_logger(__name__)
logger.info("service_started")

**完整导出器配置**,请参考[references/advanced-patterns.md](references/advanced-patterns.md#exporter-configuration)

Step 5: Implement Instrumentation

步骤5:实现埋点

Add tracing and logging to key application flows:
python
undefined
为关键业务流程添加追踪与日志:
python
undefined

app/use_cases/extract_orders.py

app/use_cases/extract_orders.py

from opentelemetry import trace from app.shared.logging_setup import get_logger
class ExtractOrdersUseCase: """Use case with observability."""
def __init__(self, gateway, publisher) -> None:
    self.gateway = gateway
    self.publisher = publisher
    self.logger = get_logger(__name__)
    self.tracer = trace.get_tracer(__name__)

async def execute(self) -> int:
    """Execute with tracing."""
    with self.tracer.start_as_current_span("extract_orders") as span:
        self.logger.info("starting_extraction")

        try:
            orders = await self.gateway.fetch_all_orders()
            self.logger.info("orders_fetched", count=len(orders))

            for order in orders:
                await self.publisher.publish(order)

            span.set_attribute("orders_processed", len(orders))
            self.logger.info("extraction_completed", total=len(orders))
            return len(orders)

        except Exception as e:
            self.logger.error("extraction_failed", error=str(e))
            span.record_exception(e)
            raise

**For advanced instrumentation patterns**, see [examples/examples.md](examples/examples.md#fastapi-instrumentation)
from opentelemetry import trace from app.shared.logging_setup import get_logger
class ExtractOrdersUseCase: """带有可观测性的用例。"""
def __init__(self, gateway, publisher) -> None:
    self.gateway = gateway
    self.publisher = publisher
    self.logger = get_logger(__name__)
    self.tracer = trace.get_tracer(__name__)

async def execute(self) -> int:
    """通过追踪执行用例。"""
    with self.tracer.start_as_current_span("extract_orders") as span:
        self.logger.info("starting_extraction")

        try:
            orders = await self.gateway.fetch_all_orders()
            self.logger.info("orders_fetched", count=len(orders))

            for order in orders:
                await self.publisher.publish(order)

            span.set_attribute("orders_processed", len(orders))
            self.logger.info("extraction_completed", total=len(orders))
            return len(orders)

        except Exception as e:
            self.logger.error("extraction_failed", error=str(e))
            span.record_exception(e)
            raise

**进阶埋点模式**,请参考[examples/examples.md](examples/examples.md#fastapi-instrumentation)

Step 6: Add Error and Exception Logging

步骤6:添加错误与异常日志

Implement comprehensive error logging with context:
python
undefined
实现包含上下文信息的全面错误日志:
python
undefined

app/error_handlers.py

app/error_handlers.py

from fastapi import FastAPI, Request from fastapi.responses import JSONResponse from app.shared.logging_setup import get_logger from app.shared.observability import ObservabilityContext
logger = get_logger(name)
async def global_exception_handler(request: Request, exc: Exception) -> JSONResponse: """Global exception handler with OTEL logging.""" request_id = ObservabilityContext.get_request_id()
logger.error(
    "unhandled_exception",
    error=str(exc),
    error_type=type(exc).__name__,
    request_id=request_id,
    path=request.url.path,
    method=request.method,
    exc_info=True,  # Include full stack trace
)

return JSONResponse(
    status_code=500,
    content={"detail": "Internal server error", "request_id": request_id},
)
def setup_error_handlers(app: FastAPI) -> None: """Register error handlers.""" app.add_exception_handler(Exception, global_exception_handler)

**For complete error handling patterns**, see [examples/examples.md](examples/examples.md#error-handling)
from fastapi import FastAPI, Request from fastapi.responses import JSONResponse from app.shared.logging_setup import get_logger from app.shared.observability import ObservabilityContext
logger = get_logger(name)
async def global_exception_handler(request: Request, exc: Exception) -> JSONResponse: """带有OTEL日志的全局异常处理器。""" request_id = ObservabilityContext.get_request_id()
logger.error(
    "unhandled_exception",
    error=str(exc),
    error_type=type(exc).__name__,
    request_id=request_id,
    path=request.url.path,
    method=request.method,
    exc_info=True,  # 包含完整堆栈跟踪
)

return JSONResponse(
    status_code=500,
    content={"detail": "内部服务器错误", "request_id": request_id},
)
def setup_error_handlers(app: FastAPI) -> None: """注册错误处理器。""" app.add_exception_handler(Exception, global_exception_handler)

**完整错误处理模式**,请参考[examples/examples.md](examples/examples.md#error-handling)

Requirements

依赖要求

Production OTEL logging requires:
  • opentelemetry-api>=1.22.0
    - OTEL API
  • opentelemetry-sdk>=1.22.0
    - OTEL SDK with logging support
  • opentelemetry-exporter-otlp>=0.43b0
    - OTLP exporter for gRPC
  • opentelemetry-instrumentation-logging>=0.43b0
    - Logging instrumentation
  • structlog>=23.2.0
    - Structured logging library
  • pydantic>=2.5.0
    - Configuration management
  • Python 3.11+ with type checking
生产环境OTEL日志系统需要以下依赖:
  • opentelemetry-api>=1.22.0
    - OTEL API
  • opentelemetry-sdk>=1.22.0
    - 带日志支持的OTEL SDK
  • opentelemetry-exporter-otlp>=0.43b0
    - gRPC协议的OTLP导出器
  • opentelemetry-instrumentation-logging>=0.43b0
    - 日志埋点工具
  • structlog>=23.2.0
    - 结构化日志库
  • pydantic>=2.5.0
    - 配置管理工具
  • Python 3.11+ 并支持类型检查

Common Patterns

通用模式

Trace Context Propagation

追踪上下文传播

Logs automatically include trace_id and span_id when logged within a span:
python
from opentelemetry import trace
from app.shared.logging_setup import get_logger

logger = get_logger(__name__)
tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("process_payment"):
    logger.info("payment_started", customer_id="cust-123")
    logger.info("payment_completed", amount=99.99)
    # All logs have same trace_id and span_id
在Span范围内记录的日志将自动包含trace_id和span_id:
python
from opentelemetry import trace
from app.shared.logging_setup import get_logger

logger = get_logger(__name__)
tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("process_payment"):
    logger.info("payment_started", customer_id="cust-123")
    logger.info("payment_completed", amount=99.99)
    # 所有日志将包含相同的trace_id和span_id

Context Variables

上下文变量

Use context variables for request-scoped data:
python
from app.shared.observability import ObservabilityContext

ObservabilityContext.set_request_id("req-12345")
ObservabilityContext.set_user_id("user-456")

logger.info("user_action", action="login")
使用上下文变量存储请求范围的数据:
python
from app.shared.observability import ObservabilityContext

ObservabilityContext.set_request_id("req-12345")
ObservabilityContext.set_user_id("user-456")

logger.info("user_action", action="login")

Log includes request_id and user_id automatically

日志将自动包含request_id和user_id


**For complete pattern examples**, see [examples/examples.md](examples/examples.md)

**完整模式示例**,请参考[examples/examples.md](examples/examples.md)

Testing OTEL Logging

OTEL日志测试

Unit Testing

单元测试

Test logging configuration without external dependencies:
python
import pytest
from opentelemetry import logs
from opentelemetry.sdk.logs import LoggerProvider

def test_logging_configuration():
    """Test OTEL logging provider setup."""
    logger_provider = LoggerProvider()
    assert logger_provider is not None
    logs.set_logger_provider(logger_provider)
    assert logs.get_logger_provider() == logger_provider
在无外部依赖的情况下测试日志配置:
python
import pytest
from opentelemetry import logs
from opentelemetry.sdk.logs import LoggerProvider

def test_logging_configuration():
    """测试OTEL日志提供者的设置。"""
    logger_provider = LoggerProvider()
    assert logger_provider is not None
    logs.set_logger_provider(logger_provider)
    assert logs.get_logger_provider() == logger_provider

Integration Testing

集成测试

Test log export with real exporters:
python
@pytest.fixture
def in_memory_log_exporter():
    """In-memory exporter for testing."""
    class InMemoryExporter:
        def __init__(self):
            self.records = []

        def emit(self, log_records):
            self.records.extend(log_records)

    return InMemoryExporter()

def test_trace_context_in_logs(in_memory_log_exporter):
    """Verify trace context is included in logs."""
    # Setup logger and tracer
    # Log within span
    # Verify trace context in exported logs
    pass
For comprehensive testing patterns, see examples/examples.md
使用真实导出器测试日志导出:
python
@pytest.fixture
def in_memory_log_exporter():
    """用于测试的内存导出器。"""
    class InMemoryExporter:
        def __init__(self):
            self.records = []

        def emit(self, log_records):
            self.records.extend(log_records)

    return InMemoryExporter()

def test_trace_context_in_logs(in_memory_log_exporter):
    """验证日志中包含追踪上下文。"""
    # 配置日志器与追踪器
    # 在Span范围内记录日志
    # 验证导出的日志中包含追踪上下文
    pass
全面测试模式,请参考examples/examples.md

Troubleshooting

故障排查

Common Issues and Solutions

常见问题与解决方案

Issue: Trace IDs not appearing in logs
  • Ensure
    LoggingInstrumentor().instrument()
    is called before logging
  • Verify logs are being created within a span context
  • Check that OTEL trace provider is set before creating loggers
Issue: OTLP exporter connection refused
  • Verify OTEL Collector is running on specified endpoint
  • Check endpoint configuration (default: localhost:4317)
  • Use console exporter in development
  • Check firewall rules for gRPC port 4317
Issue: High logging overhead
  • Use
    BatchLogRecordExporter
    instead of
    SimpleLogRecordExporter
  • Increase batch size for fewer exports (e.g., 512 records)
  • Filter verbose DEBUG logs in production
Issue: Logs not appearing in output
  • Verify logger is initialized with
    setup_structlog()
  • Check Python logging level (INFO, WARNING, ERROR)
  • Verify structlog configuration has JSONRenderer processor
For detailed troubleshooting, see references/advanced-patterns.md
问题:Trace ID未出现在日志中
  • 确保在记录日志前调用了
    LoggingInstrumentor().instrument()
  • 验证日志是在Span上下文内创建的
  • 检查OTEL追踪提供者是否在创建日志器之前已设置
问题:OTLP导出器连接被拒绝
  • 验证OTEL Collector是否在指定端点运行
  • 检查端点配置(默认:localhost:4317)
  • 开发环境使用控制台导出器
  • 检查gRPC端口4317的防火墙规则
问题:日志开销过高
  • 使用
    BatchLogRecordExporter
    替代
    SimpleLogRecordExporter
  • 增加批量大小以减少导出次数(例如:512条记录)
  • 生产环境过滤冗余的DEBUG日志
问题:日志未显示在输出中
  • 验证已通过
    setup_structlog()
    初始化日志器
  • 检查Python日志级别(INFO、WARNING、ERROR)
  • 验证structlog配置包含JSONRenderer处理器
详细故障排查指南,请参考references/advanced-patterns.md

Supporting Resources

参考资源

ResourcePurpose
examples/examples.md
10+ complete examples covering FastAPI, Kafka, background jobs, testing
references/advanced-patterns.md
Performance tuning, custom exporters, trace propagation details
资源用途
examples/examples.md
10+个完整示例,涵盖FastAPI、Kafka、后台任务、测试等场景
references/advanced-patterns.md
性能调优、自定义导出器、追踪传播细节