storage-development

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Storage Development Guide

存储开发指南

This skill provides guidance for developers working with the

@rytass/storages

base package, including creating new storage adapters.

本指南为使用

@rytass/storages

基础包的开发者提供指导，包括如何创建新的存储adapter。

Overview

概述

The

@rytass/storages

package defines the core interfaces and types that all storage adapters must implement. It follows the adapter pattern to provide a unified API across different storage providers.

Package:

@rytass/storages

(v0.2.5)

Adapters built on this base:

```
@rytass/storages-adapter-s3
```
- AWS S3
```
@rytass/storages-adapter-gcs
```
- Google Cloud Storage
```
@rytass/storages-adapter-r2
```
- Cloudflare R2
```
@rytass/storages-adapter-azure-blob
```
- Azure Blob Storage
```
@rytass/storages-adapter-local
```
- Local File System

@rytass/storages

包定义了所有存储adapter必须实现的核心接口和类型。它遵循适配器模式，为不同的存储提供商提供统一的API。

包：

@rytass/storages

（v0.2.5）

基于此基础构建的Adapter：

```
@rytass/storages-adapter-s3
```
- AWS S3
```
@rytass/storages-adapter-gcs
```
- Google Cloud Storage
```
@rytass/storages-adapter-r2
```
- Cloudflare R2
```
@rytass/storages-adapter-azure-blob
```
- Azure Blob Storage
```
@rytass/storages-adapter-local
```
- 本地文件系统

Architecture

架构

@rytass/storages (Base Package)
    │
    ├── StorageInterface              # Core interface all adapters must implement
    ├── Storage<O>                    # Base class with helper methods (not abstract)
    ├── ConverterManager              # File converter pipeline system (from @rytass/file-converter)
    ├── Types & Interfaces            # Shared type definitions
    └── Error Handling                # StorageError, ErrorCode enums

@rytass/storages-adapter-*           # Provider implementations
    │
    ├── [Provider]Storage             # Extends Storage<ProviderOptions>
    ├── typings.ts                    # Provider-specific option types
    └── index.ts                      # Package exports

@rytass/storages (Base Package)
    │
    ├── StorageInterface              # Core interface all adapters must implement
    ├── Storage<O>                    # Base class with helper methods (not abstract)
    ├── ConverterManager              # File converter pipeline system (from @rytass/file-converter)
    ├── Types & Interfaces            # Shared type definitions
    └── Error Handling                # StorageError, ErrorCode enums

@rytass/storages-adapter-*           # Provider implementations
    │
    ├── [Provider]Storage             # Extends Storage<ProviderOptions>
    ├── typings.ts                    # Provider-specific option types
    └── index.ts                      # Package exports

Core Concepts

核心概念

InputFile:
```
Buffer | Readable
```
- Files to upload can be either in-memory buffers or streams
StorageFile:
```
{ key: string }
```
- Uploaded file reference containing the storage key/path
Hash-Based Naming: Automatically generate unique filenames using SHA1 or SHA256 hash of file content
File Converters: Transform or process files during upload (e.g., image resizing, watermarking)
Unified Interface: All adapters implement the same methods for seamless provider switching

InputFile：
```
Buffer | Readable
```
- 待上传的文件可以是内存缓冲区或流
StorageFile：
```
{ key: string }
```
- 已上传文件的引用，包含存储键/路径
基于哈希的命名：通过文件内容的SHA1或SHA256哈希自动生成唯一文件名
文件转换器：在上传过程中转换或处理文件（例如图片缩放、添加水印）
统一接口：所有adapter实现相同的方法，实现存储提供商的无缝切换

Installation

安装

bash

npm install @rytass/storages

bash

npm install @rytass/storages

Quick Reference

快速参考

Core Interfaces

核心接口

StorageInterface
- Core interface defining the storage contract:

typescript

interface StorageInterface {
  // Upload operations (Note: options are NOT part of the interface)
  write(file: InputFile): Promise<StorageFile>;
  batchWrite(files: InputFile[]): Promise<StorageFile[]>;

  // Download operations
  read(key: string): Promise<Readable>;
  read(key: string, options: ReadBufferFileOptions): Promise<Buffer>;
  read(key: string, options: ReadStreamFileOptions): Promise<Readable>;

  // File management
  remove(key: string): Promise<void>;
  // Note: isExists() is NOT part of the interface, it's in Storage class
}

Storage<O>
Class - Base implementation with helper methods (not abstract, uses throw to force override):

typescript

class Storage<O extends Record<string, unknown> = Record<string, unknown>>
  implements StorageInterface {
  // Provided by base class
  readonly converterManager: ConverterManager;
  readonly hashAlgorithm: FilenameHashAlgorithm;

  constructor(options?: StorageOptions<O>);

  // File type detection helpers
  getExtension(file: InputFile): Promise<FileTypeResult | undefined>;
  getBufferFilename(buffer: Buffer): Promise<[string, string | undefined]>;
  getStreamFilename(stream: Readable): Promise<[string, string | undefined]>;

  // Methods to override (throw Error by default, subclasses must override)
  write(file: InputFile, options?: WriteFileOptions): Promise<StorageFile>;
  batchWrite(files: InputFile[], options?: WriteFileOptions[]): Promise<StorageFile[]>;
  read(key: string): Promise<Readable>;
  read(key: string, options: ReadBufferFileOptions): Promise<Buffer>;
  read(key: string, options: ReadStreamFileOptions): Promise<Readable>;
  remove(key: string): Promise<void>;

  // Additional method to override (NOT in StorageInterface)
  isExists(key: string): Promise<boolean>;
}

Note: The
Storage
class is NOT abstract. Instead, methods throw
Error('Method not implemented.')
by default, requiring subclasses to override them. The
write
and
batchWrite
methods accept
options
parameter in the implementation but NOT in
StorageInterface
.

StorageInterface
- 定义存储契约的核心接口：

typescript

interface StorageInterface {
  // Upload operations (Note: options are NOT part of the interface)
  write(file: InputFile): Promise<StorageFile>;
  batchWrite(files: InputFile[]): Promise<StorageFile[]>;

  // Download operations
  read(key: string): Promise<Readable>;
  read(key: string, options: ReadBufferFileOptions): Promise<Buffer>;
  read(key: string, options: ReadStreamFileOptions): Promise<Readable>;

  // File management
  remove(key: string): Promise<void>;
  // Note: isExists() is NOT part of the interface, it's in Storage class
}

Storage<O>
Class - 包含辅助方法的基础类（非抽象类）：

typescript

class Storage<O extends Record<string, unknown> = Record<string, unknown>>
  implements StorageInterface {
  // Provided by base class
  readonly converterManager: ConverterManager;
  readonly hashAlgorithm: FilenameHashAlgorithm;

  constructor(options?: StorageOptions<O>);

  // File type detection helpers
  getExtension(file: InputFile): Promise<FileTypeResult | undefined>;
  getBufferFilename(buffer: Buffer): Promise<[string, string | undefined]>;
  getStreamFilename(stream: Readable): Promise<[string, string | undefined]>;

  // Methods to override (throw Error by default, subclasses must override)
  write(file: InputFile, options?: WriteFileOptions): Promise<StorageFile>;
  batchWrite(files: InputFile[], options?: WriteFileOptions[]): Promise<StorageFile[]>;
  read(key: string): Promise<Readable>;
  read(key: string, options: ReadBufferFileOptions): Promise<Buffer>;
  read(key: string, options: ReadStreamFileOptions): Promise<Readable>;
  remove(key: string): Promise<void>;

  // Additional method to override (NOT in StorageInterface)
  isExists(key: string): Promise<boolean>;
}

注意：
Storage
类并非抽象类。默认情况下，其方法会抛出
Error('Method not implemented.')
，要求子类重写这些方法。
write
和
batchWrite
方法在实现中支持
options
参数，但该参数并未包含在
StorageInterface
中。

Must Implement

必须实现的内容

All adapters extending

Storage<O>

MUST override these methods (not abstract, but throw by default):

Method	Source	Description
`write(file, options?)`	Storage class (options not in interface)	Upload a single file and return storage key
`batchWrite(files, options?)`	Storage class (options not in interface)	Upload multiple files in parallel
`read(key, options?)`	StorageInterface	Download file as Buffer or Stream
`remove(key)`	StorageInterface	Delete a file
`isExists(key)`	Storage class only	Check if file exists (not in interface)

Note:
isExists()
is defined in the
Storage
class but NOT in
StorageInterface
. This means adapters must implement it, but code depending only on
StorageInterface
cannot assume it exists. Similarly, the
options
parameter for
write
and
batchWrite
is only in the
Storage
class implementation.

所有继承

Storage<O>

的adapter必须重写以下方法（这些方法并非抽象方法，但默认会抛出异常）：

方法	来源	描述
`write(file, options?)`	Storage类（options未在接口中定义）	上传单个文件并返回存储键
`batchWrite(files, options?)`	Storage类（options未在接口中定义）	并行上传多个文件
`read(key, options?)`	StorageInterface	将文件下载为Buffer或Stream
`remove(key)`	StorageInterface	删除文件
`isExists(key)`	仅Storage类	检查文件是否存在（未在接口中定义）

注意：
isExists()
定义在
Storage
类中，但并未包含在
StorageInterface
里。这意味着adapter必须实现该方法，但仅依赖
StorageInterface
的代码无法假定该方法存在。同样，
write
和
batchWrite
的
options
参数仅在Storage类的实现中存在。

Optional Features

可选功能

Adapters MAY implement these additional methods:

Method	Description	Example
`url(key, options?)`	Generate presigned/signed URL for temporary access	Cloud adapters (S3, GCS, R2, Azure)
Custom helpers	Provider-specific utilities	`getUsageInfo()` in Local adapter

Adapter可以实现以下额外方法：

方法	描述	示例
`url(key, options?)`	生成预签名/签名URL以提供临时访问权限	云adapter（S3、GCS、R2、Azure）
自定义辅助方法	提供商特定的工具函数	本地adapter中的 `getUsageInfo()`

Common Types

通用类型

typescript

// Input/Output Types (from @rytass/file-converter)
type ConvertableFile = Readable | Buffer;
type InputFile = ConvertableFile;  // Re-exported alias

type FileKey = string;
interface StorageFile {
  readonly key: FileKey;
}

// Options Types
interface StorageOptions<O extends Record<string, unknown>> {
  converters?: FileConverter<O>[];
  hashAlgorithm?: 'sha1' | 'sha256';
}

interface WriteFileOptions {
  filename?: string;      // Custom filename (overrides hash-based generation)
  contentType?: string;   // MIME type for the file
}

// Read Format Options
interface ReadBufferFileOptions {
  format: 'buffer';
}

interface ReadStreamFileOptions {
  format: 'stream';
}

typescript

// Input/Output Types (from @rytass/file-converter)
type ConvertableFile = Readable | Buffer;
type InputFile = ConvertableFile;  // Re-exported alias

type FileKey = string;
interface StorageFile {
  readonly key: FileKey;
}

// Options Types
interface StorageOptions<O extends Record<string, unknown>> {
  converters?: FileConverter<O>[];
  hashAlgorithm?: 'sha1' | 'sha256';
}

interface WriteFileOptions {
  filename?: string;      // Custom filename (overrides hash-based generation)
  contentType?: string;   // MIME type for the file
}

// Read Format Options
interface ReadBufferFileOptions {
  format: 'buffer';
}

interface ReadStreamFileOptions {
  format: 'stream';
}

Error Codes

错误码

typescript

enum ErrorCode {
  WRITE_FILE_ERROR = '101',     // Failed to upload file
  READ_FILE_ERROR = '102',      // Failed to download file
  REMOVE_FILE_ERROR = '103',    // Failed to delete file
  UNRECOGNIZED_ERROR = '104',   // Unknown error
  DIRECTORY_NOT_FOUND = '201',  // Directory doesn't exist (Local adapter)
  FILE_NOT_FOUND = '202',       // File doesn't exist
}

typescript

enum ErrorCode {
  WRITE_FILE_ERROR = '101',     // Failed to upload file
  READ_FILE_ERROR = '102',      // Failed to download file
  REMOVE_FILE_ERROR = '103',    // Failed to delete file
  UNRECOGNIZED_ERROR = '104',   // Unknown error
  DIRECTORY_NOT_FOUND = '201',  // Directory doesn't exist (Local adapter)
  FILE_NOT_FOUND = '202',       // File doesn't exist
}

Key Responsibilities

主要职责

When implementing a new storage adapter, you are responsible for:

Extending
Storage<YourOptions>
- Inherit from the base class
Defining Configuration Interface - Specify required and optional settings
Overriding Required Methods - Override methods that throw by default
Handling Buffers and Streams - Support both input formats
Using Hash-Based Filenames - Leverage
```
getBufferFilename()
```
/
```
getStreamFilename()
```
Integrating File Converters - Apply
```
converterManager.convert()
```
before upload
Throwing Appropriate Errors - Use
```
StorageError
```
with correct
```
ErrorCode
```
Writing Tests - Ensure reliability and correctness

在实现新的存储adapter时，你需要负责以下内容：

继承
Storage<YourOptions>
- 从基础类继承
定义配置接口 - 指定必填和可选配置项
重写必需方法 - 重写默认会抛出异常的方法
处理缓冲区和流 - 支持两种输入格式
使用基于哈希的文件名 - 利用
```
getBufferFilename()
```
/
```
getStreamFilename()
```
方法
集成文件转换器 - 在上传前调用
```
converterManager.convert()
```
抛出合适的错误 - 使用带有正确
```
ErrorCode
```
的
```
StorageError
```
编写测试 - 确保可靠性和正确性

File Converter System

文件转换器系统

The base package includes a converter system for processing files during upload:

typescript

// From @rytass/file-converter
type ConvertableFile = Readable | Buffer;

interface FileConverter<O = Record<string, unknown>> {
  convert<Buffer>(file: ConvertableFile): Promise<Buffer>;
  convert<Readable>(file: ConvertableFile): Promise<Readable>;
}

class ConverterManager {
  constructor(converters: FileConverter[]);
  convert<ConvertableFileFormat extends ConvertableFile>(file: ConvertableFile): Promise<ConvertableFileFormat>;
}

// Usage in adapter
const convertedFile = await this.converterManager.convert(inputFile);

Example converters:

Image resizing
Image watermarking
Format transcoding
Compression

Converters are executed in sequence before the file is uploaded to the storage provider.

基础包包含一个转换器系统，用于在上传过程中处理文件：

typescript

// From @rytass/file-converter
type ConvertableFile = Readable | Buffer;

interface FileConverter<O = Record<string, unknown>> {
  convert<Buffer>(file: ConvertableFile): Promise<Buffer>;
  convert<Readable>(file: ConvertableFile): Promise<Readable>;
}

class ConverterManager {
  constructor(converters: FileConverter[]);
  convert<ConvertableFileFormat extends ConvertableFile>(file: ConvertableFile): Promise<ConvertableFileFormat>;
}

// Usage in adapter
const convertedFile = await this.converterManager.convert(inputFile);

转换器示例：

图片缩放
图片添加水印
格式转码
压缩

转换器会在文件上传到存储提供商之前按顺序执行。

Detailed Documentation

详细文档

For complete interface specifications and step-by-step implementation guide:

Base Interfaces Reference - Complete type definitions and interface specifications
Creating an Adapter - Step-by-step guide to implementing a new storage adapter

如需完整的接口规范和分步实现指南：

基础接口参考 - 完整的类型定义和接口规范
创建Adapter - 实现新存储adapter的分步指南