axiom-realitykit
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseRealityKit Development Guide
RealityKit 开发指南
Purpose: Build 3D content, AR experiences, and spatial computing apps using RealityKit's Entity-Component-System architecture
iOS Version: iOS 13+ (base), iOS 18+ (RealityView on iOS), visionOS 1.0+
Xcode: Xcode 15+
用途:使用RealityKit的实体-组件-系统(ECS)架构构建3D内容、AR体验和空间计算应用
iOS版本:iOS 13+(基础功能),iOS 18+(iOS端支持RealityView),visionOS 1.0+
Xcode:Xcode 15+
When to Use This Skill
何时使用该技能
Use this skill when:
- Building any 3D experience (AR, games, visualization, spatial computing)
- Creating SwiftUI apps with 3D content (RealityView, Model3D)
- Implementing AR with anchors (world, image, face, body tracking)
- Working with Entity-Component-System (ECS) architecture
- Setting up physics, collisions, or spatial interactions
- Building multiplayer or shared AR experiences
- Migrating from SceneKit to RealityKit
- Targeting visionOS
Do NOT use this skill for:
- SceneKit maintenance (use )
axiom-scenekit - 2D games (use )
axiom-spritekit - Metal shader programming (use )
axiom-metal-migration-ref - Pure GPU compute (use Metal directly)
在以下场景中使用本技能:
- 构建任何3D体验(AR、游戏、可视化、空间计算)
- 创建包含3D内容的SwiftUI应用(RealityView、Model3D)
- 实现带锚点的AR功能(平面、图像、面部、身体追踪)
- 基于实体-组件-系统(ECS)架构进行开发
- 设置物理系统、碰撞检测或空间交互
- 构建多人协作或共享AR体验
- 从SceneKit迁移到RealityKit
- 开发visionOS应用
请勿在以下场景使用本技能:
- SceneKit维护(使用)
axiom-scenekit - 2D游戏开发(使用)
axiom-spritekit - Metal着色器编程(使用)
axiom-metal-migration-ref - 纯GPU计算(直接使用Metal)
1. Mental Model: ECS vs Scene Graph
1. 思维模型:ECS vs 场景图
Scene Graph (SceneKit)
场景图(SceneKit)
In SceneKit, nodes own their properties. A node IS a renderable, collidable, animated thing.
在SceneKit中,节点拥有自身属性。一个节点就是可渲染、可碰撞、可动画的对象。
Entity-Component-System (RealityKit)
实体-组件-系统(ECS,RealityKit)
In RealityKit, entities are empty containers. Components add data. Systems process that data.
Entity (identity + hierarchy)
├── TransformComponent (position, rotation, scale)
├── ModelComponent (mesh + materials)
├── CollisionComponent (collision shapes)
├── PhysicsBodyComponent (mass, mode)
└── [YourCustomComponent] (game-specific data)
System (processes entities with specific components each frame)Why ECS matters:
- Composition over inheritance: Combine any components on any entity
- Data-oriented: Systems process arrays of components efficiently
- Decoupled logic: Systems don't know about each other
- Testable: Components are pure data, Systems are pure logic
在RealityKit中,实体是空容器。组件添加数据,系统处理数据。
Entity (标识 + 层级)
├── TransformComponent (位置、旋转、缩放)
├── ModelComponent (网格 + 材质)
├── CollisionComponent (碰撞形状)
├── PhysicsBodyComponent (质量、模式)
└── [YourCustomComponent] (游戏专属数据)
System (每帧处理带有特定组件的实体)ECS的重要性:
- 组合优于继承:可在任意实体上组合任意组件
- 面向数据:系统高效处理组件数组
- 逻辑解耦:系统之间互不依赖
- 可测试性:组件是纯数据,系统是纯逻辑
The ECS Mental Shift
ECS的思维转变
| Scene Graph Thinking | ECS Thinking |
|---|---|
| "The player node moves" | "The movement system processes entities with MovementComponent" |
| "Add a method to the node subclass" | "Add a component, create a system" |
"Override | "Register a System that queries for components" |
| "The node knows its health" | "HealthComponent holds data, DamageSystem processes it" |
| 场景图思维 | ECS思维 |
|---|---|
| "玩家节点移动" | "移动系统处理带有MovementComponent的实体" |
| "为节点子类添加方法" | "添加组件,创建系统" |
"在节点中重写 | "注册一个查询组件的系统" |
| "节点知晓自身生命值" | "HealthComponent存储数据,DamageSystem处理逻辑" |
2. Entity Hierarchy
2. 实体层级
Creating Entities
创建实体
swift
// Empty entity
let entity = Entity()
entity.name = "player"
// Entity with components
let entity = Entity()
entity.components[ModelComponent.self] = ModelComponent(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: false)]
)
// ModelEntity convenience (has ModelComponent built in)
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)swift
// 空实体
let entity = Entity()
entity.name = "player"
// 带组件的实体
let entity = Entity()
entity.components[ModelComponent.self] = ModelComponent(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: false)]
)
// ModelEntity便捷方式(内置ModelComponent)
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)Hierarchy Management
层级管理
swift
// Parent-child
parent.addChild(child)
child.removeFromParent()
// Find entities
let found = root.findEntity(named: "player")
// Enumerate
for child in entity.children {
// Process children
}
// Clone
let clone = entity.clone(recursive: true)swift
// 父子关系
parent.addChild(child)
child.removeFromParent()
// 查找实体
let found = root.findEntity(named: "player")
// 遍历子实体
for child in entity.children {
// 处理子实体
}
// 克隆实体
let clone = entity.clone(recursive: true)Transform
变换
swift
// Local transform (relative to parent)
entity.position = SIMD3<Float>(0, 1, 0)
entity.orientation = simd_quatf(angle: .pi / 4, axis: SIMD3(0, 1, 0))
entity.scale = SIMD3<Float>(repeating: 2.0)
// World-space queries
let worldPos = entity.position(relativeTo: nil)
let worldTransform = entity.transform(relativeTo: nil)
// Set world-space transform
entity.setPosition(SIMD3(1, 0, 0), relativeTo: nil)
// Look at a point
entity.look(at: targetPosition, from: entity.position, relativeTo: nil)swift
// 局部变换(相对于父实体)
entity.position = SIMD3<Float>(0, 1, 0)
entity.orientation = simd_quatf(angle: .pi / 4, axis: SIMD3(0, 1, 0))
entity.scale = SIMD3<Float>(repeating: 2.0)
// 世界空间查询
let worldPos = entity.position(relativeTo: nil)
let worldTransform = entity.transform(relativeTo: nil)
// 设置世界空间变换
entity.setPosition(SIMD3(1, 0, 0), relativeTo: nil)
// 看向目标点
entity.look(at: targetPosition, from: entity.position, relativeTo: nil)3. Components
3. 组件
Built-in Components
内置组件
| Component | Purpose |
|---|---|
| Position, rotation, scale |
| Mesh geometry + materials |
| Collision shapes for physics and interaction |
| Mass, physics mode (dynamic/static/kinematic) |
| Linear and angular velocity |
| AR anchor attachment |
| Multiplayer sync |
| Camera settings |
| Directional light |
| Point light |
| Spot light |
| Character physics controller |
| Audio mixing |
| 3D positional audio |
| Non-positional audio |
| Multi-channel audio |
| Entity transparency |
| Contact shadow |
| Gesture input (visionOS) |
| Hover highlight (visionOS) |
| VoiceOver support |
| 组件 | 用途 |
|---|---|
| 位置、旋转、缩放 |
| 网格几何 + 材质 |
| 用于物理系统和交互的碰撞形状 |
| 质量、物理模式(动态/静态/运动学) |
| 线速度和角速度 |
| AR锚点附着 |
| 多人同步 |
| 相机设置 |
| 平行光 |
| 点光源 |
| 聚光灯 |
| 角色物理控制器 |
| 音频混合 |
| 3D位置音频 |
| 非位置音频 |
| 多通道音频 |
| 实体透明度 |
| 接触阴影 |
| 手势输入(visionOS) |
| 悬停高亮(visionOS) |
| VoiceOver支持 |
Custom Components
自定义组件
swift
struct HealthComponent: Component {
var current: Int
var maximum: Int
var percentage: Float {
Float(current) / Float(maximum)
}
}
// Register before use (typically in app init)
HealthComponent.registerComponent()
// Attach to entity
entity.components[HealthComponent.self] = HealthComponent(current: 100, maximum: 100)
// Read
if let health = entity.components[HealthComponent.self] {
print(health.current)
}
// Modify
entity.components[HealthComponent.self]?.current -= 10swift
struct HealthComponent: Component {
var current: Int
var maximum: Int
var percentage: Float {
Float(current) / Float(maximum)
}
}
// 使用前注册(通常在应用初始化时)
HealthComponent.registerComponent()
// 附加到实体
entity.components[HealthComponent.self] = HealthComponent(current: 100, maximum: 100)
// 读取组件
if let health = entity.components[HealthComponent.self] {
print(health.current)
}
// 修改组件
entity.components[HealthComponent.self]?.current -= 10Component Lifecycle
组件生命周期
Components are value types (structs). When you read a component, modify it, and write it back, you're replacing the entire component:
swift
// Read-modify-write pattern
var health = entity.components[HealthComponent.self]!
health.current -= damage
entity.components[HealthComponent.self] = healthAnti-pattern: Holding a reference to a component and expecting mutations to propagate. Components are copied on read.
组件是值类型(结构体)。当你读取组件、修改并写回时,你是在替换整个组件:
swift
// 读取-修改-写入模式
var health = entity.components[HealthComponent.self]!
health.current -= damage
entity.components[HealthComponent.self] = health反模式:持有组件引用并期望修改自动同步。组件在读取时会被复制。
4. Systems
4. 系统
System Protocol
系统协议
swift
struct DamageSystem: System {
// Define which components this system needs
static let query = EntityQuery(where: .has(HealthComponent.self))
init(scene: RealityKit.Scene) {
// One-time setup
}
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
var health = entity.components[HealthComponent.self]!
if health.current <= 0 {
entity.removeFromParent()
}
}
}
}
// Register system
DamageSystem.registerSystem()swift
struct DamageSystem: System {
// 定义该系统需要的组件
static let query = EntityQuery(where: .has(HealthComponent.self))
init(scene: RealityKit.Scene) {
// 一次性初始化
}
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
var health = entity.components[HealthComponent.self]!
if health.current <= 0 {
entity.removeFromParent()
}
}
}
}
// 注册系统
DamageSystem.registerSystem()System Best Practices
系统最佳实践
- One responsibility per system: MovementSystem, DamageSystem, RenderingSystem — not GameLogicSystem
- Query filtering: Use precise queries to avoid processing irrelevant entities
- Order matters: Systems run in registration order. Register dependencies first.
- Avoid storing entity references: Query each frame instead. Entity references can become stale.
- 单一职责:每个系统只负责一项功能,比如MovementSystem、DamageSystem、RenderingSystem,而不是GameLogicSystem
- 查询过滤:使用精确查询避免处理无关实体
- 执行顺序:系统按注册顺序运行,先注册依赖系统
- 避免存储实体引用:每帧查询实体,实体引用可能会失效
Event Handling
事件处理
swift
// Subscribe to collision events
scene.subscribe(to: CollisionEvents.Began.self) { event in
let entityA = event.entityA
let entityB = event.entityB
// Handle collision
}
// Subscribe to scene update
scene.subscribe(to: SceneEvents.Update.self) { event in
let deltaTime = event.deltaTime
// Per-frame logic
}swift
// 订阅碰撞事件
scene.subscribe(to: CollisionEvents.Began.self) { event in
let entityA = event.entityA
let entityB = event.entityB
// 处理碰撞
}
// 订阅场景更新
scene.subscribe(to: SceneEvents.Update.self) { event in
let deltaTime = event.deltaTime
// 每帧逻辑
}5. SwiftUI Integration
5. SwiftUI集成
RealityView (iOS 18+, visionOS 1.0+)
RealityView(iOS 18+, visionOS 1.0+)
swift
struct ContentView: View {
var body: some View {
RealityView { content in
// make closure — called once
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: false)]
)
content.add(box)
} update: { content in
// update closure — called when SwiftUI state changes
}
}
}swift
struct ContentView: View {
var body: some View {
RealityView { content in
// make闭包 — 只调用一次
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: false)]
)
content.add(box)
} update: { content in
// update闭包 — SwiftUI状态变化时调用
}
}
}RealityView with Camera (iOS)
iOS端带相机的RealityView
On iOS, provides a camera content parameter for configuring the AR or virtual camera:
RealityViewswift
RealityView { content, attachments in
// Load 3D content
if let model = try? await ModelEntity(named: "scene") {
content.add(model)
}
}在iOS上,提供相机内容参数用于配置AR或虚拟相机:
RealityViewswift
RealityView { content, attachments in
// 加载3D内容
if let model = try? await ModelEntity(named: "scene") {
content.add(model)
}
}Loading Content Asynchronously
异步加载内容
swift
RealityView { content in
// Load from bundle
if let entity = try? await Entity(named: "MyScene", in: .main) {
content.add(entity)
}
// Load from URL
if let entity = try? await Entity(contentsOf: modelURL) {
content.add(entity)
}
}swift
RealityView { content in
// 从Bundle加载
if let entity = try? await Entity(named: "MyScene", in: .main) {
content.add(entity)
}
// 从URL加载
if let entity = try? await Entity(contentsOf: modelURL) {
content.add(entity)
}
}Model3D (Simple Display)
Model3D(简单展示)
swift
// Simple 3D model display (no interaction)
Model3D(named: "toy_robot") { model in
model
.resizable()
.scaledToFit()
} placeholder: {
ProgressView()
}swift
// 简单3D模型展示(无交互)
Model3D(named: "toy_robot") { model in
model
.resizable()
.scaledToFit()
} placeholder: {
ProgressView()
}SwiftUI Attachments (visionOS)
SwiftUI附件(visionOS)
swift
RealityView { content, attachments in
let entity = ModelEntity(mesh: .generateSphere(radius: 0.1))
content.add(entity)
if let label = attachments.entity(for: "priceTag") {
label.position = SIMD3(0, 0.15, 0)
entity.addChild(label)
}
} attachments: {
Attachment(id: "priceTag") {
Text("$9.99")
.padding()
.glassBackgroundEffect()
}
}swift
RealityView { content, attachments in
let entity = ModelEntity(mesh: .generateSphere(radius: 0.1))
content.add(entity)
if let label = attachments.entity(for: "priceTag") {
label.position = SIMD3(0, 0.15, 0)
entity.addChild(label)
}
} attachments: {
Attachment(id: "priceTag") {
Text("$9.99")
.padding()
.glassBackgroundEffect()
}
}State Binding Pattern
状态绑定模式
swift
struct GameView: View {
@State private var score = 0
var body: some View {
VStack {
Text("Score: \(score)")
RealityView { content in
let scene = try! await Entity(named: "GameScene")
content.add(scene)
} update: { content in
// React to state changes
// Note: update is called when SwiftUI state changes,
// not every frame. Use Systems for per-frame logic.
}
}
}
}swift
struct GameView: View {
@State private var score = 0
var body: some View {
VStack {
Text("Score: \(score)")
RealityView { content in
let scene = try! await Entity(named: "GameScene")
content.add(scene)
} update: { content in
// 响应状态变化
// 注意:update仅在SwiftUI状态变化时调用,
// 并非每帧调用。每帧逻辑请使用Systems。
}
}
}
}6. AR on iOS
6. iOS端AR功能
AnchorEntity
AnchorEntity
swift
// Horizontal plane
let anchor = AnchorEntity(.plane(.horizontal, classification: .table,
minimumBounds: SIMD2(0.2, 0.2)))
// Vertical plane
let anchor = AnchorEntity(.plane(.vertical, classification: .wall,
minimumBounds: SIMD2(0.5, 0.5)))
// World position
let anchor = AnchorEntity(world: SIMD3<Float>(0, 0, -1))
// Image anchor
let anchor = AnchorEntity(.image(group: "AR Resources", name: "poster"))
// Face anchor (front camera)
let anchor = AnchorEntity(.face)
// Body anchor
let anchor = AnchorEntity(.body)swift
// 水平面
let anchor = AnchorEntity(.plane(.horizontal, classification: .table,
minimumBounds: SIMD2(0.2, 0.2)))
// 垂直面
let anchor = AnchorEntity(.plane(.vertical, classification: .wall,
minimumBounds: SIMD2(0.5, 0.5)))
// 世界位置
let anchor = AnchorEntity(world: SIMD3<Float>(0, 0, -1))
// 图像锚点
let anchor = AnchorEntity(.image(group: "AR Resources", name: "poster"))
// 面部锚点(前置摄像头)
let anchor = AnchorEntity(.face)
// 身体锚点
let anchor = AnchorEntity(.body)SpatialTrackingSession (iOS 18+)
SpatialTrackingSession(iOS 18+)
swift
let session = SpatialTrackingSession()
let configuration = SpatialTrackingSession.Configuration(tracking: [.plane, .object])
let result = await session.run(configuration)
if let notSupported = result {
// Handle unsupported tracking on this device
for denied in notSupported.deniedTrackingModes {
print("Not supported: \(denied)")
}
}swift
let session = SpatialTrackingSession()
let configuration = SpatialTrackingSession.Configuration(tracking: [.plane, .object])
let result = await session.run(configuration)
if let notSupported = result {
// 处理设备不支持的追踪模式
for denied in notSupported.deniedTrackingModes {
print("Not supported: \(denied)")
}
}AR Best Practices
AR最佳实践
- Anchor entities to detected surfaces rather than world positions for stability
- Use plane classification (,
.table,.floor) to place content appropriately.wall - Start with horizontal plane detection — it's the most reliable
- Test on real devices; simulator AR is limited
- Provide visual feedback during surface detection (coaching overlay)
- 将实体锚定到检测到的表面而非世界位置,以提升稳定性
- 使用平面分类(,
.table,.floor)合理放置内容.wall - 从水平面检测开始,这是最可靠的模式
- 在真实设备上测试,模拟器AR功能有限
- 在表面检测过程中提供视觉反馈(引导层)
7. Interaction
7. 交互
ManipulationComponent (iOS, visionOS)
ManipulationComponent(iOS, visionOS)
swift
// Enable drag, rotate, scale gestures
entity.components[ManipulationComponent.self] = ManipulationComponent(
allowedModes: .all // .translate, .rotate, .scale
)
// Also requires CollisionComponent for hit testing
entity.generateCollisionShapes(recursive: true)swift
// 启用拖拽、旋转、缩放手势
entity.components[ManipulationComponent.self] = ManipulationComponent(
allowedModes: .all // .translate, .rotate, .scale
)
// 还需要CollisionComponent用于命中测试
entity.generateCollisionShapes(recursive: true)InputTargetComponent (visionOS)
InputTargetComponent(visionOS)
swift
// Required for visionOS gesture input
entity.components[InputTargetComponent.self] = InputTargetComponent()
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))]
)swift
// visionOS手势输入必需
entity.components[InputTargetComponent.self] = InputTargetComponent()
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))]
)Gesture Integration with SwiftUI
SwiftUI手势集成
swift
RealityView { content in
let entity = ModelEntity(mesh: .generateBox(size: 0.1))
entity.generateCollisionShapes(recursive: true)
entity.components.set(InputTargetComponent())
content.add(entity)
}
.gesture(
TapGesture()
.targetedToAnyEntity()
.onEnded { value in
let tappedEntity = value.entity
// Handle tap
}
)
.gesture(
DragGesture()
.targetedToAnyEntity()
.onChanged { value in
value.entity.position = value.convert(value.location3D,
from: .local, to: .scene)
}
)swift
RealityView { content in
let entity = ModelEntity(mesh: .generateBox(size: 0.1))
entity.generateCollisionShapes(recursive: true)
entity.components.set(InputTargetComponent())
content.add(entity)
}
.gesture(
TapGesture()
.targetedToAnyEntity()
.onEnded { value in
let tappedEntity = value.entity
// 处理点击
}
)
.gesture(
DragGesture()
.targetedToAnyEntity()
.onChanged { value in
value.entity.position = value.convert(value.location3D,
from: .local, to: .scene)
}
)Hit Testing
命中测试
swift
// Ray-cast from screen point
if let result = arView.raycast(from: screenPoint,
allowing: .estimatedPlane,
alignment: .horizontal).first {
let worldPosition = result.worldTransform.columns.3
// Place entity at worldPosition
}swift
// 从屏幕点发射射线
if let result = arView.raycast(from: screenPoint,
allowing: .estimatedPlane,
alignment: .horizontal).first {
let worldPosition = result.worldTransform.columns.3
// 在worldPosition处放置实体
}8. Materials and Rendering
8. 材质与渲染
Material Types
材质类型
| Material | Purpose | Customization |
|---|---|---|
| Solid color or texture | Color, metallic, roughness |
| Full PBR | All PBR maps (base color, normal, metallic, roughness, AO, emissive) |
| No lighting response | Color or texture, always fully lit |
| Invisible but occludes | AR content hiding behind real objects |
| Video playback on surface | AVPlayer-driven |
| Custom shader graph | Reality Composer Pro |
| Metal shader functions | Full Metal control |
| 材质 | 用途 | 自定义程度 |
|---|---|---|
| 纯色或纹理 | 颜色、金属度、粗糙度 |
| 完整PBR材质 | 所有PBR贴图(基础色、法线、金属度、粗糙度、环境光遮蔽、自发光) |
| 无光照响应 | 颜色或纹理,始终全亮 |
| 不可见但遮挡内容 | 隐藏真实物体后方的3D内容 |
| 表面播放视频 | 由AVPlayer驱动 |
| 自定义着色器图 | 使用Reality Composer Pro |
| Metal着色器函数 | 完全控制Metal |
PhysicallyBasedMaterial
PhysicallyBasedMaterial
swift
var material = PhysicallyBasedMaterial()
material.baseColor = .init(tint: .white,
texture: .init(try! .load(named: "albedo")))
material.metallic = .init(floatLiteral: 0.0)
material.roughness = .init(floatLiteral: 0.5)
material.normal = .init(texture: .init(try! .load(named: "normal")))
material.ambientOcclusion = .init(texture: .init(try! .load(named: "ao")))
material.emissiveColor = .init(color: .blue)
material.emissiveIntensity = 2.0
let entity = ModelEntity(
mesh: .generateSphere(radius: 0.1),
materials: [material]
)swift
var material = PhysicallyBasedMaterial()
material.baseColor = .init(tint: .white,
texture: .init(try! .load(named: "albedo")))
material.metallic = .init(floatLiteral: 0.0)
material.roughness = .init(floatLiteral: 0.5)
material.normal = .init(texture: .init(try! .load(named: "normal")))
material.ambientOcclusion = .init(texture: .init(try! .load(named: "ao")))
material.emissiveColor = .init(color: .blue)
material.emissiveIntensity = 2.0
let entity = ModelEntity(
mesh: .generateSphere(radius: 0.1),
materials: [material]
)OcclusionMaterial (AR)
OcclusionMaterial(AR)
swift
// Invisible plane that hides 3D content behind it
let occluder = ModelEntity(
mesh: .generatePlane(width: 1, depth: 1),
materials: [OcclusionMaterial()]
)
occluder.position = SIMD3(0, 0, 0)
anchor.addChild(occluder)swift
// 隐藏后方3D内容的不可见平面
let occluder = ModelEntity(
mesh: .generatePlane(width: 1, depth: 1),
materials: [OcclusionMaterial()]
)
occluder.position = SIMD3(0, 0, 0)
anchor.addChild(occluder)Environment Lighting
环境光照
swift
// Image-based lighting
if let resource = try? await EnvironmentResource(named: "studio_lighting") {
// Apply via RealityView content
}swift
// 基于图像的光照
if let resource = try? await EnvironmentResource(named: "studio_lighting") {
// 通过RealityView content应用
}9. Physics and Collision
9. 物理与碰撞
Collision Shapes
碰撞形状
swift
// Generate from mesh (accurate but expensive)
entity.generateCollisionShapes(recursive: true)
// Manual shapes (prefer for performance)
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [
.generateBox(size: SIMD3(0.1, 0.2, 0.1)), // Box
.generateSphere(radius: 0.1), // Sphere
.generateCapsule(height: 0.3, radius: 0.05) // Capsule
]
)swift
// 从网格生成(精确但性能开销大)
entity.generateCollisionShapes(recursive: true)
// 手动创建形状(优先选择以提升性能)
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [
.generateBox(size: SIMD3(0.1, 0.2, 0.1)), // 盒子
.generateSphere(radius: 0.1), // 球体
.generateCapsule(height: 0.3, radius: 0.05) // 胶囊体
]
)Physics Body
物理体
swift
// Dynamic — physics simulation controls movement
entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
massProperties: .init(mass: 1.0),
material: .generate(staticFriction: 0.5,
dynamicFriction: 0.3,
restitution: 0.4),
mode: .dynamic
)
// Static — immovable collision surface
ground.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
mode: .static
)
// Kinematic — code-controlled, participates in collisions
platform.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
mode: .kinematic
)swift
// 动态 — 物理模拟控制移动
entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
massProperties: .init(mass: 1.0),
material: .generate(staticFriction: 0.5,
dynamicFriction: 0.3,
restitution: 0.4),
mode: .dynamic
)
// 静态 — 不可移动的碰撞表面
ground.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
mode: .static
)
// 运动学 — 代码控制,参与碰撞
platform.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
mode: .kinematic
)Collision Groups and Filters
碰撞组与过滤器
swift
// Define groups
let playerGroup = CollisionGroup(rawValue: 1 << 0)
let enemyGroup = CollisionGroup(rawValue: 1 << 1)
let bulletGroup = CollisionGroup(rawValue: 1 << 2)
// Filter: player collides with enemies and bullets
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [.generateSphere(radius: 0.1)],
filter: CollisionFilter(
group: playerGroup,
mask: enemyGroup | bulletGroup
)
)swift
// 定义组
let playerGroup = CollisionGroup(rawValue: 1 << 0)
let enemyGroup = CollisionGroup(rawValue: 1 << 1)
let bulletGroup = CollisionGroup(rawValue: 1 << 2)
// 过滤器:玩家与敌人、子弹碰撞
entity.components[CollisionComponent.self] = CollisionComponent(
shapes: [.generateSphere(radius: 0.1)],
filter: CollisionFilter(
group: playerGroup,
mask: enemyGroup | bulletGroup
)
)Collision Events
碰撞事件
swift
// Subscribe in RealityView make closure or System
scene.subscribe(to: CollisionEvents.Began.self, on: playerEntity) { event in
let otherEntity = event.entityA == playerEntity ? event.entityB : event.entityA
handleCollision(with: otherEntity)
}swift
// 在RealityView的make闭包或System中订阅
scene.subscribe(to: CollisionEvents.Began.self, on: playerEntity) { event in
let otherEntity = event.entityA == playerEntity ? event.entityB : event.entityA
handleCollision(with: otherEntity)
}Applying Forces
施加力
swift
if var motion = entity.components[PhysicsMotionComponent.self] {
motion.linearVelocity = SIMD3(0, 5, 0) // Impulse up
entity.components[PhysicsMotionComponent.self] = motion
}swift
if var motion = entity.components[PhysicsMotionComponent.self] {
motion.linearVelocity = SIMD3(0, 5, 0) // 向上的冲量
entity.components[PhysicsMotionComponent.self] = motion
}10. Animation
10. 动画
Transform Animation
变换动画
swift
// Animate to position over duration
entity.move(
to: Transform(
scale: SIMD3(repeating: 1.5),
rotation: simd_quatf(angle: .pi, axis: SIMD3(0, 1, 0)),
translation: SIMD3(0, 2, 0)
),
relativeTo: entity.parent,
duration: 2.0,
timingFunction: .easeInOut
)swift
// 在指定时长内动画到目标位置
entity.move(
to: Transform(
scale: SIMD3(repeating: 1.5),
rotation: simd_quatf(angle: .pi, axis: SIMD3(0, 1, 0)),
translation: SIMD3(0, 2, 0)
),
relativeTo: entity.parent,
duration: 2.0,
timingFunction: .easeInOut
)Playing USD Animations
播放USD动画
swift
if let entity = try? await Entity(named: "character") {
// Play all available animations
for animation in entity.availableAnimations {
entity.playAnimation(animation.repeat())
}
}swift
if let entity = try? await Entity(named: "character") {
// 播放所有可用动画
for animation in entity.availableAnimations {
entity.playAnimation(animation.repeat())
}
}Animation Playback Control
动画播放控制
swift
let controller = entity.playAnimation(animation)
controller.pause()
controller.resume()
controller.speed = 2.0 // 2x playback speed
controller.blendFactor = 0.5 // Blend with current stateswift
let controller = entity.playAnimation(animation)
controller.pause()
controller.resume()
controller.speed = 2.0 // 2倍速播放
controller.blendFactor = 0.5 // 与当前状态混合11. Audio
11. 音频
Spatial Audio
空间音频
swift
// Load audio resource
let resource = try! AudioFileResource.load(named: "engine.wav",
configuration: .init(shouldLoop: true))
// Create entity with spatial audio
let audioEntity = Entity()
audioEntity.components[SpatialAudioComponent.self] = SpatialAudioComponent()
let controller = audioEntity.playAudio(resource)
// Position the audio source in 3D space
audioEntity.position = SIMD3(2, 0, -1)swift
// 加载音频资源
let resource = try! AudioFileResource.load(named: "engine.wav",
configuration: .init(shouldLoop: true))
// 创建带空间音频的实体
let audioEntity = Entity()
audioEntity.components[SpatialAudioComponent.self] = SpatialAudioComponent()
let controller = audioEntity.playAudio(resource)
// 在3D空间中定位音频源
audioEntity.position = SIMD3(2, 0, -1)Ambient Audio
环境音频
swift
entity.components[AmbientAudioComponent.self] = AmbientAudioComponent()
entity.playAudio(backgroundMusic)swift
entity.components[AmbientAudioComponent.self] = AmbientAudioComponent()
entity.playAudio(backgroundMusic)12. Performance
12. 性能优化
Entity Count
实体数量
- Under 100 entities: No concerns
- 100-1000 entities: Monitor with RealityKit debugger
- 1000+ entities: Use instancing and LOD strategies
- 少于100个实体:无需担心
- 100-1000个实体:使用RealityKit调试器监控
- 1000+个实体:使用实例化和LOD策略
Instancing
实例化
swift
// Share mesh and material across many entities
let sharedMesh = MeshResource.generateSphere(radius: 0.01)
let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
for i in 0..<1000 {
let entity = ModelEntity(mesh: sharedMesh, materials: [sharedMaterial])
entity.position = randomPosition()
parent.addChild(entity)
}RealityKit automatically batches entities with identical mesh and material resources.
swift
// 为多个实体共享网格和材质
let sharedMesh = MeshResource.generateSphere(radius: 0.01)
let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
for i in 0..<100 {
let entity = ModelEntity(mesh: sharedMesh, materials: [sharedMaterial])
entity.position = randomPosition()
parent.addChild(entity)
}RealityKit会自动批处理具有相同网格和材质资源的实体。
Component Churn
组件频繁更新
Anti-pattern: Creating and replacing components every frame.
swift
// BAD — component allocation every frame
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
entity.components[ModelComponent.self] = ModelComponent(
mesh: .generateBox(size: 0.1),
materials: [newMaterial] // New allocation every frame
)
}
}
// GOOD — modify existing component
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
// Only update when actually needed
if needsUpdate {
var model = entity.components[ModelComponent.self]!
model.materials = [cachedMaterial]
entity.components[ModelComponent.self] = model
}
}
}反模式:每帧创建和替换组件。
swift
// 错误示例 — 每帧分配组件
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
entity.components[ModelComponent.self] = ModelComponent(
mesh: .generateBox(size: 0.1),
materials: [newMaterial] // 每帧新分配
)
}
}
// 正确示例 — 修改现有组件
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: query, updatingSystemWhen: .rendering) {
// 仅在需要时更新
if needsUpdate {
var model = entity.components[ModelComponent.self]!
model.materials = [cachedMaterial]
entity.components[ModelComponent.self] = model
}
}
}Collision Shape Optimization
碰撞形状优化
- Use simple shapes (box, sphere, capsule) instead of mesh-based collision
- is convenient but expensive
generateCollisionShapes(recursive: true) - For static geometry, generate shapes once during setup
- 使用简单形状(盒子、球体、胶囊体)替代基于网格的碰撞
- 便捷但性能开销大
generateCollisionShapes(recursive: true) - 对于静态几何体,在初始化时一次性生成形状
Profiling
性能分析
Use Xcode's RealityKit debugger:
- Entity Inspector: View entity hierarchy and components
- Statistics Overlay: Entity count, draw calls, triangle count
- Physics Visualization: Show collision shapes
使用Xcode的RealityKit调试器:
- 实体检查器:查看实体层级和组件
- 统计面板:实体数量、绘制调用、三角形数量
- 物理可视化:显示碰撞形状
13. Multiplayer
13. 多人协作
Synchronization Basics
同步基础
swift
// Components sync automatically if they conform to Codable
struct ScoreComponent: Component, Codable {
var points: Int
}
// SynchronizationComponent controls what syncs
entity.components[SynchronizationComponent.self] = SynchronizationComponent()swift
// 符合Codable的组件会自动同步
struct ScoreComponent: Component, Codable {
var points: Int
}
// SynchronizationComponent控制同步内容
entity.components[SynchronizationComponent.self] = SynchronizationComponent()MultipeerConnectivityService
MultipeerConnectivityService
swift
let service = try MultipeerConnectivityService(session: mcSession)
// Entities with SynchronizationComponent auto-sync across peersswift
let service = try MultipeerConnectivityService(session: mcSession)
// 带有SynchronizationComponent的实体会在设备间自动同步Ownership
所有权
- Only the owner of an entity can modify it
- Request ownership before modifying shared entities
- Non-Codable component data does not sync
- 只有实体的所有者可以修改它
- 修改共享实体前请求所有权
- 非Codable组件数据不会同步
14. Anti-Patterns
14. 反模式
Anti-Pattern 1: UIKit-Style Thinking in ECS
反模式1:ECS中使用UIKit风格思维
Time cost: Hours of frustration from fighting the architecture
swift
// BAD — subclassing Entity for behavior
class PlayerEntity: Entity {
func takeDamage(_ amount: Int) { /* logic in entity */ }
}
// GOOD — component holds data, system has logic
struct HealthComponent: Component { var hp: Int }
struct DamageSystem: System {
static let query = EntityQuery(where: .has(HealthComponent.self))
func update(context: SceneUpdateContext) {
// Process damage here
}
}时间成本:与架构对抗导致数小时的挫败
swift
// 错误示例 — 为实体子类添加行为
class PlayerEntity: Entity {
func takeDamage(_ amount: Int) { /* 逻辑在实体中 */ }
}
// 正确示例 — 组件存储数据,系统处理逻辑
struct HealthComponent: Component { var hp: Int }
struct DamageSystem: System {
static let query = EntityQuery(where: .has(HealthComponent.self))
func update(context: SceneUpdateContext) {
// 在这里处理伤害逻辑
}
}Anti-Pattern 2: Monolithic Entities
反模式2:单体实体
Time cost: Untestable, inflexible architecture
Don't put all game logic in one entity type. Split into components that can be mixed and matched.
时间成本:不可测试、架构僵化
不要将所有游戏逻辑放在一个实体类型中。拆分为可组合的组件。
Anti-Pattern 3: Frame-Based Updates Without Systems
反模式3:不使用系统的帧更新
Time cost: Missed frame updates, inconsistent behavior
swift
// BAD — timer-based updates
Timer.scheduledTimer(withTimeInterval: 1/60, repeats: true) { _ in
entity.position.x += 0.01
}
// GOOD — System update
struct MovementSystem: System {
static let query = EntityQuery(where: .has(VelocityComponent.self))
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
let velocity = entity.components[VelocityComponent.self]!
entity.position += velocity.value * Float(context.deltaTime)
}
}
}时间成本:错过帧更新、行为不一致
swift
// 错误示例 — 基于定时器的更新
Timer.scheduledTimer(withTimeInterval: 1/60, repeats: true) { _ in
entity.position.x += 0.01
}
// 正确示例 — 系统更新
struct MovementSystem: System {
static let query = EntityQuery(where: .has(VelocityComponent.self))
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
let velocity = entity.components[VelocityComponent.self]!
entity.position += velocity.value * Float(context.deltaTime)
}
}
}Anti-Pattern 4: Not Generating Collision Shapes for Interactive Entities
反模式4:交互实体不生成碰撞形状
Time cost: 15-30 min debugging "why taps don't work"
Gestures require . If an entity has (visionOS) or but no , gestures will never fire.
CollisionComponentInputTargetComponentManipulationComponentCollisionComponent时间成本:15-30分钟调试“为什么手势不生效”
手势需要。如果实体有(visionOS)或但没有,手势永远不会触发。
CollisionComponentInputTargetComponentManipulationComponentCollisionComponentAnti-Pattern 5: Storing Entity References in Systems
反模式5:在系统中存储实体引用
Time cost: Crashes from stale references
swift
// BAD — entity might be removed between frames
struct BadSystem: System {
var playerEntity: Entity? // Stale reference risk
func update(context: SceneUpdateContext) {
playerEntity?.position.x += 0.1 // May crash
}
}
// GOOD — query each frame
struct GoodSystem: System {
static let query = EntityQuery(where: .has(PlayerComponent.self))
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
entity.position.x += Float(context.deltaTime)
}
}
}时间成本:引用失效导致崩溃
swift
// 错误示例 — 实体可能在帧间被移除
struct BadSystem: System {
var playerEntity: Entity? // 引用失效风险
func update(context: SceneUpdateContext) {
playerEntity?.position.x += 0.1 // 可能崩溃
}
}
// 正确示例 — 每帧查询
struct GoodSystem: System {
static let query = EntityQuery(where: .has(PlayerComponent.self))
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
entity.position.x += Float(context.deltaTime)
}
}
}15. Code Review Checklist
15. 代码审查清单
- Custom components registered via before use
registerComponent() - Systems registered via before scene loads
registerSystem() - Components are value types (structs), not classes
- Read-modify-write pattern used for component updates
- Interactive entities have
CollisionComponent - visionOS interactive entities have
InputTargetComponent - Collision shapes are simple (box/sphere/capsule) where possible
- No entity references stored across frames in Systems
- Mesh and material resources shared across identical entities
- Component updates only occur when values actually change
- USD/USDZ format used for 3D assets (not .scn)
- Async loading used for all model/scene loading
- in closure-based subscriptions if retaining view/controller
[weak self]
- 自定义组件在使用前通过注册
registerComponent() - 系统在场景加载前通过注册
registerSystem() - 组件是值类型(结构体),而非类
- 组件更新使用读取-修改-写入模式
- 交互实体带有
CollisionComponent - visionOS交互实体带有
InputTargetComponent - 尽可能使用简单碰撞形状(盒子/球体/胶囊体)
- 系统中不跨帧存储实体引用
- 相同实体共享网格和材质资源
- 仅在值实际变化时更新组件
- 3D资源使用USD/USDZ格式(而非.scn)
- 所有模型/场景加载使用异步方式
- 闭包订阅中如果持有视图/控制器,使用
[weak self]
16. Pressure Scenarios
16. 压力场景
Scenario 1: "ECS Is Overkill for Our Simple App"
场景1:“ECS对我们的简单应用来说太复杂”
Pressure: Team wants to avoid learning ECS, just needs one 3D model displayed
Wrong approach: Skip ECS, jam all logic into RealityView closures.
Correct approach: Even simple apps benefit from ECS. A single in a is already using ECS — you're just not adding custom components yet. Start simple, add components as complexity grows.
ModelEntityRealityViewPush-back template: "We're already using ECS — Entity and ModelComponent. The pattern scales. Adding a custom component when we need behavior is one struct definition, not an architecture change."
压力:团队不想学习ECS,只需要展示一个3D模型
错误做法:跳过ECS,将所有逻辑塞进RealityView闭包。
正确做法:即使简单应用也能从ECS获益。RealityView中的单个已经在使用ECS——只是还没添加自定义组件。从简单开始,随着复杂度增加再添加组件。
ModelEntity反驳模板:“我们已经在使用ECS了——Entity和ModelComponent就是ECS的一部分。这种模式具备扩展性。当我们需要行为时,只需要定义一个结构体组件,而非改变架构。”
Scenario 2: "Just Use SceneKit, We Know It"
场景2:“就用SceneKit吧,我们熟悉它”
Pressure: Team has SceneKit experience, RealityKit is unfamiliar
Wrong approach: Build new features in SceneKit.
Correct approach: SceneKit is soft-deprecated. New features won't be added. Invest in RealityKit now — the ECS concepts transfer to other game engines (Unity, Unreal, Bevy) if needed.
Push-back template: "SceneKit is in maintenance mode — no new features, only security patches. Every line of SceneKit we write is migration debt. RealityKit's concepts (Entity, Component, System) are industry-standard ECS."
压力:团队有SceneKit经验,RealityKit不熟悉
错误做法:用SceneKit开发新功能。
正确做法:SceneKit已进入维护模式,不会再添加新功能。现在投入RealityKit——ECS概念可迁移到其他游戏引擎(Unity、Unreal、Bevy)。
反驳模板:“SceneKit处于维护模式——不再添加新功能,仅提供安全补丁。我们写的每一行SceneKit代码都是技术债务。RealityKit的概念(实体、组件、系统)是行业标准的ECS。”
Scenario 3: "Make It Work Without Collision Shapes"
场景3:“不用碰撞形状让它工作起来”
Pressure: Deadline, collision shape setup seems complex
Wrong approach: Skip collision shapes, use position-based proximity detection.
Correct approach: takes one line. Without it, gestures won't work and physics won't collide. The "shortcut" creates more debugging time than it saves.
entity.generateCollisionShapes(recursive: true)Push-back template: "Collision shapes are required for gestures and physics. It's one line: . Skipping it means gestures silently fail — a harder bug to diagnose."
entity.generateCollisionShapes(recursive: true)压力:截止日期临近,碰撞形状设置看起来复杂
错误做法:跳过碰撞形状,使用基于位置的 proximity 检测。
正确做法:只需要一行代码。没有它,手势无法工作,物理系统无法碰撞。“捷径”会导致比节省的时间更多的调试时间。
entity.generateCollisionShapes(recursive: true)反驳模板:“碰撞形状是手势和物理系统的必需条件。只需要一行代码:。跳过它会导致手势无声失效——这是更难诊断的bug。”
entity.generateCollisionShapes(recursive: true)Resources
资源
WWDC: 2019-603, 2019-605, 2021-10074, 2022-10074, 2023-10080, 2023-10081, 2024-10103, 2024-10153
Docs: /realitykit, /realitykit/entity, /realitykit/realityview, /realitykit/modelentity, /realitykit/anchorentity, /realitykit/component
Skills: axiom-realitykit-ref, axiom-realitykit-diag, axiom-scenekit, axiom-scenekit-ref
WWDC:2019-603, 2019-605, 2021-10074, 2022-10074, 2023-10080, 2023-10081, 2024-10103, 2024-10153
文档:/realitykit, /realitykit/entity, /realitykit/realityview, /realitykit/modelentity, /realitykit/anchorentity, /realitykit/component
技能:axiom-realitykit-ref, axiom-realitykit-diag, axiom-scenekit, axiom-scenekit-ref