axiom-realitykit-diag

Original🇺🇸 English
Translated

Use when RealityKit entities not visible, anchors not tracking, gestures not responding, performance drops, materials wrong, or multiplayer sync fails

2installs
Added on

NPX Install

npx skill4agent add charleswiltgen/axiom axiom-realitykit-diag

RealityKit Diagnostics

Systematic diagnosis for common RealityKit issues with time-cost annotations.

When to Use This Diagnostic Skill

Use this skill when:
  • Entity added but not visible in the scene
  • AR anchor not tracking or content floating
  • Tap/drag gestures not responding on 3D entities
  • Frame rate dropping or stuttering
  • Material looks wrong (too dark, too bright, incorrect colors)
  • Multiplayer entities not syncing across devices
  • Physics bodies not colliding or passing through each other
For RealityKit architecture patterns and best practices, see
axiom-realitykit
. For API reference, see
axiom-realitykit-ref
.

Mandatory First Step: Enable Debug Visualization

Time cost: 10 seconds vs hours of blind debugging
swift
// In your RealityView or ARView setup
#if DEBUG
// Xcode: Debug → Attach to Process → Show RealityKit Statistics
// Or enable in code:
arView.debugOptions = [
    .showStatistics,       // Entity count, draw calls, FPS
    .showPhysics,          // Collision shapes
    .showAnchorOrigins,    // Anchor positions
    .showAnchorGeometry    // Detected plane geometry
]
#endif
If you can't see collision shapes with
.showPhysics
, your
CollisionComponent
is missing or misconfigured. Fix collision before debugging gestures or physics.

Symptom 1: Entity Not Visible

Time saved: 30-60 min → 2-5 min
Entity added but nothing appears
├─ Is the entity added to the scene?
│   └─ NO → Add to RealityView content:
│        content.add(entity)
│        ✓ Entities must be in the scene graph to render
├─ Does the entity have a ModelComponent?
│   └─ NO → Add mesh and material:
│        entity.components[ModelComponent.self] = ModelComponent(
│            mesh: .generateBox(size: 0.1),
│            materials: [SimpleMaterial(color: .red, isMetallic: false)]
│        )
│        ✓ Bare Entity is invisible — it's just a container
├─ Is the entity's scale zero or nearly zero?
│   └─ CHECK → Print: entity.scale
│        USD models may import with unexpected scale.
│        Try: entity.scale = SIMD3(repeating: 0.01) for meter-scale models
├─ Is the entity behind the camera?
│   └─ CHECK → Print: entity.position(relativeTo: nil)
│        In RealityKit, -Z is forward (toward screen).
│        Try: entity.position = SIMD3(0, 0, -0.5) (half meter in front)
├─ Is the entity inside another object?
│   └─ CHECK → Move to a known visible position:
│        entity.position = SIMD3(0, 0, -1)
├─ Is the entity's isEnabled set to false?
│   └─ CHECK → entity.isEnabled = true
│        Also check parent: entity.isEnabledInHierarchy
├─ Is the entity on an untracked anchor?
│   └─ CHECK → Verify anchor is tracking:
│        entity.isAnchored (should be true)
│        If using plane anchor, ensure surface is detected first
└─ Is the material transparent or OcclusionMaterial?
    └─ CHECK → Inspect material:
         If using PhysicallyBasedMaterial, check baseColor is not black
         If using blending = .transparent, check opacity > 0

Quick Diagnostic

swift
func diagnoseVisibility(_ entity: Entity) {
    print("Name: \(entity.name)")
    print("Is enabled: \(entity.isEnabled)")
    print("In hierarchy: \(entity.isEnabledInHierarchy)")
    print("Is anchored: \(entity.isAnchored)")
    print("Position (world): \(entity.position(relativeTo: nil))")
    print("Scale: \(entity.scale)")
    print("Has model: \(entity.components[ModelComponent.self] != nil)")
    print("Children: \(entity.children.count)")
}

Symptom 2: Anchor Not Tracking

Time saved: 20-45 min → 3-5 min
AR content not appearing or floating
├─ Is the AR session running?
│   └─ For RealityView on iOS 18+, AR runs automatically
│       For ARView, check: arView.session.isRunning
├─ Is SpatialTrackingSession configured? (iOS 18+)
│   └─ CHECK → Ensure tracking modes requested:
│        let config = SpatialTrackingSession.Configuration(
│            tracking: [.plane, .object])
│        let result = await session.run(config)
│        if let notSupported = result {
│            // Handle unsupported modes
│        }
├─ Is the anchor type appropriate for the environment?
│   ├─ .plane(.horizontal) → Need a flat surface visible to camera
│   ├─ .plane(.vertical) → Need a wall visible to camera
│   ├─ .image → Image must be in "AR Resources" asset catalog
│   ├─ .face → Front camera required (not rear)
│   └─ .body → Full body must be visible
├─ Is minimumBounds too large?
│   └─ CHECK → Reduce minimum bounds:
│        AnchorEntity(.plane(.horizontal, classification: .any,
│            minimumBounds: SIMD2(0.1, 0.1)))  // Smaller = detects sooner
├─ Is the device supported?
│   └─ CHECK → Plane detection requires A12+ chip
│       Face tracking requires TrueDepth camera
│       Body tracking requires A12+ chip
└─ Is the environment adequate?
    └─ CHECK → AR needs:
         - Adequate lighting (not too dark)
         - Textured surfaces (not blank walls)
         - Stable device position during initial detection

Symptom 3: Gesture Not Responding

Time saved: 15-30 min → 2-3 min
Tap/drag on entity does nothing
├─ Does the entity have a CollisionComponent?
│   └─ NO → Add collision shapes:
│        entity.generateCollisionShapes(recursive: true)
│        // or manual:
│        entity.components[CollisionComponent.self] = CollisionComponent(
│            shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))])
│        ✓ Collision shapes are REQUIRED for gesture hit testing
├─ [visionOS] Does the entity have InputTargetComponent?
│   └─ NO → Add it:
│        entity.components[InputTargetComponent.self] = InputTargetComponent()
│        ✓ Required on visionOS for gesture input
├─ Is the gesture attached to the RealityView?
│   └─ CHECK → Gesture must be on the view, not the entity:
│        RealityView { content in ... }
│            .gesture(TapGesture().targetedToAnyEntity().onEnded { ... })
├─ Is the collision shape large enough to hit?
│   └─ CHECK → Enable .showPhysics to see shapes
│        Shapes too small = hard to tap.
│        Try: .generateBox(size: SIMD3(repeating: 0.1)) minimum
├─ Is the entity behind another entity?
│   └─ CHECK → Front entities may block gestures on back entities
│        Ensure collision is on the intended target
└─ Is the entity enabled?
    └─ CHECK → entity.isEnabled must be true
         Disabled entities don't receive input

Quick Diagnostic

swift
func diagnoseGesture(_ entity: Entity) {
    print("Has collision: \(entity.components[CollisionComponent.self] != nil)")
    print("Has input target: \(entity.components[InputTargetComponent.self] != nil)")
    print("Is enabled: \(entity.isEnabled)")
    print("Is anchored: \(entity.isAnchored)")

    if let collision = entity.components[CollisionComponent.self] {
        print("Collision shapes: \(collision.shapes.count)")
    }
}

Symptom 4: Performance Problems

Time saved: 1-3 hours → 10-20 min
Frame rate dropping or stuttering
├─ How many entities are in the scene?
│   └─ CHECK → Print entity count:
│        var count = 0
│        func countEntities(_ entity: Entity) {
│            count += 1
│            for child in entity.children { countEntities(child) }
│        }
│        Under 100: unlikely to be entity count
│        100-500: review for optimization
│        500+: definitely needs optimization
├─ Are mesh/material resources shared?
│   └─ NO → Share resources across identical entities:
│        let sharedMesh = MeshResource.generateBox(size: 0.05)
│        let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
│        // Reuse for all instances
│        ✓ RealityKit batches entities with identical resources
├─ Is a System creating components every frame?
│   └─ CHECK → Look for allocations in update():
│        Creating ModelComponent, CollisionComponent, or materials
│        every frame causes GC pressure.
│        Cache resources, only update when values change.
├─ Are collision shapes mesh-based?
│   └─ CHECK → Replace generateCollisionShapes(recursive: true)
│        with simple shapes (box, sphere, capsule) for dynamic entities
├─ Is generateCollisionShapes called repeatedly?
│   └─ CHECK → Call once during setup, not every frame
├─ Are there too many physics bodies?
│   └─ CHECK → Dynamic bodies are most expensive.
│        Convert distant/static objects to .static mode.
│        Remove physics from non-interactive entities.
└─ Is the model polygon count too high?
    └─ CHECK → Decimate models for real-time use.
         Target: <100K triangles total for mobile AR.
         Use LOD (Level of Detail) for distant objects.

Symptom 5: Material Looks Wrong

Time saved: 15-45 min → 5-10 min
Colors, lighting, or textures look incorrect
├─ Is the scene too dark?
│   └─ CHECK → Missing environment lighting:
│        Add DirectionalLightComponent or EnvironmentResource
│        In AR, RealityKit uses real-world lighting automatically
│        In non-AR, you must provide lighting explicitly
├─ Is the baseColor set?
│   └─ CHECK → PhysicallyBasedMaterial defaults to white
│        material.baseColor = .init(tint: .red)
│        If using a texture, verify it loaded:
│        try TextureResource(named: "albedo")
├─ Is metallic set incorrectly?
│   └─ CHECK → metallic = 1.0 makes surfaces mirror-like
│        Most real objects: metallic = 0.0
│        Only metals (gold, silver, chrome): metallic = 1.0
├─ Is the texture semantic wrong?
│   └─ CHECK → Use correct semantic:
│        .color for albedo/baseColor textures
│        .raw for data textures (metallic, roughness)
│        .normal for normal maps
│        .hdrColor for HDR textures
├─ Is the model upside down or inside out?
│   └─ CHECK → Try:
│        material.faceCulling = .none (shows both sides)
│        If that fixes it, the model normals are flipped
└─ Is blending/transparency unexpected?
    └─ CHECK → material.blending
         Default is .opaque
         For transparency: .transparent(opacity: ...)

Symptom 6: Physics Not Working

Time saved: 20-40 min → 5-10 min
Objects pass through each other or don't collide
├─ Do both entities have CollisionComponent?
│   └─ NO → Both sides of a collision need CollisionComponent
├─ Does the moving entity have PhysicsBodyComponent?
│   └─ NO → Add physics body:
│        entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
│            mode: .dynamic)
├─ Are collision groups/filters configured correctly?
│   └─ CHECK → Entities must be in compatible groups:
│        Default: group = .default, mask = .all
│        If using custom groups, verify mask includes the other group
├─ Is the physics mode correct?
│   ├─ Two .static bodies → Never collide (both immovable)
│   ├─ .dynamic + .static → Correct (common setup)
│   ├─ .dynamic + .dynamic → Both move on collision
│   └─ .kinematic + .dynamic → Kinematic pushes dynamic
├─ Is the collision shape appropriate?
│   └─ CHECK → .showPhysics debug option
│        Shape may be too small, offset, or wrong type
└─ Are entities on different anchors?
    └─ CHECK → "Physics bodies and colliders affect only
         entities that share the same anchor" (Apple docs)
         Move entities under the same anchor for physics interaction

Symptom 7: Multiplayer Sync Issues

Time saved: 30-60 min → 10-15 min
Entities not appearing on other devices
├─ Does the entity have SynchronizationComponent?
│   └─ NO → Add it:
│        entity.components[SynchronizationComponent.self] =
│            SynchronizationComponent()
├─ Is the MultipeerConnectivityService set up?
│   └─ CHECK → Verify MCSession is connected before syncing
├─ Are custom components Codable?
│   └─ NO → Non-Codable components don't sync
│        struct MyComponent: Component, Codable { ... }
├─ Does the entity have an owner?
│   └─ CHECK → Only the owner can modify synced properties
│        Request ownership before modifying:
│        entity.requestOwnership { result in ... }
└─ Is the entity anchored?
    └─ CHECK → Unanchored entities may not sync position correctly
         Use a shared world anchor for reliable positioning

Common Mistakes

MistakeTime CostFix
No CollisionComponent on interactive entity15-30 min
entity.generateCollisionShapes(recursive: true)
Missing InputTargetComponent on visionOS10-20 minAdd
InputTargetComponent()
Gesture on wrong view (not RealityView)10-15 minAttach
.gesture()
to
RealityView
Entity scale wrong for USD model15-30 minCheck units: meters vs centimeters
No lighting in non-AR scene10-20 minAdd
DirectionalLightComponent
Storing entity refs in System30-60 min crash debuggingQuery with
EntityQuery
each frame
Components not registered10-15 minCall
registerComponent()
in app init
Systems not registered10-15 minCall
registerSystem()
before scene load
Physics across different anchors20-40 minPut interacting entities under same anchor
Calling generateCollisionShapes every framePerformance degradationCall once during setup

Diagnostic Quick Reference

SymptomFirst CheckTime Saved
Not visibleHas ModelComponent? Scale > 0?30-60 min
No gesture responseHas CollisionComponent?15-30 min
Not trackingAnchor type matches environment?20-45 min
Frame dropsEntity count? Resource sharing?1-3 hours
Wrong colorsHas lighting? Metallic value?15-45 min
No collisionBoth have CollisionComponent? Same anchor?20-40 min
No syncSynchronizationComponent? Codable?30-60 min

Resources

WWDC: 2019-603, 2019-605, 2023-10080, 2024-10103
Docs: /realitykit, /realitykit/entity, /realitykit/collisioncomponent, /realitykit/physicsbodycomponent
Skills: axiom-realitykit, axiom-realitykit-ref