nightingale-karaoke

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Nightingale Karaoke Skill

Nightingale 卡拉OK Skill

Skill by ara.so — Daily 2026 Skills collection.
Nightingale is a self-contained, ML-powered karaoke application written in Rust (Bevy engine). It scans a local music folder, separates vocals from instrumentals (UVR Karaoke model or Demucs), transcribes lyrics with word-level timestamps (WhisperX), and plays back with synchronized highlighting, real-time pitch scoring, player profiles, and GPU shader / video backgrounds. Everything — ffmpeg, Python, PyTorch, ML models — is bootstrapped automatically on first launch.

ara.so开发的Skill — 属于Daily 2026 Skills合集。
Nightingale是一款独立的、基于机器学习的卡拉OK应用,使用Rust语言(Bevy引擎)开发。它可以扫描本地音乐文件夹,分离人声与伴奏(使用UVR Karaoke模型或Demucs),通过WhisperX生成带逐词时间戳的歌词转录,并支持同步高亮播放、实时音高评分、玩家档案以及GPU着色器/视频背景。首次启动时,所有依赖项——ffmpeg、Python、PyTorch、机器学习模型——都会自动完成配置。

Installation

安装

Pre-built Binary (Recommended)

预构建二进制文件(推荐)

Download the latest release from the Releases page for your platform and run it.
macOS only — remove quarantine after extracting:
bash
xattr -cr Nightingale.app
发布页面下载对应平台的最新版本并运行。
macOS 专属操作 —— 解压后移除隔离限制:
bash
xattr -cr Nightingale.app

Build from Source

从源码构建

Prerequisites:
  • Rust 1.85+ (edition 2024)
  • Linux additionally needs:
    libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev
bash
git clone https://github.com/rzru/nightingale
cd nightingale
前置依赖:
  • Rust 1.85+(2024版本)
  • Linux 系统还需要安装:
    libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev
bash
git clone https://github.com/rzru/nightingale
cd nightingale

Development build

开发构建

cargo build --release
cargo build --release

Run directly

直接运行

./target/release/nightingale
undefined
./target/release/nightingale
undefined

Release Packaging

发布打包

bash
undefined
bash
undefined

Linux / macOS

Linux / macOS

scripts/make-release.sh
scripts/make-release.sh

Windows (PowerShell)

Windows(PowerShell)

powershell -ExecutionPolicy Bypass -File scripts/make-release.ps1

Outputs a `.tar.gz` (Linux/macOS) or `.zip` (Windows) ready for distribution.

---
powershell -ExecutionPolicy Bypass -File scripts/make-release.ps1

输出可分发的 `.tar.gz`(Linux/macOS)或 `.zip`(Windows)文件。

---

First Launch / Bootstrap

首次启动 / 自动配置

On first run, Nightingale downloads and configures:
  • ffmpeg
    binary
  • uv
    (Python package manager)
  • Python 3.10 via uv
  • PyTorch + WhisperX + audio-separator in a virtual environment
  • UVR Karaoke ONNX model and WhisperX
    large-v3
    model
This takes 2–10 minutes depending on network speed. A progress screen is shown in-app.
To force re-bootstrap at any time:
bash
./nightingale --setup
Bootstrap completion is marked by
~/.nightingale/vendor/.ready
.

首次运行时,Nightingale会下载并配置以下内容:
  • ffmpeg
    二进制文件
  • uv
    (Python包管理器)
  • 通过uv安装Python 3.10
  • 虚拟环境中的PyTorch + WhisperX + audio-separator
  • UVR Karaoke ONNX模型和WhisperX
    large-v3
    模型
根据网络速度,这个过程需要2-10分钟,应用内会显示进度界面。
如需强制重新执行配置流程,可运行:
bash
./nightingale --setup
配置完成后会生成标记文件
~/.nightingale/vendor/.ready

CLI Flags

CLI 参数

FlagDescription
--setup
Force re-run of the first-launch bootstrap (re-downloads vendor deps)

参数描述
--setup
强制重新执行首次启动的配置流程(重新下载依赖项)

Keyboard & Gamepad Controls

键盘与游戏手柄控制

Navigation

导航操作

ActionKeyboardGamepad
MoveArrow keysD-pad / Left stick
ConfirmEnterA (South)
BackEscapeB (East) / Start
Switch panelTab
SearchType to filter
操作键盘游戏手柄
移动方向键十字键 / 左摇杆
确认Enter键A键(南方向)
返回Escape键B键(东方向) / 开始键
切换面板Tab键
搜索直接输入关键词过滤

Playback

播放控制

ActionKeyboardGamepad
Pause / ResumeSpaceStart
Exit to menuEscapeB (East)
Toggle guide vocalsG
Guide volume up/down+ / -
Cycle backgroundT
Cycle video flavorF
Toggle microphoneM
Next microphoneN
Toggle fullscreenF11

操作键盘游戏手柄
暂停/继续空格键开始键
返回菜单Escape键B键(东方向)
切换引导人声G键
引导人声音量增减+ / -
切换背景主题T键
切换视频风格F键
切换麦克风M键
切换下一个麦克风N键
切换全屏F11键

Configuration

配置

Main Config

主配置文件

Located at
~/.nightingale/config.json
. Edit directly or via in-app settings.
json
{
  "music_folder": "/home/user/Music",
  "separator": "uvr",
  "guide_vocal_volume": 0.3,
  "background_theme": "plasma",
  "video_flavor": "nature",
  "default_profile": "Alice"
}
separator
options:
"uvr"
(default, preserves backing vocals) |
"demucs"
background_theme
options:
"plasma"
,
"aurora"
,
"waves"
,
"nebula"
,
"starfield"
,
"video"
,
"source_video"
video_flavor
options:
"nature"
,
"underwater"
,
"space"
,
"city"
,
"countryside"
位于
~/.nightingale/config.json
,可直接编辑或通过应用内设置修改。
json
{
  "music_folder": "/home/user/Music",
  "separator": "uvr",
  "guide_vocal_volume": 0.3,
  "background_theme": "plasma",
  "video_flavor": "nature",
  "default_profile": "Alice"
}
separator
可选值:
"uvr"
(默认,保留伴唱) |
"demucs"
background_theme
可选值:
"plasma"
,
"aurora"
,
"waves"
,
"nebula"
,
"starfield"
,
"video"
,
"source_video"
video_flavor
可选值:
"nature"
,
"underwater"
,
"space"
,
"city"
,
"countryside"

Profiles

玩家档案

Located at
~/.nightingale/profiles.json
:
json
{
  "profiles": [
    {
      "name": "Alice",
      "scores": {
        "blake3_hash_of_song": {
          "stars": 4,
          "score": 87250,
          "played_at": "2026-03-18T21:00:00Z"
        }
      }
    }
  ]
}
位于
~/.nightingale/profiles.json
json
{
  "profiles": [
    {
      "name": "Alice",
      "scores": {
        "blake3_hash_of_song": {
          "stars": 4,
          "score": 87250,
          "played_at": "2026-03-18T21:00:00Z"
        }
      }
    }
  ]
}

Pixabay Video Backgrounds (Dev)

Pixabay视频背景(开发环境)

API key is embedded in release builds. For local development, create
.env
at project root:
bash
undefined
发布版本中已内置API密钥。在本地开发环境中,需在项目根目录创建
.env
文件:
bash
undefined

.env

.env

PIXABAY_API_KEY=$PIXABAY_API_KEY

The release script (`make-release.sh`) sources `.env` automatically.

---
PIXABAY_API_KEY=$PIXABAY_API_KEY

发布脚本(`make-release.sh`)会自动读取 `.env` 文件中的配置。

---

Data Storage Layout

数据存储结构

~/.nightingale/
├── cache/              # Per-song stems, transcripts, lyrics (keyed by blake3 hash)
├── config.json         # App settings
├── profiles.json       # Player profiles and per-song scores
├── videos/             # Pre-downloaded Pixabay video backgrounds
├── sounds/             # Sound effects
├── vendor/
│   ├── ffmpeg          # ffmpeg binary
│   ├── uv              # uv binary
│   ├── python/         # Python 3.10
│   ├── venv/           # ML virtualenv (WhisperX, Demucs, audio-separator)
│   ├── analyzer/       # Python analyzer scripts
│   └── .ready          # Bootstrap completion marker
└── models/
    ├── torch/          # Demucs model weights
    ├── huggingface/    # WhisperX large-v3 weights
    └── audio_separator/ # UVR Karaoke ONNX model
Cache keys are blake3 hashes of the source file — re-analysis only triggers if the file changes or is manually invalidated.

~/.nightingale/
├── cache/              # 按歌曲blake3哈希存储的音轨、转录文件、歌词
├── config.json         # 应用设置
├── profiles.json       # 玩家档案与歌曲评分
├── videos/             # 预下载的Pixabay视频背景
├── sounds/             # 音效文件
├── vendor/
│   ├── ffmpeg          # ffmpeg二进制文件
│   ├── uv              # uv二进制文件
│   ├── python/         # Python 3.10
│   ├── venv/           # ML虚拟环境(WhisperX、Demucs、audio-separator)
│   ├── analyzer/       # Python分析脚本
│   └── .ready          # 配置完成标记
└── models/
    ├── torch/          # Demucs模型权重
    ├── huggingface/    # WhisperX large-v3模型权重
    └── audio_separator/ # UVR Karaoke ONNX模型
缓存键为源文件的blake3哈希值,只有当文件修改或手动清除缓存时,才会重新分析歌曲。

Supported File Formats

支持的文件格式

Audio:
.mp3
,
.flac
,
.ogg
,
.wav
,
.m4a
,
.aac
,
.wma
Video:
.mp4
,
.mkv
,
.avi
,
.webm
,
.mov
,
.m4v
Video files: audio track is extracted, vocals separated, original video plays as background automatically.

音频文件:
.mp3
,
.flac
,
.ogg
,
.wav
,
.m4a
,
.aac
,
.wma
视频文件:
.mp4
,
.mkv
,
.avi
,
.webm
,
.mov
,
.m4v
视频文件处理逻辑:提取音频轨道,分离人声,原视频会自动作为背景播放。

Hardware Acceleration

硬件加速

PyTorch backend is auto-detected:
BackendDeviceNotes
CUDANVIDIA GPUFastest; ~2–5 min/song
MPSApple SiliconmacOS; WhisperX alignment falls back to CPU
CPUAnyAlways works; ~10–20 min/song
UVR Karaoke model uses ONNX Runtime with CUDA (NVIDIA) or CoreML (Apple Silicon) automatically.

PyTorch后端会自动检测:
后端设备说明
CUDANVIDIA GPU速度最快;每首歌约2-5分钟
MPSApple Silicon芯片macOS系统;WhisperX对齐步骤会回退到CPU处理
CPU任意设备始终可用;每首歌约10-20分钟
UVR Karaoke模型会自动使用ONNX Runtime的CUDA(NVIDIA)或CoreML(Apple Silicon)后端。

Processing Pipeline

处理流程

Audio/Video file
 UVR Karaoke (ONNX) or Demucs (PyTorch)
       │  vocals.ogg + instrumental.ogg
 LRCLIB API  ──▶  Synced lyrics fetch (if available)
 WhisperX large-v3  ──▶  Transcription + word-level timestamps
 Bevy App (Rust)
   - Plays instrumental audio
   - Synchronized word highlighting
   - Real-time pitch detection & scoring
   - GPU shader / video backgrounds
   - Scoreboards per profile

音频/视频文件
 UVR Karaoke(ONNX)或Demucs(PyTorch)
       │  生成 vocals.ogg + instrumental.ogg
 LRCLIB API  ──▶  获取同步歌词(如果可用)
 WhisperX large-v3  ──▶  生成歌词转录与逐词时间戳
 Bevy应用(Rust语言)
   - 播放伴奏音频
   - 同步高亮歌词
   - 实时音高检测与评分
   - GPU着色器/视频背景
   - 按玩家档案统计评分

Code Patterns

代码示例

Adding a New Background Theme (Bevy System)

添加新的背景主题(Bevy系统)

rust
// In your Bevy plugin, register a new background variant
use bevy::prelude::*;

#[derive(Component)]
pub struct MyCustomBackground;

pub fn spawn_custom_background(mut commands: Commands) {
    commands.spawn((
        MyCustomBackground,
        // ... your background components
    ));
}

pub struct CustomBackgroundPlugin;

impl Plugin for CustomBackgroundPlugin {
    fn build(&self, app: &mut App) {
        app.add_systems(OnEnter(AppState::Playing), spawn_custom_background);
    }
}
rust
// 在Bevy插件中注册新的背景变体
use bevy::prelude::*;

#[derive(Component)]
pub struct MyCustomBackground;

pub fn spawn_custom_background(mut commands: Commands) {
    commands.spawn((
        MyCustomBackground,
        // ... 你的背景组件
    ));
}

pub struct CustomBackgroundPlugin;

impl Plugin for CustomBackgroundPlugin {
    fn build(&self, app: &mut App) {
        app.add_systems(OnEnter(AppState::Playing), spawn_custom_background);
    }
}

Extending Config Deserialization

扩展配置反序列化

rust
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NightingaleConfig {
    pub music_folder: String,
    #[serde(default = "default_separator")]
    pub separator: StemSeparator,
    #[serde(default = "default_guide_volume")]
    pub guide_vocal_volume: f32,
}

#[derive(Debug, Clone, Serialize, Deserialize, Default)]
#[serde(rename_all = "lowercase")]
pub enum StemSeparator {
    #[default]
    Uvr,
    Demucs,
}

fn default_guide_volume() -> f32 { 0.3 }
fn default_separator() -> StemSeparator { StemSeparator::Uvr }

// Load config
fn load_config() -> NightingaleConfig {
    let path = dirs::home_dir()
        .unwrap()
        .join(".nightingale/config.json");
    let raw = std::fs::read_to_string(&path).unwrap_or_default();
    serde_json::from_str(&raw).unwrap_or_default()
}
rust
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NightingaleConfig {
    pub music_folder: String,
    #[serde(default = "default_separator")]
    pub separator: StemSeparator,
    #[serde(default = "default_guide_volume")]
    pub guide_vocal_volume: f32,
}

#[derive(Debug, Clone, Serialize, Deserialize, Default)]
#[serde(rename_all = "lowercase")]
pub enum StemSeparator {
    #[default]
    Uvr,
    Demucs,
}

fn default_guide_volume() -> f32 { 0.3 }
fn default_separator() -> StemSeparator { StemSeparator::Uvr }

// 加载配置
fn load_config() -> NightingaleConfig {
    let path = dirs::home_dir()
        .unwrap()
        .join(".nightingale/config.json");
    let raw = std::fs::read_to_string(&path).unwrap_or_default();
    serde_json::from_str(&raw).unwrap_or_default()
}

Triggering Re-analysis Programmatically

程序化触发重新分析

rust
use std::fs;
use std::path::PathBuf;

/// Remove cached stems/transcript for a song to force re-analysis
fn invalidate_song_cache(song_hash: &str) {
    let cache_dir = dirs::home_dir()
        .unwrap()
        .join(".nightingale/cache")
        .join(song_hash);

    if cache_dir.exists() {
        fs::remove_dir_all(&cache_dir)
            .expect("Failed to remove cache directory");
        println!("Cache invalidated for {}", song_hash);
    }
}
rust
use std::fs;
use std::path::PathBuf;

/// 删除歌曲的缓存音轨/转录文件,强制重新分析
fn invalidate_song_cache(song_hash: &str) {
    let cache_dir = dirs::home_dir()
        .unwrap()
        .join(".nightingale/cache")
        .join(song_hash);

    if cache_dir.exists() {
        fs::remove_dir_all(&cache_dir)
            .expect("Failed to remove cache directory");
        println!("Cache invalidated for {}", song_hash);
    }
}

Computing a Song's Blake3 Hash (for Cache Lookup)

计算歌曲的Blake3哈希值(用于缓存查找)

rust
use blake3::Hasher;
use std::fs::File;
use std::io::{BufReader, Read};

fn hash_file(path: &std::path::Path) -> String {
    let file = File::open(path).expect("Cannot open file");
    let mut reader = BufReader::new(file);
    let mut hasher = Hasher::new();
    let mut buf = [0u8; 65536];
    loop {
        let n = reader.read(&mut buf).unwrap();
        if n == 0 { break; }
        hasher.update(&buf[..n]);
    }
    hasher.finalize().to_hex().to_string()
}
rust
use blake3::Hasher;
use std::fs::File;
use std::io::{BufReader, Read};

fn hash_file(path: &std::path::Path) -> String {
    let file = File::open(path).expect("Cannot open file");
    let mut reader = BufReader::new(file);
    let mut hasher = Hasher::new();
    let mut buf = [0u8; 65536];
    loop {
        let n = reader.read(&mut buf).unwrap();
        if n == 0 { break; }
        hasher.update(&buf[..n]);
    }
    hasher.finalize().to_hex().to_string()
}

Profile Score Update Pattern

玩家档案评分更新示例

rust
use serde::{Deserialize, Serialize};
use std::collections::HashMap;

#[derive(Debug, Serialize, Deserialize)]
pub struct SongScore {
    pub stars: u8,
    pub score: u32,
    pub played_at: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct Profile {
    pub name: String,
    pub scores: HashMap<String, SongScore>, // key = blake3 hash
}

fn update_score(profile: &mut Profile, song_hash: &str, stars: u8, score: u32) {
    profile.scores.insert(song_hash.to_string(), SongScore {
        stars,
        score,
        played_at: chrono::Utc::now().to_rfc3339(),
    });
}

rust
use serde::{Deserialize, Serialize};
use std::collections::HashMap;

#[derive(Debug, Serialize, Deserialize)]
pub struct SongScore {
    pub stars: u8,
    pub score: u32,
    pub played_at: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct Profile {
    pub name: String,
    pub scores: HashMap<String, SongScore>, // 键 = blake3哈希值
}

fn update_score(profile: &mut Profile, song_hash: &str, stars: u8, score: u32) {
    profile.scores.insert(song_hash.to_string(), SongScore {
        stars,
        score,
        played_at: chrono::Utc::now().to_rfc3339(),
    });
}

Troubleshooting

故障排查

Bootstrap Fails / Stuck on Setup Screen

配置流程失败 / 卡在设置界面

bash
undefined
bash
undefined

Force re-bootstrap

强制重新执行配置

./nightingale --setup
./nightingale --setup

Or manually remove the vendor directory and restart

或手动删除vendor目录后重启

rm -rf ~/.nightingale/vendor ./nightingale
undefined
rm -rf ~/.nightingale/vendor ./nightingale
undefined

Song Analysis Hangs or Errors

歌曲分析停滞或报错

bash
undefined
bash
undefined

Check the analyzer venv is healthy

检查分析器虚拟环境是否正常

~/.nightingale/vendor/venv/bin/python -c "import whisperx; print('ok')"
~/.nightingale/vendor/venv/bin/python -c "import whisperx; print('ok')"

Re-bootstrap if broken

若环境损坏则重新执行配置

./nightingale --setup
undefined
./nightingale --setup
undefined

macOS "App is damaged" Error

macOS 提示“应用已损坏”错误

bash
xattr -cr Nightingale.app
bash
xattr -cr Nightingale.app

GPU Not Being Used

GPU未被使用

  • NVIDIA: Ensure CUDA drivers are installed and
    nvidia-smi
    shows your GPU.
  • Apple Silicon: MPS is used automatically on macOS with Apple Silicon; WhisperX alignment falls back to CPU (normal behavior).
  • Check
    ~/.nightingale/vendor/venv
    — if PyTorch installed the CPU-only build, re-bootstrap after installing CUDA drivers.
  • NVIDIA显卡: 确保已安装CUDA驱动,且
    nvidia-smi
    能识别你的GPU。
  • Apple Silicon芯片: macOS系统会自动使用MPS后端;WhisperX对齐步骤回退到CPU处理属于正常现象。
  • 检查
    ~/.nightingale/vendor/venv
    目录:如果PyTorch安装的是CPU版本,安装CUDA驱动后重新执行配置流程。

Cache Corruption / Wrong Lyrics

缓存损坏 / 歌词显示错误

bash
undefined
bash
undefined

Find the blake3 hash of your file (build a small tool or use b3sum)

获取文件的blake3哈希值(可使用b3sum工具或自行编写小工具)

b3sum /path/to/song.mp3
b3sum /path/to/song.mp3

Remove that song's cache

删除该歌曲的缓存

rm -rf ~/.nightingale/cache/<hash>

Then re-open the song in Nightingale to re-analyze.
rm -rf ~/.nightingale/cache/<hash>

然后重新在Nightingale中打开该歌曲以重新分析。

Audio Playback Issues (Linux)

Linux系统音频播放问题

Ensure ALSA/PulseAudio/PipeWire is running. Install missing deps:
bash
sudo apt install libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev
确保ALSA/PulseAudio/PipeAudio服务正在运行,安装缺失的依赖:
bash
sudo apt install libasound2-dev libudev-dev libwayland-dev libxkbcommon-dev

Video Backgrounds Not Loading

视频背景无法加载

Video backgrounds are pre-downloaded during setup via the Pixabay API. For development builds, ensure
.env
contains a valid
PIXABAY_API_KEY
. If videos are missing in a release build, run
--setup
to re-trigger the download.

视频背景会在配置阶段通过Pixabay API预下载。开发环境中需确保
.env
文件包含有效的
PIXABAY_API_KEY
。发布版本中若视频缺失,可运行
--setup
重新触发下载。

Platform Targets

支持的平台目标

PlatformTarget Triple
Linux x86_64
x86_64-unknown-linux-gnu
Linux aarch64
aarch64-unknown-linux-gnu
macOS ARM
aarch64-apple-darwin
macOS Intel
x86_64-apple-darwin
Windows x86_64
x86_64-pc-windows-msvc
Cross-compile with:
bash
rustup target add aarch64-unknown-linux-gnu
cargo build --release --target aarch64-unknown-linux-gnu

平台目标三元组
Linux x86_64
x86_64-unknown-linux-gnu
Linux aarch64
aarch64-unknown-linux-gnu
macOS ARM
aarch64-apple-darwin
macOS Intel
x86_64-apple-darwin
Windows x86_64
x86_64-pc-windows-msvc
交叉编译命令:
bash
rustup target add aarch64-unknown-linux-gnu
cargo build --release --target aarch64-unknown-linux-gnu

License

许可证

GPL-3.0-or-later. See LICENSE.
采用GPL-3.0-or-later许可证,详情见LICENSE