# ComfyUI Audio Segment Plugin (ComfyUI-AudioSegment)

[中文说明 (Chinese Readme)](Readme-zh.md)

This is a custom node suite for ComfyUI designed for advanced audio processing. It allows users to segment a long audio file into multiple shorter clips in two ways:
1.  **Silence-based**: Intelligently splits the audio into a list of shorter clips based on silence detection and a set maximum duration.
2.  **Paragraph-based**: Accurately aligns and splits the audio based on a provided reference text using AI-powered speech recognition.

## Features

This plugin suite includes five nodes that work together:

### 1. Audio Segment By Paragraph
This is the new core feature node. It accurately segments audio by aligning it with user-provided text paragraphs using a Whisper ASR model.

-   **Inputs**:
    -   `audio`: The original ComfyUI audio object.
    -   `text`: A multiline string where each line represents a paragraph. The node splits paragraphs based on newlines.
    -   `model`: A dropdown menu to select the Whisper model to use. It supports all standard models (e.g., `base`, `large-v3`), local `.pt` files, and Hugging Face model folders located in `ComfyUI/models/stt/whisper/`. If a selected model is not found locally, it will be downloaded automatically.
-   **Outputs**:
    -   `AudioClip list`: A custom list of `AudioClip` objects. Each `AudioClip` contains the `audio` (as a ComfyUI audio object), `startTime` (in milliseconds), `endTime` (in milliseconds), and `text` (the corresponding original text paragraph).
    -   `size`: The number of `AudioClip` objects in the list.
    -   `srt`: An SRT-formatted subtitle string generated from the segmentation results.

### 2. Select AudioClip From List
This node allows you to select a specific `AudioClip` from the list generated by `Audio Segment By Paragraph` for further processing.

-   **Inputs**:
    -   `audio_clip_list`: The list of `AudioClip` objects from the `Audio Segment By Paragraph` node.
    -   `index`: The index of the `AudioClip` to select (starting from 0).
-   **Outputs**:
    -   `audio`: The ComfyUI audio object from the selected `AudioClip`.
    -   `text`: The text string from the selected `AudioClip`.

### 3. Audio Segmenter (Silence-based)
The original core node, which segments audio based on silence.

-   **Inputs**:
    -   `audio`: The original ComfyUI audio object.
    -   `max_length_s`: The maximum length of each output clip in seconds.
    -   `silence_thresh_db`: The silence threshold in decibels (dB). Anything below this volume is considered silence.
-   **Outputs**:
    -   `audio_list`: A list of individual audio clips.
    -   `count`: The number of audio clips in the list.

### 4. Select Audio From List
This node allows you to select a specific audio clip from the list generated by `Audio Segmenter`.

-   **Inputs**:
    -   `audio_list`: The audio list from the `Audio Segmenter` node.
    -   `index`: The index of the audio clip to select (starting from 0).
-   **Outputs**:
    -   `AUDIO`: The single audio clip at the specified index.

### 5. Audio List to Batch
This node converts a list of audio clips back into the standard ComfyUI batch audio format, making it compatible with other nodes that require batch input (like ASR models).

-   **Inputs**:
    -   `audio_list`: The audio list from the `Audio Segmenter` node.
-   **Outputs**:
    -   `AUDIO`: A batched audio object containing all clips from the list. All clips are padded to match the length of the longest clip.

## Installation

1.  **Navigate to the `custom_nodes` directory**:
    Open a terminal and go to the `custom_nodes` folder in your ComfyUI installation directory.
    ```bash
    cd path/to/ComfyUI/custom_nodes/
    ```

2.  **Clone the repository**:
    Clone this repository into the `custom_nodes` directory.
    ```bash
    git clone git@gitee.com:dreamidea/comfyui-audio-segment.git
    ```

3.  **Install dependencies**:
    Navigate into the plugin directory and install the required libraries using the `pip` from your ComfyUI environment.
    ```bash
    cd comfyui-audio-segment
    pip install -r requirements.txt
    ```
    *(Note: If you are running ComfyUI in a virtual environment (e.g., venv or conda), make sure to activate it first.)*

4.  **Restart ComfyUI**:
    Completely shut down and restart ComfyUI. You will then find the new nodes under the "Audio/Segmentation" category.

## Implementation Details

### Silence-Based Segmentation
The original segmentation logic is in `audio_segmenter.py` and relies on the `pydub` library. It first splits the audio stream based on silence detection and then intelligently merges or splits the resulting chunks to ensure they are as close as possible to the user-defined maximum length.

### Paragraph-Based Segmentation
The new core logic resides in `audio_paragraph_segmenter.py` and follows this workflow:

1.  **Speech Recognition (ASR)**:
    -   It uses the `openai-whisper` library for high-accuracy speech recognition, complete with word-level timestamps.
    -   It supports dynamic model loading for various Whisper models, including standard versions (e.g., `base`, `large-v3`), local `.pt` files, and Hugging Face models stored in `ComfyUI/models/stt/whisper/`. The system automatically downloads models if they are not found locally.

2.  **Text Alignment**:
    -   To handle discrepancies between the user's reference text and the ASR results (e.g., homophones, traditional/simplified Chinese, missing words), the plugin implements a robust text alignment algorithm.
    -   The algorithm first uses the `opencc-python-reimplemented` library to normalize both the reference text and the ASR output to simplified Chinese.
    -   It then uses Python's `difflib.SequenceMatcher` to find all matching blocks between the two texts. By calculating a cumulative similarity score, it can tolerate scattered ASR errors and accurately determine the start and end times for each paragraph.

3.  **Audio Slicing and Data Encapsulation**:
    -   Based on the timestamps from the alignment algorithm, `pydub` is used to precisely slice the original audio.
    -   Each audio slice is then encapsulated into a custom `AudioClip` object, along with its corresponding text paragraph and start/end times, making it easy to use with downstream nodes like `Select AudioClip From List`.