fix(omnivoice): pass duration to model.generate instead of truncating#2011
Open
Arturogpj wants to merge 1 commit into
Open
fix(omnivoice): pass duration to model.generate instead of truncating#2011Arturogpj wants to merge 1 commit into
Arturogpj wants to merge 1 commit into
Conversation
OmniVoice's model.generate(duration=X) is the upstream-supported way (k2-fsa/OmniVoice) to fix output length: it sets target_lens = X * frame_rate and computes a speed ratio so the model fits the text into that duration. The WanGP pipeline never passed duration= to model.generate(), and instead generated at the model's natural pace and truncated the tail afterwards. Lowering the duration slider therefore returned the end of a naturally-paced generation instead of compressed speech. Fix: give each segment a proportional duration budget (total minus pauses, distributed by each segment's estimated speech seconds) and pass it to model.generate(duration=...). Pauses count toward the request so the final file length matches the request. A small safety trim handles post-processing silence overshoot only — it is never a content cut. Auto mode (duration_seconds=0/None) is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a user lowers the duration slider for OmniVoice, WanGP returns a slice of the end of a naturally-paced generation instead of compressing the speech to fit the requested length.
Root cause
OmniVoice's
model.generate(duration=X)is the upstream-supported way to fix output length (seek2-fsa/OmniVoice— "duration: fixed output duration in seconds. Overrides speed when set."). It setstarget_lens = X * frame_rateand computes aspeedratio so the model fits the text into that duration.WanGP's
OmniVoicePipelinenever passedduration=tomodel.generate()and instead generated at the model's natural pace and truncated the tail afterwards (pipeline.py_generate_segments):So lowering the duration just cut off whatever the model hadn't had time to produce — not compression.
Fix
In
app/models/TTS/omnivoice/pipeline.py:_run_segmentnow accepts asegment_durationparam and passes it toself.model.generate(duration=segment_duration). The model now fits each segment's text into its allotted seconds — true compression via the upstream API._generate_segmentsnow distributes the requested duration across segments proportionally to their estimated speech seconds (using the existing_estimate_text_secondshelper). Inter-segment pauses count toward the request, so the final file length ≈ the requested duration. The old post-hoc truncation is replaced with a small safety trim (a few hundred ms) that only guards against post-processing silence overshoot — never a content cut.Auto mode (
duration_seconds=0/None) is unchanged —segment_duration=Noneis passed to every segment, preserving the previous behavior.Verification
Tested locally via the Wan2GP UI:
duration=0(auto) → unchanged behavior. ✅Scope
Single file, ~36 lines added, 2 removed. No changes to
modeling_omnivoice.py, the handler, or any other TTS pipeline.