[debug] support flow cache, for sharper tts_mel output #410
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boji123
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
我是柏基
#379 问题2的解决方案
flowmatching中的z和mu，跨chunk时对于每个index不是定值，是导致衔接处频谱模糊的因素之一（本质是flow的attention context问题，无解）
图中是flow的tts_mel输出，用于对比上下文及频谱模糊的问题
大图1列：不带cache；2列：带cache
小图左：前chunk最后34；中：（前+后）/2；右：后chunk开头34
可以发现带cache的，tts_mel频谱更清晰
*由于后续的mel fade、hifigan cache、speech fade的挽救，该项虽然更本质，但最终听感提升概率较小，多测测的确是有badcase得到改善的