基于文本和参考音频生成音效的AI模型
问题概述
The user seeks an AI solution that can generate new sound effects by providing a reference audio clip and a text description, but no commercial product currently offers this capability. Existing text‑to‑audio tools such as ElevenLabs SFX only support text‑to‑audio without reference, and Stable Audio 3 produces unsatisfactory results. Community members have suggested various workarounds, yet none fully meet the need for accurate, reference‑guided sound generation.
痛点标签
- No commercial AI model accepts both a reference sound and text instruction to generate new audio.
- Existing text‑to‑audio tools (e.g., ElevenLabs SFX) do not support reference audio, limiting control over generated sound.
- Current models like Stable Audio 3 yield poor results for generating sound effects from reference audio.
待验证问题
- Do you have a specific type of sound (e.g., Foley, UI sounds) you need to generate?
- Would you prefer an open‑source model you can run locally, or a hosted SaaS solution?
- Are you looking to generate a single sound per text+reference pair, or a series of variations?
证据
原话
I've been having a hard time finding a solution. Is there really no commercialized model that I can feed in a reference sound and text instruction and get another sound out?
原话
Tried elevenlabs SFX but it's only text->audio and without the reference, it's hard to guide with just text and recreate a sound with any accuracy.
原话
I tried the Stable Audio 3 model which is supposed to do what I want but the results were terrible.
原话
Even though they're technically trained for music, it might be worth testing Suno/Lyria to see if they're able to do this.
原话
Plenty of models take in text + audio and spits out audio. It's the format of most newer generation accent conversion/voice cloning models.
原话
Hi! I’m unsure if this is what you were looking for, QWEN3 TTS has a "clone" feature. You can use the local version, give it a voice reference and text and it will read the text with your instructions with the sample voice you selected.
免责声明：本卡片由 AI 从社区讨论中提炼，属于「痛点证据」，是需求分析与验证的起点，不等同于「已验证的市场需求」。AI 存在分析局限，关键决策前请打开上方原始讨论人工复核。