降低文本‑图像生成模型的 VAE 与注意力计算成本
问题概述
当前文本‑图像生成模型在训练和推理时因注意力机制和 VAE 的高计算开销而成本高昂，导致难以向普通用户提供低价服务。作者希望通过压缩上下文窗口、替代或改进 VAE 来降低费用并实现更大模型的可扩展性。
痛点标签
- 注意力机制导致 token 密集、计算成本高
- VAE 是瓶颈，替代或改进可降低成本
- 希望集成到 ComfyUI 以便用户直接使用
待验证问题
- 您是否愿意为此类模型支付额外费用？
- 您在现有工作流中使用过 ComfyUI，是否需要此类模型的插件？
- 您对替代 VAE 的方法（如 Iris‑3B）的兴趣程度如何？
证据
原话 · @Anonymous
Video models are incredibly token dense (e.g. 110K tokens for a several second clip). If we can condense that context window more aggressively, we can train bigger models for a lot less $$ and offer them to prosumers at reasonable price points (unlike the big models today like Seedance, which cost an arm and a leg to run).
原话 · @Anonymous
I wonder how much of that is due to the use of a suboptimal VAE? The same optimisations could be applied to it, and my intuition tells me that your total compute spend would be even more optimal if you retrained the VAE yourself with a better approach (esp. ensuring translation and rotation invariance, + ability to rescale/blur the latents).
原话 · @Anonymous
Is there a way to use this in ComfyUI today?
免责声明：本卡片由 AI 从社区讨论中提炼，属于「痛点证据」，是需求分析与验证的起点，不等同于「已验证的市场需求」。AI 存在分析局限，关键决策前请打开上方原始讨论人工复核。