Reducing VAE and Attention Compute Costs in Text‑to-Image Models
Problem
Current text‑to‑image models incur high training and inference costs due to attention and VAE compute, limiting affordability for prosumers. The author seeks to compress context windows, replace or improve VAEs to reduce expenses and enable larger models.
Pain points
- Attention mechanism makes tokens dense, resulting in high compute cost
- VAE is a bottleneck; replacing or improving it can reduce cost
- Desire to integrate with ComfyUI for direct user access
Questions to verify
- Would you be willing to pay extra for such a model?
- Do you use ComfyUI in your workflow and would you need a plugin for this model?
- How interested are you in alternative VAE approaches such as Iris‑3B?
Evidence
quote · @Anonymous
Video models are incredibly token dense (e.g. 110K tokens for a several second clip). If we can condense that context window more aggressively, we can train bigger models for a lot less $$ and offer them to prosumers at reasonable price points (unlike the big models today like Seedance, which cost an arm and a leg to run).
quote · @Anonymous
I wonder how much of that is due to the use of a suboptimal VAE? The same optimisations could be applied to it, and my intuition tells me that your total compute spend would be even more optimal if you retrained the VAE yourself with a better approach (esp. ensuring translation and rotation invariance, + ability to rescale/blur the latents).
quote · @Anonymous
Is there a way to use this in ComfyUI today?
Disclaimer: this card is distilled by AI from community discussions as “pain point evidence”. It is a starting point for need analysis and validation, not a “validated market need”. AI has analytical limits; please open the original discussion above to manually verify before key decisions.