本地语音转文字速度与易用性不足

原话 · @Anonymous

Hi all. I am Shahir. This was my experiment. First of all - Thanks to whoever posted this on HN. It came as a surprise. Just to clarify - This started as just an experiment to see if we can use the fact that text and audio share a vector space in Embedding Gemma 2 to convert audio to text. Had to hack around a few things, but, it worked. Speed was just a side benefit. Was not aiming for it. The accuracy is good, but not in the range of cloud models, but decent enough for everyday use, and this is just a couple of days work. The WER can improve a lot with more training on varied data. Will try that next. I will opensource the whole thing soon. Might need a few days though. Also, releasing the TTS version for this soon. Happy to answer any questions you have.

原话 · @Anonymous

For people interested in super quick models: "One cool trick" I like for running those kinds of models very quickly on macOS is converting them to CoreML so they run in the Neural Engine. I had amazing success with Spleeter, Demucs, MVSEP-MDX23-music-separation, DeepFilterNet, Whisper, Hunyuan3D for 3D mesh generation. Plus a lot of others for both work and leisure. I also convert the code to Swift without dependencies, but that's a personal preference. The runtime on some of them went sometimes from 60s to 10s. And since it's just Swift, I can just give them to non-technical co‑workers and they don't have to setup a whole Python environment. If you don't wanna do this by hand, this is something that existing LLM models are good at. Even Luna can do it without much fuss.

原话 · @Anonymous

I recently swapped to using Parakeet v3 on my old M1 Macbook Air, using MacWhisper. It was so fast I thought it had failed. As in, by the time I clicked "transcribe" and looked at my screen again a few seconds later the transcript had finished. Absolutely insanely fast.