Voxtral 3

Been using it to implement a mixed audiobook player + ebook reader that uses Parakeet to sync the narration with its position in the text. Runs fast enough even if CPU-only

However, I found Qwen ASR and QwenTTS to be quite good, and not too large. Also https://github.com/kyutai-labs/pocket-tts was also great (but only a few languages)

It's small enough to be run on modern phones realtime, I use it for listening to PDF's/EPUB as Audiobooks

https://huggingface.co/hexgrad/Kokoro-82M

https://x.com/GradiumAI/status/2112563128487514512?s=20

Not because of the models, but because it comes in a nice (easy-to-set-up) package, and the latency was lower than I expected with such a setup.

What does this mean then?

This method aligns the transcript of a 60 hour book in 20 minutes on my GPU using Parakeet CTC 0.6B which is english only, but there's similarly sized models in other languages, the models don't need to be that good at transcribing to generate good alignments.

It can sync from audio in the video or from another srt. Target srt can be in different language than reference.

EDIT: Two forks, but I haven't tried them as I'm still using the original as it still works well even though it has been archived.

https://github.com/Sefzz15/subsync/releases

https://github.com/blaaaakuu/subsync/releases

One thing to watch for, make sure it never tries to 2 subtitles at the same time or they will stack in a weird way. It's easy to avoid that but my first pass on a few episodes screwed up the subtitles.

Also, you almost certainly want to extract the subtitles for this work, or at least that's what I did. A combo of pulling out the subtitles in the video file and using OpenSubtitles (VIP) to clean up my media.

Not an answer to your current query, but my first published rust crate shortly after the language hit 1.0 was a tool to both calculate and correct linear drift, long before the era of LLMs or quality audio recognition. It might be fun to tackle this again with the new tech we now have at hand!

Srt-resync: https://github.com/mqudsi/srtresync

Download links and a bit of write up here - https://news.ycombinator.com/item?id=48528779#48535207

Honestly you can probably build your own quite easily. I put together a plex player for firetv that does on device subtitling for whatever you're watching. I got sick of having to find subtitles.

Imagine the same subtitles working across different encodings and (minor) edits of the same video…

> Weights drop end of this month.

> (..) and it continues to improve rapidly as we refine it.

There’s also Shieldstral[1] (2 months ago) and Timestral (week ago; already mentioned).

It’s a shame Voxtral isn’t open weight though.

[1] https://mistral.ai/news/mistral-large-4/ [2] https://news.ycombinator.com/item?id=49171268

[0] https://huggingface.co/collections/mistralai/timestral

I'm not happy about it, but I can see why they would. I want to incorporate some of these things locally myself.

But I'm not convinced. With today's models, is it really that much easier to defraud an AI agent than a human?

I do think a lot of people are much more WILLING to try and defraud an AI agent than a person, so maybe that's the difference. There's a parallel there to people being much more willing to mess with Waymos. I think both are a huge shame.

Every minute, 500 hours of video are uploaded to YouTube.

Even Google can't afford to run the biggest baddest model on every single one of those hours.

For example the sentence may be "They decided to rewrite the application in Rust", and the subtitle be "They decided to rewrite the application in rats". They made a single mistake, and it is on the most important word. I guess it can be explained by the fact that these key words tend to be uncommon while the rest of the sentence just flows naturally, that is, you barely need to listen, you can guess most of it.

It worries me a bit because as models get better, they may come up with more and more consistent, but wrong transcriptions. Like a typical LLM hallucination, so confidently wrong that you don't question it.

It mixed French, English, Japanese and Hindi in maybe 3 lines of text and the French and English parts had a subtle erotic touch to them

The real question to ask is why must we after 40 years of being subject to that horrible company still suffer it's existence upon the face of the earth? Corporate corruption? I have no clue honestly but I wish I could give you a hug for having to deal with it.

Producing the plugin is easy. It’s getting your corporate IT to approve the connection (and potentially legal + data governance). Especially if you work with sensitive data (either your company in general or your role/team specifically)

1. Bulgarian

2. Croatian

3. Czech

4. Danish

5. Estonian

6. Finnish

7. Greek

8. Irish

9. Latvian

10. Lithuanian

11. Maltese

12. Slovak

13. Slovenian

Be happy it was written by a human!