I believe in SOTA models over custom ones

Nah, fine-tuned models can be probability calibrated, I think we're going to understand someday that the genius of 'predict the next character' LLMs is that they have a deranged ability to reason about uncertain events and that deranged ability to reason about uncertain events is central to the human 'language instinct'.