Vlad Sokolovskii

AI engineer building production software with AI

All writing

Telegram post

Open-weight models and the future of AI infrastructure

  • #ai

My first commercial experience was an internship at Red Hat.

Back then, it felt counterintuitive that a company could make so much money from open-source products such as RHEL and OpenShift.

My small-town brain kept asking: “Why would customers pay for software they can download and run on their own servers?” Eventually, of course, I understood that the real value for a business is not the program itself. It is support, a clear SLA, updates, and security fixes. Those have always been the most expensive part of any software.

But why not buy a proprietary solution instead? Why pay a third company for an open-source Linux distribution? Aren’t there other decent products on the market?

There are three answers:

  1. Transparency. Your security team can audit the code and confirm that there are no backdoors or hidden telemetry. If your engineers find a bug, they do not have to wait forever for the vendor to fix it: anyone can open a PR and propose a change.
  2. A shared standard. It is much easier to hire people who already know the technology. You do not have to build an entire team around one vendor’s closed SDK.
  3. Less vendor lock-in. Yes, some lock-in still exists. But if the supplier seriously lets you down, replacing it is much easier than walking away from an annual proprietary software licence.

With your permission, I will leave aside the fact that Linux is simply better for business than any closed-source OS.

Where am I going with this?

Chinese — and not only Chinese — open-weight models will take a huge share of the emerging market. Not because every one of them is better than Claude or GPT, but because even large Western companies, never mind companies in sanctioned markets, are reaching the point where it makes more sense to run a model inside their own perimeter, tune it on their data, control the cost, and avoid depending on a single API provider.

The quality gap is also shrinking. Kimi K3 already appears to be competitive with the leading models, and I doubt this progress comes only from distilling American systems. China builds the fastest trains, drones, and electric cars. Why should training AI models be uniquely impossible, and why do some Americans assume all of that progress is merely a derivative of their own work? 😁

Fun fact: open weights and open source are not quite the same thing. Publicly available weights do not guarantee that the licence imposes no restrictions.

Companies that focus on serving these models efficiently and build a proper agent-style harness around them will make a fortune. I do not mean “a chat window with a model selector.” I mean the whole layer people pay Red Hat for: deployment, access control, observability, evals, fine-tuning, support, and an SLA. The weights themselves will, with near certainty, become a commodity.

Does that mean OpenAI and Anthropic are doomed? Of course not. Outside a relatively small group of enthusiasts, the entire B2C market will keep using proprietary AI products because they are packaged better and usually provide the strongest experience out of the box. Open source has never been particularly strong in consumer software. Some business customers will choose OSS, but many will stay with the convenient Anthropic or OpenAI offering.

The pie is large enough for everyone. Still, businesses will increasingly think twice before going all-in on closed solutions.

It is great to see new players entering the open-source scene. Thinking Machines, founded by OpenAI’s former CTO, released Inkling, its first open-weight model. The company openly says it is not the most powerful model on the market and sells the surrounding layer instead: tuning through Tinker, an API, and infrastructure. I am sure we will see more examples like this. Training your own model is optional; you can always start with one from China.

Now we wait for Sutskever to emerge from hiding. I am convinced his models will be open source. If they are not, you have my permission to spit on the top of my head when we meet 🤠

This continues the argument from my post about a long-term career bet 👉 here. I explained why local inference, serving, and AI infrastructure are skills I would start developing now.

Viva la open source! 🐧

AI digest

Weekly newsletter

New articles, tools, and thoughts on where AI development is going. Only things I have tested myself.

AI DIGEST

New articles, tools, and thoughts on where AI development is going. Only things I have tested myself.

You can unsubscribe at any time