Why do we see no companies offering to deploy local LLMs?
It seems an obvious idea but we cannot find anyone offering b2b local LLMs setup to corporations that doesn't want to use external ones & keep their data in-house.. ie airlines, governments, attorneys, accountants..
I see large companies leveraging their large technical teams and external consulancies and technology providers to build their own e.g. banks on IBM. And I think for small companies it's impossible to beat the value for money a Saas type solution provides and the big APIs like ChatGPT and Claude. For the companies inbetween I'd imageine the risk to reward of picking a stack might be too high right now given how fast everything is moving. There is probably room for someone to do it next year but I don't get the sense that the market is ready for it just yet.
My understanding, and I could be wrong here so I’m happy to be corrected, is that AI companies like OpenAI are perfectly capable of operating an instance of their product that doesn’t send data back to the source.
They discuss this on the Enterprise page:
Protect your data with enterprise-grade privacy, security, and deployment tools You own and control your business data in ChatGPT Enterprise. We do not train on your business data or conversations, and our models don’t learn from your usage. ChatGPT Enterprise is also SOC 2 compliant and all conversations are encrypted in transit and at rest. Our new admin console lets you manage team members easily and offers domain verification, SSO, and usage insights, allowing for large-scale deployment into enterprise. See our privacy page and our Trust Portal (opens in a new window) for more details on how we treat your data.
https://openai.com/index/introducing-chatgpt-enterprise/
So it’s very likely that most companies find this reassuring enough and therefore don’t necessarily care too much about running models locally. Anyone that needs security greater than this probably has the resources to develop AI capabilities in-house.
IMHO, local LLMs still can't compete on quality and speed.
https://azure.microsoft.com/en-us/products/ai-services/opena...
https://openai.com/index/introducing-chatgpt-enterprise/
I think this is a the value prop for AWS Bedrock (no affiliation)
AFAIK You get a managed instance where the model data doesn’t get sent to the model provider, but you pay PAYG rates.
When I last looked it seemed like cloud costs were much higher.