Launch HN: Onyx (YC W24) – Open-source chat UI
252 points by Weves 8 days ago
252 points by Weves 8 days ago
Hey HN, Chris and Yuhong here from Onyx (https://github.com/onyx-dot-app/onyx). We’re building an open-source chat that works with any LLM (proprietary + open weight) and gives these LLMs the tools they need to be useful (RAG, web search, MCP, deep research, memory, etc.).
Demo: https://youtu.be/2g4BxTZ9ztg
Two years ago, Yuhong and I had the same recurring problem. We were on growing teams and it was ridiculously difficult to find the right information across our docs, Slack, meeting notes, etc. Existing solutions required sending out our company's data, lacked customization, and frankly didn't work well. So, we started Danswer, an open-source enterprise search project built to be self-hosted and easily customized.
As the project grew, we started seeing an interesting trend—even though we were explicitly a search app, people wanted to use Danswer just to chat with LLMs. We’d hear, “the connectors, indexing, and search are great, but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them”.
Many users would add RAG, agents, and custom tools later, but much of the usage stayed ‘basic chat’. We thought: “why would people co-opt an enterprise search when other AI chat solutions exist?”
As we continued talking to users, we realized two key points:
(1) just giving a company secure access to an LLM with a great UI and simple tools is a huge part of the value add of AI
(2) providing this well is much harder than you might think and the bar is incredibly high
Consumer products like ChatGPT and Claude already provide a great experience—and chat with AI for work is something (ideally) everyone at the company uses 10+ times per day. People expect the same snappy, simple, and intuitive UX with a full feature set. Getting hundreds of small details right to take the experience from “this works” to “this feels magical” is not easy, and nothing else in the space has managed to do it.
So ~3 months ago we pivoted to Onyx, the open-source chat UI with:
- (truly) world class chat UX. Usable both by a fresh college grad who grew up with AI and an industry veteran who’s using AI tools for the first time.
- Support for all the common add-ons: RAG, connectors, web search, custom tools, MCP, assistants, deep research.
- RBAC, SSO, permission syncing, easy on-prem hosting to make it work for larger enterprises.
Through building features like deep research and code interpreter that work across model providers, we've learned a ton of non-obvious things about engineering LLMs that have been key to making Onyx work. I'd like to share two that were particularly interesting (happy to discuss more in the comments).
First, context management is one of the most difficult and important things to get right. We’ve found that LLMs really struggle to remember both system prompts and previous user messages in long conversations. Even simple instructions like “ignore sources of type X” in the system prompt are very often ignored. This is exacerbated by multiple tool calls, which can often feed in huge amounts of context. We solved this problem with a “Reminder” prompt—a short 1-3 sentence blurb injected at the end of the user message that describes the non-negotiables that the LLM must abide by. Empirically, LLMs attend most to the very end of the context window, so this placement gives the highest likelihood of adherence.
Second, we’ve needed to build an understanding of the “natural tendencies” of certain models when using tools, and build around them. For example, the GPT family of models are fine-tuned to use a python code interpreter that operates in a Jupyter notebook. Even if told explicitly, it refuses to add `print()` around the last line, since, in Jupyter, this last line is automatically written to stdout. Other models don’t have this strong preference, so we’ve had to design our model-agnostic code interpreter to also automatically `print()` the last bare line.
So far, we’ve had a Fortune 100 team fork Onyx and provide 10k+ employees access to every model within a single interface, and create thousands of use-case specific Assistants for every department, each using the best model for the job. We’ve seen teams operating in sensitive industries completely airgap Onyx w/ locally hosted LLMs to provide a copilot that wouldn’t have been possible otherwise.
If you’d like to try Onyx out, follow https://docs.onyx.app/deployment/getting_started/quickstart to get set up locally w/ Docker in <15 minutes. For our Cloud: https://www.onyx.app/. If there’s anything you'd like to see to make it a no-brainer to replace your ChatGPT Enterprise/Claude Enterprise subscription, we’d love to hear it!
This is great, the value is there. I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI, not to make their own complicated agents. Your biggest competition in the enterprise space, Copilot, has terrible UI and we only put up with it because it has access to email, SharePoint and Teams. > every product manager fundamentally misunderstands that users just want a chat window for AI, I would actually argue chat windows are terrible ui/ux for most cases and users. It does the opposite of `don't make me think`. Too much potential for user error. Not saying there shouldn't be any LLM integration/features, just that it should be in the form of a button press or something (familiar ux), not the same chatgpt interface that all the early apps are trying to mimic for no good reason. Haha, yea we've seen that exact story many times! Dissatisfied with Copilot and building a (not great) internal solution that is missing polish + most of the "advanced" feature set. > I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI If you're a non-tech company, why doesn't your org dictate a single model provider? How do these decisions work internally, and how to the departments consume them? (Are they consuming the tools?) > make their own complicated agents. Asking a non-tech employee to make an agent sounds like hell. > we only put up with it because it has access to email, SharePoint and Teams. Ah, that's how a third party can make money. Bake in external org-wide knowledge and enable search. I immediately thought of Google's Agentspace when I saw this product. The value for me sits in its ability to do RAG via connectors. RAG + connectors is a huge reason why people deploy Onyx (enterprise search roots means we do a pretty good job there). Also, open-source works really here, since connectors are a long-tail game. We've tried to make it easy to add connectors (a single python interface), and as a result over half of our connectors are contributed by the community. We expect that percentage to grow over time. This means that compared to something like Agentspace, we'll very likely be connected to all of the key tools at your company (and if we aren't, you can easily add an integration). Do you accept connectors? For instance, I saw that someone wished to develop a file system connector (which I saw that many people need) by himself, but there didn't appear to be much cooperation. I was pretty excited for Onyx as a way to stand up a useful open source RAG + LLM at small scale but as of two weeks ago it was clearly full of features ticked off a list that nobody has actually tried to use. For example, you can scrape sites and upload docs but you can’t really keep track of what’s been processed within the UI or map back to the documents cleanly. It’s nice to see an attempt at an end to end stack (for all that it seems this is “obvious” … there are not that many functional options) but wow we’ve forgotten the basis of making useful products. I’m hoping it gets enough time to bake. Really appreciate the feedback (and glad to hear the core concept resonated with you). The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left. In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat. Hey - good response! I'm sure you guys are thinking about this, but please just go through the steps of setting up via docker, uploading say a grad student's worth of papers and docs, scrape a small topic off wikipedia, try and use it for three days and take a look at the ergonomics. It's not easy to regroup sets of documents, get results that link to the document to view post indexing for RAG etc. etc. etc. In general there are a lot of low hanging RAG optimizations that you could do to make this usable for people who don't want to write their own bits of code to make it usable. I ended up fiddling a bit more with anythingllm which, while having fewer features, understands the workflows a bit more. Something like this has a very limited shelf life as a product. What users need from chat is very user specific, trying to be the one chat to rule them all is not gonna end well, and as models get more capable each chat experience is going to need to be more customized. Something like this could have a nice future as an open source chat framework for building custom UIs if it's well made and modular, but that isn't gonna work well with a SaaS model. I've been using Onyx (and Danswer before it) for over a year, and I'd push back on this.
We have Freshdesk, Bookstack, Google Drive, YouTrack, and Slack all connected. It seamlessly answers questions like: "What's Max's GitHub username?"
"I need wire transfer instructions for an incoming wire" We also index competitors' helpdesks and KB articles to track new features they're rolling out.
Our tech support team uses it daily because Freshdesk's AI is terrible and their internal KB search is lackluster. Onyx actually finds things.
The value isn't in being "one chat to rule them all" — it's in unified search across disparate systems with citations. That's not getting commoditized anytime soon.
Keep up the good work, team. > "I need wire transfer instructions for an incoming wire" Ooooof. Careful with that. I disagree. This has both APIs as well as connectors. One of the reasons I use Google Workspace as SaaS is because of the extensive API, that gives me the flexibility I need with a great starting point (and continued development, that I continue to benefit from). Yes, but imagine a chat app that's designed for accountants, that has widgets for accounting, and it's set up for accounting workflows. That's _HUGE_ but not something that a "one chat to rule them all" is going to just go and do. You could use that same example for lab technicians and any other role. Same "verticalization" argument people made for search in the early 2000s. There's a lot more use cases for tech than just vertical SaaS apps. I don't think that comparison holds, search is a task you do as part of a workflow, there isn't a big difference in search across verticals other than curation of the data set you're searching from. If chat is becoming how people do their work, I don't see how product proliferation across verticals isn't going to be a thing. That's only obvious in hindsight. Back in the day, people said how in the world am I going to use the same search box for buying a car (color, make, model, photos) as I do for finding the latest concerts in my area (map, seating charts, etc.) And you're right vertical SaaS DID become a thing, and so will vertical AI, but the horizontal versions of both (search and SaaS) crush the vertical ones (Google for search Microsoft for SaaS), and I believe it will be the same. Theres a layer above what you are talking about (e.g. Teams as the product vs Slack as the company). Horizontal has a higher TAM and Vertical is easier to execute. But this is besides the point. My point was that productivity is a minority of the TAM. Hmm, will have to disagree here. I think "one chat to rule them all" is the way it will end. It does requires having UI components for many different types of interactions (e.g. many ways to collect user input mid-session + display different tools responses like graphs and interactives). With this, people should be able to easily build complex tools/flows on top of that UI, and get a nice, single interface (no siloed tools/swapping) for free. And having this UI be open-source make this easier. I agree with an end state something like you describe, but I don't think it will be a chat app, I think you'll have an agent lives outside your apps, that managers your apps. Side question: It looks interesting but what's with the trend of open source projects providing such bloated installs? The recommended getting started with docker (which first recommends cloning a 350mb repo) seems to assume you need to scale to 100s+ users. At a glance, in their default docker compose I counted no less than 12 containers including nginx, redis and minio. I can't imagine any of these are necessary to run an app on a single localhost machine. I understand they're trying to attract enterprisy customers, but even some of those are probably going to want to try it out first. Would be nice to have an easy minimal install option that doesn't require a deep dive into the project to figure out. Have you checked out https://docs.onyx.app/deployment/getting_started/quickstart#...? We should definitely update the Local guide to reflect the ^, thanks for pointing that out. For the container footprint - pretty much all of them are necessary to run locally. We hope that just a large # of containers shouldn't be a problem, as long as they are all lightweight (e.g. total resource utilization is low). We of course could bundle them together into a single "mega container", but that seems against the best practices of Docker. On a related note, we are looking to have a version that doesn't require the vector database (the heaviest part of the system in terms of resource utilization). Our goal is to have a deployment mode with less than 1gb of RAM required. Hey, the bottom line is the project looks promising and I'm sure it's a lot of hard work. That said, because time is limited, for now I'll have to pass on spinning it up for the reasons mentioned. I took a look at the shell script, and it just seems like a helpful wrapper (with cleanup) over a manual install. I didn't mean to say it's about the amount of containers. All of those services take up a lot of resources, especially compared to the alternatives. For example, Openwebui can be run with just a sqlite database and a backend. Why is nginx needed, or Minio on a single machine with a nice local file system? But I also understand it takes more work to support multiple service configurations so please accept the criticism as constructive (and it's more a general observation of what I've noticed over the past few years). In a landscape where every week we have a different leading model, these systems are really useful for the power users because they keep the interface and models constant and allow to switch easily using API via openrouter or naga. I have been using openwebui which is under active development but I'll give this a try. > allow to switch easily using API via openrouter or naga Ideally not "open"router? It's not open, and don't they charge a margin? Have you tried LibreChat? I spun that up to great success inside an org. Personally, I use Raycast for all my personal work We need more diversity in this space. The fundamental UI experience, and environment hooks, aren't set in stone so we need many players, open and closed, to more fully explore this space. Does it support multimodal documents? My main gripe with openwebui, in addition to it being slow is the fact that it mangles documents in the OCR step. tables that could have been understood great by an multi modal llm, just gets mangled by the ocr and lost instead of storing both a text and original representation. Being able to properly searcbin the knowlege base lime the llm does, but manually would be nice (like get recommendations for docs to add). My usecase is mostly writing, so having a integrated document refinery editor is also a nice feature list. I'm probably rambling but these are my base use-cases for a llm ui I personally have found. What format are the docs being uploaded as? By default, images uploaded into the chat would be directly passed through. PDFs would be parsed and fed to the LLM as text. Writing is a really common use case, and something we'd like to explore more. Currently people often use Onyx for "write something combining X, Y, and Z documents", but I feel that's just scratching the surface. I was mostly ranting about open-webui and hoping onyx would be better than the current state. My usecase involves pdfs with lots of complex figures, ocrd through mistral ocr witch gives text, and images for figures (have tried multiple other as well). I would really like to keep the figures as images, as ocr captions really struggles getting the full semantic meaning. But stoked to get alternatives to the area, will try it out once i get time soon. If we are already comfortable with our enterprise chatgpt subscription, how might this be of value. Given that it does RAG, tool calling, has all the SSO stuff/collab? Or are we not the target customer. Just curious. Especially with both OpenAI and Anthropic really also outpacing startups in release cadence unlike previous cycles. Guessing your selling point is any model no locking (Assuming we are happy with the privacy SOC 2 etc guarantees on enterprise contracts here) A few reasons: 1/ No model lock-in / ability to use the ideal model for each use case. 2/ More connectivity. A fuller connector library (contributed to by the open-source community). More built-in tools (similar to the ^). 3/ Customizability and flexibility. If you really need a feature, you can build it rather than waiting months (years?) for your request to go through. 4/ White-labeling. You can make it feel like a product built for/by your company rather than generic. Do you know what's completely missing from all of these products like anything LLM and Onyx... a mobile application that has parity on the same features that ChatGPT and Claude does... Hmm, yea that's a great callout. Something we definitely have in our sights longer term (focus for now is to make sure that the desktop chat experience is truly amazing). The title could contain "LLM-chat UI", because "chat ui" also means like an instant messaging UI. I'm nitpicking, congrats on the launch! What does this do that OpenWebUI (or one of the many of other solutions) does not? As someone building another competitor in the field, I'll relay some reasons why some of our customers ruled out OpenWebUI in their decision-making process: - Instability when self-hosting - Hard to get in touch with sales when looking for SLA-based contracts - Cluttered product; Multiple concepts seemingly serving the same purpose (e.g. function calling vs. MCP); Most pre-MCP tools suffer from this - Trouble integrating it with OIDC - Bad docs that are mostly LLM generated Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows. Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good. Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools. Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots). OpenWebUI isn't Open Source anymore. Open WebUI has an egregious CLA if I want to contribute back to it (Which I wouldn't do anyway because it isn't Open Source...) Onyx Devs: This looks awesome, I will definitely add it to my list of things to try out... close to the top! Thanks, and please keep it cool! One thing I really care about is extensibility. Every now and then one of the big consumer apps adds a feature I really like and the self-hosted solution should have some way to integrate that. The main thing I really care about is voice mode, as that's my far preferred way of interacting with LLMs for longer backs and forths (most apps I've seen disable a lot of other functionality during it, which I hate, btw). Two other things I would like to see are canvas mode and scheduled actions (with decision making capability - e.g. "send a notification if X happens"). I assume such features are going to continue being invented, so I find extensibility to be a huge deal. So much so that one thing I could imagine going really well would be a UI on top of Langchain, which already has most of the facilities for that! > One thing I really care about is extensibility Totally makes sense! We've defined simple `Connector`, `Tool` and (soon) `Agent` interfaces to make it easy to plug in your own implementations/apps. If you wanted, you could just use Langchain under our Agent class to build arbitrary flows. Additionally, the main chat is created from a series of `Renderer` components, and it should be easy to build your own. Do you think that's in-line with what you're thinking of, or do you want to build outside of those confines? > canvas mode and scheduled actions Yes, writing and recurring are two big areas we want to tackle next year. > The main thing I really care about is voice mode Interesting. Why do you think that is? > Do you think that's in-line with what you're thinking of It definitely sounds good. Of course it's hard to anticipate what interfaces new features will require, it probably won't always be possible to integrate new features with a generic plugin (voice mode is most likely such a feature), but if the code is well architected, this should be okay. > Interesting. Why do you think that is? Writing has a lot of friction to me. It's much more comfortable for me to provide context through verbal rambling, which LLMs are great at processing. I like to research stuff and throw around ideas while pacing, doing chores or just lying with my eyes closed. Unfortunately I just discovered that I won't be able to run Onyx on my low powered home server anyway (https://docs.onyx.app/deployment/getting_started/resourcing#...). I understand that a Vector database requires significant resources to run, but I wish there was a version without it. I would love to set this up, I really want all my chats to be on one platform. The problem is, the AI companies seem to want the opposite. My main concern is how well do all the extra features work compared to the native versions? Like web search, RAG on a document, or deep research, adding images, voice chats - my understanding is the models providers don't provide any API's for any of this stuff, so you have your own implementations of all the extra stuff right? Usually I find the open source versions of all these features aren't up to par with the corporate versions and they lag behind in development. Yep, we have our own implementations! We've spent a lot of time on them, and in our internal benchmarks they compare pretty favorable to the native versions. RAG specifically is our speciality - we've done a ton of optimizations there (hybrid search, document age-based weighting, giving the LLM the ability to read more from interesting docs and less of irrelevant docs, etc.) and we outperform the implementation within ChatGPT quite substantially in internal blind testing. Curious what you find if you compare them head to head though! The license on this project is pretty confusing. The license at the root of the project links to backend/cc/LICENSE.md which says you need a subscription license to use the code. Can you call it open source if you need a subscription license to run / edit the code? You don't need any subscription to run the code! By default, none of the enterprise code runs (and it can all be completely removed and the app will work as expected). Fully FOSS version here: https://github.com/onyx-dot-app/onyx-foss. that's fair. What does the enterprise code do vs the FOSS? As I see it has whitelisting and enterprise integrations.. as for the OS version maybe you need to roll your own. This is a usual monetization method though. All of the core chat UX + "add-ons" is in FOSS! In the enterprise: - permission syncing - UI-based white labeling - Advanced RBAC - Usage analytics UI It's not really confusing at all. Content under backend/ee requires a license, everything else is MIT Expat.
Pretty standard stuff. > Can you call it open source if you need a subscription license to run / edit the code? MIT is open source, their other stuff isn't. Pretty clear. That's exactly the same approach employed by Gitlab and is actively being deployed and used by GNOME and F-Droid. Could you elaborate why this approach is confusing? Yes. Open source doesn’t mean free. It really does, by any definition I've ever heard. I suppose the authoritative one would be [1]. A common "trick" for commercial open source software is to use a copyleft license, which restricts redistribution as part of commercial products, and to offer a paid license to get around that. GNU disagrees. > Many people believe that the spirit of the GNU Project is that you should not charge money for distributing copies of software, or that you should charge as little as possible—just enough to cover the cost. This is a misunderstanding. > Actually, we encourage people who redistribute free software to charge as much as they wish or can. If a license does not permit users to make copies and sell them, it is a nonfree license. Fascinating, from skimming that, it does indeed appear that it would be within the GNU philosophy to distribute source code solely in exchange for payment. Doesn't cover a case where the source code is _already_ distributed though, then it's free to run. And even if the source code was only distributed to paying customers, that'd likely be a temporary situation. A relevant quote: "With free software, users don't have to pay the distribution fee in order to use the software. They can copy the program from a friend who has a copy, or with the help of a friend who has network access." I do read the GPLv3 such that if someone _does_ buy the code in any fashion, you must provide the source code to them for free. Relevant excerpt from section 6: "[...] give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge." But yeah, no obligation to provide the source code for free to non-customers, fair point. Just no ability to stop customers from sharing it with non-customers. Does make sense. > > Actually, we encourage people who redistribute free software to charge as much as they wish or can. If a license does not permit users to make copies and sell them, it is a nonfree license. This is interesting. If it had a limitation on reselling or a non-commercial / non-compete clause, it'd be almost perfect. Today lots of companies come in and take open source software and "steal" the profits. (You could argue that theft is invalid since the license allows for this.) This makes it hard for the authors to build a durable business. Certainly difficult to build into a large-scale company. Open source needs a better mechanism for authors to make money with what they create while still enabling user freedom to do what they want with the software - modify, reuse, publish changes, etc. "Open core" is one strategy, but it feels like stepping around limitations in the license. Just spelling out "we want to make money in a defensible way" and giving user freedoms seems like a step in the right direction. More companies would probably opt to share their code if this happened. Nothing in that "authoritative" definition says you cannot charge for binaries, for example. It's talking mainly about source code itself. Something you just publish the source for but charge for anything else, would be fair game and still "open source" by that definition. Agreed, "free" is too broad. I was responding to parent's question though: "Can you call it open source if you need a subscription license to run / edit the code?" I'd say no. If you have the code in front of you, it shouldn't require a license to run. Even if the whole point of the open source software is to interact with a proprietary piece of software or service, you could still run it for free, it probably just wouldn't have much utility. > open-source How do you plan to make money? I'm very serious about this question. I'm in an adjacent (but highly non-LLM field) and I'm grappling with this myself. Selling compute or being a middle man and paying for API use seems like a low-margin game. It'd have to be with giving convenient search access to org data or something along those lines? Perhaps expanding into some kind of agent product? My creativity on making money with LLMs is pretty sparse as I'm in the graphics world. But I'm certainly interested in "making money on aggregation" ideas. Also, are you worried about competitors forking your code? Aren't most of the large frontier model providers SOC 2 compliant? I think AWS Bedrock is also SOC 2 compliant. Not sure why you would need to self host anything then as you'll get turnkey secure solutions from the bigger plays Outside of pure security, we've seen many choose Onyx for flexibility + connectedness. One of our largest users has forked the repo and has 20+ commits back to the repo of small customizations that are important for them (and that they could never get with ChatGPT Enterprise). Lots of companies we talk to value having the best model for the job (e.g. not being tied to ONLY OpenAI models for example). Compared to model provider offerings, we also (thanks to open-source contributions) cover many more existing apps when it comes to connectors. fabulous work. ive been following you since danswer. you certainly create a lot of value and have been successful in getting the community to cover the long tail of integrations. its interesting to see how "lock-in" is the main pitch here. all things considered, i don't think "lock-in" is relevant at all unless the activity performed with the tool is highly strategic to the company. you could argue that some orgs may not want openai/anthropic to have their sensitive data leave the parameter, but im also here to tell you that even the most privacy sensitive companies in the world probably resolve this by having a proxy in between the users and the LLM APIs from the labs. so where does this leave you ? cost savings from OSS? maybe, but its hard to imagine that we are in the phase of the adoption cycle where companies have become as acutely aware of costs as you think they are. my 2c - focus on the integrations and see which one gets most traction. that will be your value capture mechanism long-term. Impressive work on Onyx! It’s rare to see an open-source AI platform nail both enterprise-grade security (RBAC, SSO, 40+ connectors) and a truly polished UX. The attention to model-specific quirks—like GPT’s Jupyter assumptions—shows deep practical experience. Already spinning it up locally; thanks for keeping it open and extensible It’s rare to see an open-source AI platform nail both enterprise-grade security (RBAC, SSO, 40+ connectors) and a truly polished UX. The attention to model-specific quirks—like GPT’s Jupyter assumptions—shows deep practical experience. Already spinning it up locally; thanks for keeping it open and extensible That sidebar of past chats is where they go to be lost forever. Nobody came up with a UI that has decent search experience. It's like reddit internal search engine, but a bit worse. Exactly. The best they came up with is a generated subject-like summary. So many options to explore here. Categorization by topic, by date, by customer account, clustering by topic, search with various ranking options, conversation(s) tree view, histogram per date/topic/account, integration with email, with an issue tracker, various states per chat/thread e.g. resolved/ongoing/non-viable, a knowledge bank to quickly save stuff you learned (code snippets, commands, facts), integration with Notion or a wiki etc etc. Just off the top of my head. I was told there would be rapid prototyping with AI. Haven't seen any of the above. I am wishing I had tags for past chats, and the sections of them. Would be much faster to find with searching and other small ui tweaks. I face the same problem and agree with your "search experience" remark. But it triggered something and raised this idea: just add the search bar to the top of the page instead of hiding it in the sidebar. That would solve 100% of my need to open the sidebar. Maybe show 5 most recent chats under the search bar when I click it before typing anything. The UI looks so close to Open WebUI I was shocked this isn't a fork. It even looks like it takes OWUI's unique model customization features, but makes it agents. Might have to try this out. OWUI's lagging docs has made managing my own self hosted instance a pain. PS: Your _See All Connectors_ button on the homepage is 404ing. Haha, yea the UIs certainly have similarities (much of the industry converges to standard places to put different components, since users are familiar). "Agents" is a particular area where we feel like we're better than the alternatives (especially if you want something that effectively calls multiple tools in sequence). Curious to hear your thoughts after trying it out! Interesting product and best of luck with it. > but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them I did get a little giggle out of that because I've never heard anyone say that hooking up 3rd party llms to anything was any way secure. Thanks for the kind words! The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly. Is running your llm through azure insecure? I mean more so than running anything on cloud? My understanding was that azure gpt instances were completely independent with the same security protocols as databases, vms, etc. Azure wouldn't be if you have your company AD/Oauth, I'm GUESSING running local models with data transfer might expose that communication if your local machine is compromised, or someone else's, potentially is multiple points of leakage, companies generally like to limit that risk. This is all an assumption btw. Edit : grammar Ah I see.. That makes a bit more sense and definitely adds a value multiplier for enterprises I would imagine! I'll try out the open source one and see how it works out! I saw that in the new versions (2.x), you kind of broke the use of on-prem deployed LLMs, which is the whole point of Onyx. Kudos on the launch! Most of the real work isn’t the chat box. It’s keeping context stable, memory reliable, and tool calls from drifting when things get complex. That’s where projects usually break, and also where the interesting problems are now. :) Yes indeed! Our belief is that tool design, compaction (e.g. tool result summarization), and reminders are the things that separate a product that works magically vs one that falls over on any slightly more complex task. And this is all made all the more important when supporting the wide range of models, even "weaker" open-source models. Do you let organizations white-label it so its more customized (i.e. remove the Onyx branding, preload it with their internal MCP servers / docs) and feels like their own internal chat tool? Yes absolutely! There's no license restriction on white-labeling, so we've seen lots of companies do that. In our opinion, it's a bit silly to build completely in house when you can take something like Onyx as the starting point and be >95% of the way there + have a tons of bells and whistles built in. Congrats on the launch! Curious, it's a crowded space with other enterprise search companies like Glean and Elastic, and other companies coming in like Notion and Slack. Why should a prospect choose Onyx over the others? Great question! Depends on the specific alternative, but the broad points are: - "pure chat" experience. From our community (and personal use), we've observed that most queries don't actually involve enterprise search. They much more likely to just require the LLMs internal knowledge (or web search / code execution). Compared to all the companies you've mentioned, we've spent a lot more time refining this more common flow. - Larger connector suite. As soon as one key source isn't connected, the trustworthiness of the system is dramatically decreased. You second guess "is the info needed to answer this question in there?" for every question. We have a community who builds out connectors for themselves, and then contribute it back for everyone to use. This allows us to cover the long-tail better than companies like Notion and Slack. - Customizability. An open-source application is the perfect middle ground between a SaaS offering and building blocks. A SaaS option doesn't allow for any customization (we have many customers who have contributed back ux enhancements, small features like guardrails, or enhanced configurations that their users want). Building blocks demand too much domain expertise (search, frontend/UX, ...) for it to be realistic for companies to build something great. how is this different from Librechat? Some of the key differences: 1/ large connector suite + good RAG. Answers at scale is hard, and from our enterprise search roots, we've spent a lot of time with it. It's something that many teams expect from their chat UI. 2/ deep research + open-source code interpreter. 3/ simpler UX. LibreChat has a lot of customizability exposed front and center to the user, which is great for the power user but can be overwhelming for someone new to using AI systems. What struck me most was the requirement. I've been self-hosting librechat on a raspberry pi. I was hoping to test Onyx on it, but no way. Its requirement is 4 vCPU and 10GB RAM? I wish I can take out components that resource hungry and have only the basic so that I can serve it on a less demanding server. LibreChat has recently been acquired by ClickHouse, so who knows what their future holds. Sorry, but why use this over oobabooga/sillytavern? Why do we have to yet again poorly copy an oversimplified UI? The value of local models comes from their huge amount of settings/control that they offer. Why must we throw that all away? Yet again, the world waits for good UI/UX for pro/prosumers with AI systems. No one is learning from ComfyUI, Automatic1111, or SillyTavern. No, LM-Studio is not actually prosumer We're definitely looking to add back some of that flexibility / customizability. I don't think you have to sacrifice a nice, simple UI to provide what power users are looking for. For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice. You are not the target user. The target customers are enterprises where 70% of the target users have never used an AI system before and have to go through a AI training before being allowed access to the system. If I'm understanding this right. I can train it on private data only for my company and then I can use the chat bot only for my site to acquire customers? You could! We have a number of users who do something like that (e.g. Ramp's in-app chat bot is powered by Onyx) Congrats on the launch! Every enterprise deserves to use a beautiful AI chat UI (and Onyx is a fantastic and easy to try option). It's never too late to refine a common idea and take it to the next level. Congrats! Have you considered making a version that is packaged as a desktop app? We have! Likely not for a while, since we want to focus on the web app for a bit more. But long term that would be the plan. Things like shortcuts to access the chat + ability to reach into the local file system are very compelling. Cherry Studio is my daily go-to, I hope Onyx deskktop can be a great alternative for personal users who just want a dedicated app to access any LLMs with full power of MCP and various tools What is the use case for a desktop app? Just local storage or using local compute for RAG? or perhaps privacy? GW! How does Onyx differ from Open WebUI and its alike? Reposting from https://news.ycombinator.com/item?id=46047430: --- Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows.
Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good. Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools. Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots). Is this sort of like self-hosted NotebookLM? That's one way people use Onyx! Specifically, the Projects feature (see the left sidebar) works similarly e.g. you can upload arbitrary numbers of files - going well beyond the context limit of your model - and then ask questions of them. why use a name, although spelled differently onyx vs onnx, that's already used and known in the ML/AI community? That's a fair point! We were aware of onnx, but felt it was okay since they are very different products so we felt that there wouldn't be too much confusion (people generally know which onyx/onnx they are looking for). You're almost literally in the same ecosystem, it's not like one is a Chat UI for LLMs and the other a super market, but a ecosystem of open source machine learning software, libraries and tools. That the pronunciation is identical makes it untenable, you really need to reconsider the name, discussions in person will get confusing. A bit like mastra.ai - my goto SOTA solution for these kind of LLM flow coordinations (though more dev-focused). (yes I realise this is more user-facing) Congrats on the launch! Can you clarify the license and if this actually meets the definition of Open Source as outlined by the OSI [1] or if this is actually just source available similar to OpenWebUI? Specifically can / does this run without the /onyx/backend/ee and web/src/app/ee directories which are licensed under a proprietary license? Yes, it absolutely does! The chat UX, add-ons (deep research, code interpreter, RAG, etc.), and SSO are MIT licensed. Most deployments of Onyx are using the pure FOSS version of Onyx. Many individuals / teams have done extensive white labeling (something that OpenWebUI doesn't allow). We have https://github.com/onyx-dot-app/onyx-foss, for a fully MIT licensed version of the repo if you want to be safe about the license/feel freedom to modify every file. Looks awesome! I'm really trying to switch off Open WebUI due to its general slowness and bugginess, as well as documentation which is almost entirely slop. However it doesn't seem to have MinerU as a supported backend, which is hands-down the best PDF extraction tool I've ever used (and is self-hostable on a machine with a modest GPU). Could it be added? Great suggestion! I haven't heard of it before, but I'll definitely add it to the list. Would love to hear your feedback after you get a chance to try Onyx out. - can't branch-off/fork a chat - can't fold/unfold code - lack syntax highlight for some languages (Zig, Odin, D) ... just because OpenAI call their product "ChatGPT" doesn't mean the term "chat" should now mean "interacting with an LLM". Congrats on the launch! We are building a competing open source tool[0] with a very similar focus (strongly relying on interoperable standards like MCP; built for enterprise needs, etc.), though bootstrapping with customers rather than being VC funded. It's nice to see a competitor in the field following similar "OSS Friends" principles, while many of the other ones seem to have strong proprietary tendencies. (Small heads up: The "view all integrations" button goes to a 404) Congratulations for the launch. Actually we launched a similar product recently named Vision Workspace (https://vision.pixlab.io). The general chat niche is quite saturated and practically locked by the major players. I recommend that you focus on one core feature and pivot from there. For us it was the built-in OCR and document query interface inside the UI that initiated the traction and the app is quite popular now in Japan and Malaysia. What is it with these Chat apps having strange and not-real open source licenses? OpenWebUI is the same. Is there something about these chat apps that seems to make them more prone to weird and strange licenses? Just opportunist? MIT core + "ee" (enterprise edition) commercially licensed extension subdirectory isn't that strange of a occurrence nowadays. I also wouldn't pin it as chat app specific. Quite a few VC funded open core software has adopted that pattern post ~2020(?): cal.com, Dagster, Gitlab Yea, the license is modeled after the Gitlab license. All of the core chat/RAG/agent logic is fully MIT, and >99% of deployments of Onyx are using the "community edition"! Copilotkit is in the same boat. There are parts of the open source codebase that require an enterprise license to use. Basic things like "on error" handlers that are completely offline features. (They might have moved away from this, I haven't checked in a while) If you tack on these faux-pen source VC licenses and complicate things, you're signaling dishonesty and dark patterns. It might not be the case, but it's not a good look imo. VCs don't seem to care, though - it's all about securing the future payoff, doesn't matter what principles or norms get trampled in the process, and it's only a small set of FOSS nerds that ever get bothered by it, anyway. Thanks, lawyers, you make everything better! > We’re building an open-source chat that works As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness. It is open code for sure but it is a business and so incentive is to run it like a business which will conflate with how the project is used by the community. Btw, there is nothing wrong with that but let's be honest here if you get this funded (perhaps it already is) who are you going to align your mission with - the open source community or shareholders? I don't think you can do both. Especially if a strong competitor comes along that simply deploys the same version of the product. We have seen this story many times before. Now, this is completely different from let's say Onyx being an enterprise search product where you create a community-driven version. You might say that fundamentally it is the same code but the way it is presented is different. Nobody will think this is open-source but more of "the source is available" if you want to check. I thought perhaps it will benefit to share this prospective here if it helps at all. Btw, I hear good things about Onyx and I have heard that some enterprises are already using it - the open-source version. > As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness. It's an MIT license. That IS open source. If they have a commercial strategy - that's a GoodThing. It means they have a viable strategy for staying in business, and keeping the project maintained. MIT == OpenSource. Pricing == Sustainable. That's a horse worth backing IMO. Exactly, if everything looked too good to be true and there was no transparency or hint of a business model it’s actually less attractive for some who value predictability. > It's an MIT license. That IS open source. The source is available and you can do much with it, but the incentive is that this alone should not be enough. You are not wrong but in most cases this is a trojan horse. It has the characteristics of a classic rugpullware. At the top level looks like open source but it is not really because parts (the most useful ones) of the project are not. Imagine if python was open source but the core libraries where not. You wont call this open source in the true spirit of open source. You could make the argument that at least it is sustainable because they a have now a business model. It doesn't add up. I prefer more of a honest take on software. There is nothing wrong to make money while contributing back to the community in some meaningful way or by simply being transparent. In fact this is the best kind and there are plenty of good examples. All I am saying is that when I see such projects I tend to think that in most cases they are dishonest to themselves or their communities or both. > As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness I think there can be other valid perspectives than your own. Great to hear that you've heard good things. And yea we have many large (>1k+) teams using just the open-source version (something we love to see). Actually - if you have bunch of VCs on your back, you can't even align with your very own user base let alone any other wider community. Honestly surprised something like this can get funded "Chat UI" can "feel" a bit thin from an eng/product when you initially think about, and that's something we've had to grapple with over time. As we've dug deeper, my worry about that has gone down over time. For most people, the chat is the entrypoint to LLMs, and people are growing to expect more and more. So now it might be basic chat, web search, internal RAG, deep research, etc. Very soon, it will be more complex flows kicked off via this interface (e.g. cleaning up a Linear project). The same "chat UI" that is used for basic chat must (imo) support these flows to stay competitive. On the engineering side, things like Deep Research are quite complex/open-ended, and there can be huge differences in quality between implementations (e.g. ChatGPTs vs Claude). Code interpreter as well (to do it securely) is quite a tricky task. My understanding of YC is that they place more emphasis on the founders than the initial idea, and teams often pivot. That being said, I think there is an opportunity for them to discover and serve an important enterprise use case as AI in enterprise hits exponential growth. w24, those were different times. Yeah that's like so long ago. But yeah, good luck competing with ChatGPT. There are many markets (Europe), and highly regulated industries with air-gapped deployments where the typical players (ChatGPT, MS Copilot) in the field are having a hard time. On another axis, if you are able to offer BYOK deployments and the customers have huge staff with low usage, it's pretty easy to compete with the big players due to their high per-seat pricing. There are also many teams we work with that want to (1) retain model flexibility and (2) give everyone at the company the best model for the job. Every week? a model from a different provider comes out that is better at some tasks than anyone else. It's not great to be locked out from using that model since you're a "ChatGPT" company. why? there's a million other project just like this one, many that are much more advanced and mature, including from Vercel. There's no moat. Agree that's a lot of other projects out there, but why do you say the Vercel option is more advanced/mature? The common trend we've seen is that most of these other projects are okay for a true "just send messages to an AI and get responses" use case, but for most things beyond that they fall short / there a lot of paper cuts. For an individual, this might show up when they try more complex tasks that require multiple tool calls in sequence or when they have a research task to accomplish. For an org, this might show up when trying to manage access to assistants / tools / connected sources. Our goal is to make sure Onyx is the most advanced and mature option out there. I think we've accomplished that, so if there's anything missing I'd love to hear about it. Alright let's say im tasked with building a fancy AI-powered research assistant and I need onyx or Vercel's ai-chatbot sdk. Why would I reach for onyx? I have used vercel for several projects and I'm not tied to it, but would like to understand how onyx is comparable. Benefits for my use cases for using vercel have been ease of installation, streaming support, model agnosticity, chat persistence and blob support. I definitely don't like the vendor lock in, though. > ease of installation, streaming support, model agnosticity, chat persistence and blob support we have all of those! > how onyx is comparable For an AI-powered research assistant, Onyx might just work out of the box. We have ~45 connectors to common apps (https://github.com/onyx-dot-app/onyx/blob/main/backend/onyx/...), integrations with the most popular web search providers (https://github.com/onyx-dot-app/onyx/blob/main/backend/onyx/...), and a built in tool calling loop w/ deep research support (https://github.com/onyx-dot-app/onyx/blob/main/backend/onyx/...). If you wanted to customize, you could pretty easily tweak this / add additional tools (or even rip this out completely and build your own agent loop). Not wanting to use Vercel is honestly a good enough reason. If you’re a heavy Vercel user you probably aren’t their target market since they’re aiming at enterprise types from what it looks like. I wasn't trying to be a hater, i think it is great they got funded for this. It just felt like there are so many free options and alternatives out there that are addressing basically the same things (and look almost exactly the same) it genuinely surprised me. [flagged] Totally get this concern. How we think about it: the chat product should be completely open-source and free (forever). To that end we've moved features like SSO (that used to be "enterprise") to be MIT licensed. The chat interface is something pretty much every team needs (be it a proprietary or open-source solution). You can think of this like Apache Spark for Databricks or Ray for Anyscale. Also, as other folks have pointed out in the thread, there are quite a few other open source options out there. So there's a a ton of outside pressure for our open-source only offering to be very competitive. We hope this reduces the "enshitification" risk that you speak of. This was funded by YC? Why? It's more of a developer project/tool. It will become useless really quickly. The truth is more important than your points. "The reason is because VC needs to show that their flagship investments have "traction" so they manufacture ecosystem interest by funding and encouraging ecosystem product usage. It's a small price to pay. If someone builds a wrapper that gets 100 business users then token use on the foundation layer gets that passed down. Big scheme."
tomasphan - 8 days ago
paool - 6 days ago
Weves - 8 days ago
echelon - 7 days ago
katzskat - 8 days ago
Weves - 8 days ago
adam2221 - 4 days ago
rao-v - 8 days ago
Weves - 8 days ago
rao-v - 6 days ago
CuriouslyC - 7 days ago
bks - 7 days ago
nrhrjrjrjtntbt - 7 days ago
jstummbillig - 7 days ago
CuriouslyC - 7 days ago
motoxpro - 7 days ago
CuriouslyC - 7 days ago
motoxpro - 21 hours ago
Weves - 7 days ago
CuriouslyC - 7 days ago
unleashit - 6 days ago
Weves - 6 days ago
unleashit - 5 days ago
crocowhile - 7 days ago
echelon - 7 days ago
toobulkeh - 6 days ago
jmward01 - 6 days ago
gunalx - 7 days ago
Weves - 7 days ago
gunalx - 7 days ago
Tsarp - 7 days ago
Weves - 6 days ago
jryio - 7 days ago
Weves - 7 days ago
jiehong - 7 days ago
panki27 - 7 days ago
hobofan - 7 days ago
Weves - 7 days ago
limagnolia - 7 days ago
solarkraft - 7 days ago
Weves - 7 days ago
solarkraft - 7 days ago
zaptheimpaler - 6 days ago
Weves - 6 days ago
dannylmathews - 7 days ago
Weves - 7 days ago
dannylmathews - 7 days ago
bilekas - 7 days ago
Weves - 7 days ago
martypitt - 7 days ago
maxloh - 7 days ago
csomar - 7 days ago
fhd2 - 7 days ago
fragmede - 7 days ago
fhd2 - 7 days ago
echelon - 7 days ago
embedding-shape - 7 days ago
fhd2 - 7 days ago
echelon - 7 days ago
asdev - 7 days ago
Weves - 7 days ago
nibab - 6 days ago
oliya88 - 7 days ago
oliya88 - 7 days ago
visarga - 7 days ago
aargh_aargh - 7 days ago
stevenicr - 2 days ago
lostmsu - 7 days ago
terminalkeys - 7 days ago
Weves - 7 days ago
bilekas - 7 days ago
Weves - 7 days ago
thinkloop - 7 days ago
bilekas - 7 days ago
bilekas - 7 days ago
adam2221 - 4 days ago
johnxie - 7 days ago
Weves - 7 days ago
alalani1 - 7 days ago
Weves - 7 days ago
jbuendia829 - 7 days ago
Weves - 7 days ago
dberg - 8 days ago
Weves - 7 days ago
sbinnee - 7 days ago
hobofan - 7 days ago
Der_Einzige - 8 days ago
Weves - 7 days ago
hobofan - 7 days ago
thedangler - 7 days ago
Weves - 7 days ago
pablo24602 - 8 days ago
novoreorx - 7 days ago
NullCascade - 7 days ago
Weves - 7 days ago
novoreorx - 7 days ago
Tsarp - 7 days ago
enoch2090 - 7 days ago
Weves - 7 days ago
Axsuul - 7 days ago
Weves - 7 days ago
winddude - 7 days ago
Weves - 7 days ago
embedding-shape - 7 days ago
mentalgear - 8 days ago
nawtagain - 8 days ago
Weves - 8 days ago
throw4039 - 7 days ago
Weves - 7 days ago
WhereIsTheTruth - 7 days ago
simianparrot - 7 days ago
hobofan - 7 days ago
symisc_devel - 7 days ago
turblety - 7 days ago
hobofan - 7 days ago
Weves - 7 days ago
cjonas - 7 days ago
observationist - 7 days ago
_pdp_ - 8 days ago
martypitt - 7 days ago
WhitneyLand - 7 days ago
zb3 - 7 days ago
_pdp_ - 7 days ago
ranger_danger - 7 days ago
Weves - 7 days ago
wg0 - 8 days ago
phildougherty - 8 days ago
Weves - 8 days ago
gip - 7 days ago
mritchie712 - 8 days ago
koakuma-chan - 7 days ago
hobofan - 7 days ago
Weves - 7 days ago
kurtis_reed - 8 days ago
xenospn - 8 days ago
Weves - 7 days ago
elpakal - 7 days ago
Weves - 7 days ago
dbish - 7 days ago
phildougherty - 7 days ago
haolez - 7 days ago
Weves - 7 days ago
zach_moore - 7 days ago
zach_moore - 6 days ago