Microsoft's Copilot chatbot is running into problems

wsj.com

241 points by fortran77 a day ago


Gift link: https://www.wsj.com/tech/ai/microsofts-pivotal-ai-product-is...

_fat_santa - 20 hours ago

The biggest issue I see is Microsoft's entire mentality around AI adoption that focuses more on "getting the numbers up" then actually delivering a product people want to use.

Most of the announcements I hear about Copilot, it's always how they've integrated it into some other piece of software or cut a deal with yet another vendor to add it to that vendors product offering. On the surface there's nothing wrong with doing that but that just seems to be the ONLY thing Microsoft is focused on.

Worse yet, most of these integrations seem like a exercise in ticking boxes rather than actually thinking through how integrating Copilot into a product will actually improve user experience. A great example was someone mentioned that Copilot was now integrated into the terminal app but beyond an icon + a chat window, there is zero integration.

Overall, MS just reeks of an organization that is cares more about numbers on a dashboard and pretty reports than they are on what users are actually experiencing.

nitwit005 - 15 hours ago

I always remember the pointless integration of Google+ into YouTube that simply annoyed everyone. There's surprising willingness to damage an existing successful product to try to save a new struggling product.

Microsoft has also tried hard to push Edge, annoying nearly every Windows user on the planet, with no real success.

chakintosh - 17 minutes ago

I opened Excel today to view a spreadsheet and just out of curiosity, I decided to use the Copilot integration for the first time to ask it about a column's content. Copilot was clueless. The spreadsheet was open, the Copilot button is right there in the UI, as a user, the affordance of that button for me is that Copilot should already have the context (which is the spreadsheet content). But it didn't, and it kept asking me for the spreadsheet like it's open in a browser window.

JohnMakin - 20 hours ago

Microsoft's fumble here is pretty spectacular.

Back in early 2023, the state of google search was abysmal (despite that their leaders insisted it wasn't, it had become nearly unusable for me and I don't think was that unfounded of an opinion). Microsoft rolled out a new version of bing, which became bing chat - search worked for me again for a very brief window of time.

They could have pounced on this opportunity to take a big chunk out of google's search, because google didn't really catch up there til the AI overview was rolled out, and even that is notorious for having issues. Eventually chatGPT seems to have carved out some of this search space with web-search being native to the tools now.

But microsoft was way ahead of everyone here for a brief period! Instead they just rolled everything into bloatware vaguely called "Copilot" and called it a day.

keeda - 13 hours ago

Important snippet that has bearing on the adoption of AI as a whole:

> “Disorganized data silos” have been an issue for Copilot, analysts wrote.

This is true in almost every large organization, and will affect every enterprise AI product out there. There was a relevant subthread just a couple of days ago recounting this exact, same dynamic: https://news.ycombinator.com/item?id=46861209

In fact, Palantir's secret sauce may not be their tech, but their "Forward Deployed Engineers" model (i.e. a rebranding of "partner engineers embedded within their customers' organizations"). Because it turns out that's a lot of what they do is navigating these bureaucratic and political hurdles to unlock access to the data: https://nabeelqu.co/reflections-on-palantir

It gets even worse if you consider this data is going to be extremely messy, with multiple bespoke, partially-duplicate / overlapping, potentially conflicting versions of the data with varying levels of out-of-datedness, scattered across these silos. (I would know, in a past life, I worked on a months-long project called, self-explanatorily enough, "Stale Docs".)

Yeah, untangling these bureaucratic webs and data horrors is not a quarter-long or year-long project, so investors are gonna be waiting a long time for the impact of AI to be visible. On the bright side, as TFA also hints at, AI providers themselves have been severely capacity-constrained. So hopefully by the time these issues get sorted out enough new capacity would be coming online to actually serve that traffic.

In the meantime, I expect a prolonged period of AI companies feverishly splurging on AI CapEx spend even as Wall Street punishes them repeatedly for the lack of impact of AI being reflected anywhere.

pjmlp - 20 hours ago

It is remarkable how during the last 25 years (approximately), Microsoft has been improving their ability to deliver first (or be among the first), followed by messing up the whole process so that late comers end up taking the crown jewels.

PDAs, mobile phones, tablets, tablets with detachable keyboards, managed OS userspace, HoloLens, the XBox mess, and now AI.

There certainly other examples that I failed to address.

This is what happens when divisions fight among themselves for OKRs and whatever other goals.

ProfessorZoom - 14 minutes ago

Is anyone surprised? Windows can't even open and close file explorer reliably without lagging or refusing to respond. How could they possibly rival these models?

Eddy_Viscosity2 - 21 hours ago

> push by Chief Executive Satya Nadella to transform Microsoft into an AI-first company

Why can't we have a 'user-first' company. Maybe think about the user of your products a wee tiny bit. But no, it is not to be.