Code Wiki: Accelerating your code understanding
developers.googleblog.com55 points by geoffbp 7 days ago
55 points by geoffbp 7 days ago
I burned a ton of tokens this summer trying to document our legacy codebase in hopes of quantifying parts ahead of a refactor. My conclusion was that LLMs are bad at this. It waffled between unhelpfully verbose to omitting key aspects. I had to manually review each page. It really struggled with cross file references and inheritance. I tried several approaches, top down, bottom up, text first, diagram first. Maybe I'm not the prompt wizard I need to be. But I would never trust AI summary of any code longer than 500 lines.
I've seen a few of this type of thing pop up in search results ("DeepWiki" by Cognition.) I'm not a fan. It is just LLM contentslop, basically. Actual wikis written by humans are made of actual insight from developers and consumers. "We intend you use it in X way", "If you encounter Y issue, do Z." etc. Look at arch wiki. Peak wiki-style documentation, LLMs could never recreate. Well, maybe with a future iteration of the technology they can be useful. But for now, you do not gain much by essentially restating code, API interfaces, and tests in prose. They take up space from legitimate documentation and developer instruction in search results.
Related:
Code wikis are documentation theater as a service
I want to opt out for my code. How do I opt out.
I had a repo ingested by some AI-slop "product showcase" tool. The dev behind it emailed me and welcomed me to the platform excitedly. Seeing the page made me feel sick and I told him to take it down. But it's the era we live in I suppose.
Realistically, the alternative to code wiki is not good documentation, it's no documentation.
Nice! I've been using deepwiki and loving it! Obviously goggle's gemini powered alternative would be much better and trustworthy.
I just hope Google doesn't kill this one as quickly as they did Stadia etc.
Why after everything you've seen Google do, do you use the terms 'better' and 'trustworthy'?
What about DeepWiki has been untrustworthy?
I looked at the facebook/react one.
Right off the bat, I'm really excited about how it talks about the "optimizing compiller," and how these pieces go from modules that do something to 'infrastructure.'
If this is a flagship demo, it doesn't fill me with hope about the project.
Surprising that they haven't made a podcast (NotebookLM-esque) based on the repo - that one can listen to on a bus ride. Something I had created a while back https://gitpodcast.com
I hoped this might be like an externalization of g3doc. Nope.
Instead, I started reading through one of their highlighted examples --- the Go repo (https://codewiki.google/github.com/golang/go). This might be the worst high level overview of Go and its repo I've read. Mostly accurate but unhelpfully verbose, spending lots of words on trivia, and not at all making a compelling pitch for Go as a language or toolchain, how to use it, or how to work on it.
I was just looking through the Go example as well. For a first attempt its ok. I don't think its accurate to criticize that it doesn't make a case for using Go or teaching how to use it. It's attempting to be a more useful contributing.md. I think it does a decent job at that. Enough that you could find an area of interest and feel confident to start reading and understanding it yourself.
It just doesn't seem to be worth the effort though. I see myself using something like this for ~30 minutes to so I don't feel lost when getting started. After that it becomes significantly less useful.
Also, the video wasn't particularly helpful and if I have to here an AI voice say how fantastic something is again, Im going to unplug it (jk future overlords).
it is quite impressively bad
even the front diagram is completely contentless ("guides usage", "influences"?)
and you can't even link it
How does it know about the tradeoffs and discussions imbued in the code, unless someone has already put it in writing?
This. What I really want to know is if this tool has also gone through the git history for the repo.
Most online documentation doesn't cover the tradeoffs and discussions imbued in the code already, so this is at worst a side-grade.