Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping
sphinx-six-theta.vercel.app128 points by critiquebrowser 5 hours ago
128 points by critiquebrowser 5 hours ago
FOSS, Basically it plugs into your existing coding agent workflow and every time you push to prod it'll throw up a questionnaire that checks if you understand how the code you're pushing works.
I certainly feel like certain AI coding practices I have leave me with nothing learned in a frustrating way. Sometimes the most effective way to getting what I want is just describing a goal really clearly, and then bashing the AI until it gets there. That bashing process can take a lot of energy and yet I don’t gain useful skills or a “better way” that I would if I was still coding. I generate more training data for BigAI corp, but that’s a slow return cycle. Not sure if this is the right solution, but I would like to learn/understand/grow more as I pursue reckless efficiency. Reason #1 to keep writing code by hand is that you don't ever lose that connection between you and the code. There was an excellent post here on HN that unfortunately went unnoticed. I highly recommend reading it as it exactly points out this problem: understanding. Here is the link: https://news.ycombinator.com/item?id=50128468 Note: The post has now been flagged. Why? I have no idea but it seems incorrect. Code by hand? The good old fashioned way? Might be "less efficient", but you will understand what you did and build experience. After noticing a drop in my understanding of the code I ship, I built and started using a similar skill ("Quiz me on the PR or the current branch"). I didn't tie it to a specific commit, though - to me it makes more sense to look at the PR as a whole. So far I was positively surprised with the questions it could come up with. Lol I built this and immediately got frustrated with it and stopped using it. I think I don't like being questioned by someone who I expect to work like magic. It’s ironic that all negative comments on here are dismissive because “slop’s baaaaad”. Yes, slop is bad. Carelessness is bad. Lack of understanding is bad. It always has been. Code has been copied from stackoverflow. Built-in methods or std libraries have been used in a wrong way. Edge cases haven’t been covered. Rate-limiting and other hardening tasks haven’t been taken care of until the system broke. The code wasn’t DRY. Now, people are building tools that not only hide the solutions to these problems in some obscure coding tool, markdown file, or agent harness, but they build gates that help developers to elevate themselves from the darkness of not knowing better. The number of developers roughly doubles every ~3 years, so at any given time, half of the people building software have less than 3 years of experience. Most of them didn’t even study CS before starting their careers. Tools like this will be the new warning voice that told people like us 5 to 10 years ago to never copy code from stackoverflow that you didn’t understand. It’s great. The problem with AI SE is that AI writes un-architecture-ed code. The problem #2 is that a lot of people suck even more.... > The problem with AI SE is that AI writes un-architecture-ed code Not necessarily. I use SOTA models with established stacks like .NET which allow my agent to easily follow decades of best practiced patterns (DI, EF Core, ASP.NET Core...) so you still and up with proper architecture. In fact I can easily still manually inspect everything bc I already know how it's supposed to be written. I considered doing this with an open source app I have that let's you get a visual summary on a PR + get quizzed on it, but ultimately decided it's better to keep it an optional thing for people that want to understand AI PRs rather than force it as a gate, the volume is slowly becoming too high for even that. I agree with the sentiment behind this though as I felt this was needed as well. I can kinda understand the motivation here: you push a bunch of slop (or is it deep, genius code?) but you have a kind of nagging suspicion that you should get to know it more familiarly, like back in the good old days a few months back when you sort of understood what was going on. I... don't... think this is going to work...? Remember Confirmation Fatigue? Soon we will have agents that interact with Sphinx for us. Honestly I love to see people working on this problem ! I actually built a similar project myself (https://github.com/bhurghundii/vibecheck.dev) except it prevents you from merging on the PR level. I’d love to get in touch to talk about it sometimes. Edit: I see there are a lot of people questioning the value of this. The thing is that there are teams where people are just pumping tons of AI code without understanding their work (literally take the ticket and dump it) and shipping it. culture change is the right answer but v hard but this at least slows things down and is quick to implement Whenever I stop for a moment to think about it: It is absolutely batsh*t crazy insane what SWE has come to in the last cple years. And yet part of me is loving it. Its the right problem statement, but we normally let the code review process take care of this. I mean I see the idea, but at least I use LLMs for code generation in two distinct ways. The first is when I care about the code. That's when I actually read what it produces and force it to use my mental model of how things should be. The second is when I just need a solution that sorta fits the requirements provided. Then I don't read the code because I really do not care. So this hook would either ask me things I know, or it would ask me things I do not care about. But I guess this is very workflow-dependent. I think this tool is meant to be something you impose on your colleagues who treat all code like the second bucket. This is cute and novel, but practically useless. For those who use AI efficiently and effectively, this is useless. For those who don't, this becomes friction they don't want. Who really needs this? a tool written by AI (mostly) to check code written by (possibly other) AI. however, it still feels like a lot of work. what if AI hallucinates during the check? And then you have review process done by AI as well. The trick is they all need to hallucinate the same thing and if they do, the initial requirements were wrong. Sun Tzu said: If your people (AI agents) repeatedly fail to follow instructions, examine the clarity of your orders and the effectiveness of your leadership before blaming them. Your slop generator should check if a name exist before claiming it. People who care about the problem will definitely not be using this as a method to "understand" their code. People who don't care about this problem will continue to not care. So, who is this for?
theParadox42 - 3 hours ago
liendolucas - 2 hours ago
liendolucas - 24 minutes ago
rootnod3 - 2 hours ago
maximg68 - 16 minutes ago
chacha-bong - 3 hours ago
nkmnz - 4 hours ago
xiaoyu2006 - 3 hours ago
jdthedisciple - 2 hours ago
sshah03 - 4 hours ago
jamienk - 5 hours ago
sailorganymede - 3 hours ago
jdthedisciple - 2 hours ago
cjimesh - 3 hours ago
hypfer - 4 hours ago
mikepurvis - 3 hours ago
compounding-err - 3 hours ago
jan_m_savage - 3 hours ago
hatch_q - 2 hours ago
linkdd - 5 hours ago
altmanaltman - 4 hours ago