There is more to code review than (automatable) detection
adaptivecapacitylabs.com159 points by utiiiD 2 days ago
159 points by utiiiD 2 days ago
A defense of human code review I wish I saw more often, especially in light of the concerns people have about cognitive/comprehension debt: comprehension redundancy. At the end, if taken seriously, at least two people understand how the feature works (even if that number is, on average, trending closer to between one and zero). Ideally at least one of the two also comes away with a better understanding of the wider system and how the feature fits into or stands out from that landscape.
If only one person knows the code, then PR time isn't going to save you.
I'm tech lead and I basically don't review PRs, and I tell people this, with a caveat - if you can tell me what you specifically want me to review, for what specific purpose, I'm happy to!
So "can you review this bit for race conditions" is great, love it. This forces people to actually think about what in their code they should be suspicious of, if anything.
"Can you review this" [link to PR] is getting a rubber stamp because humans have never been good enough at "just spotting bugs" to make this worth it and now any LLM is better than a human.
And for the purpose of understanding - PR time is too late. I have not reviewed a PR ("for real") in a long time and yet I could tell you how every system my people have built works down to a very fine level of detail. And it's because _we talk to each other!_ We don't just chill in the same slack channel and code independently, we all value each others brains and want each others inputs because we know it will improve our product and we value what perspectives others will bring.
Trying to learn via PR is a sad substitute for real collaboration and teamwork.
This is wild to read, I always review PRs and frequently find bugs or significant problems in them that get them bounced back
That's a strong signal that your team isn't doing well. Significant problems should have been spotted at a software design stage, or raised in standups, or identified in a pairing session. The earlier you can find an issue the simpler it is to fix, so waiting until the last moment (e.g. PR) means you're spending far more time fixing issues than necessary.
As for finding bugs, what happens if you miss them? Do they go out to production and potentially lose user data? Finding bugs in PR is a big problem. For a start it shows your automated tests aren't good enough, and secondly it shows the devs aren't checking their code works well enough.
If you do them at all, PRs should be a gate for checking whether the code meets the team's quality bar, not if it even works. The team should be able to deliver working code without them.
I'm guessing you work on some highly technical domain that receives highly structured data and is amenable to TDD.
Not all domains are like that. The majority of bugs I see are business/domain logic bugs. It's akin to having misunderstood or missed some aspect of the question, not causing data loss.
I'm guessing you work on some highly technical domain that receives highly structured data and is amenable to TDD.
I lead a group of teams that build frontend software, so not really. :)
The majority of bugs I see are business/domain logic bugs. It's akin to having misunderstood or missed some aspect of the question, not causing data loss.
Financial loss, reputational harm, degraded UX, etc. They're all significant problems. They're more recoverable than a data loss, but equally bad from an accepted low quality standpoint.
I'm also going to guess that you don't have BAs, PMs, or people responsible for the logic reviewing the code in a PR. Consequently you can't spot those problems in at the PR gate unless the issue is that the dev didn't understand the requirements and wrote code that didn't do what it's supposed to. In which case we're back to the quality and testing problem. By raising questions in standup ("Can I clarify that I understand the AC right?"), pair programming ("Let's check the code against the AC") and communicating properly ("Can you demo the feature to the BA so we can be sure it's correct") you move the problem to the people who can answer, and stop the devs needing to review that someone wrote working code.
I just don't believe PRs are the right point to be finding out that the requirements were wrong or that the dev didn't understand what to build. That needs to happen as early as possible. PR is as late as possible.
It sounds like you work at a company where the responsibilities of engineering are split across at least three separate roles, things move slowly and in a structured way, there's a large amount of coordination work, and a PR is a methodical translation of some step-by-step process. Perhaps you have bi-weekly meetings to review RFCs or similar.
At a much smaller company, you might find that a single person does part of the job of a BA, PM, and engineer, that they can produce a PR much more quickly as a result, and that it's more common for a PR to prompt the first detailed discussion about how something will work. A small team has quicker turnaround time on PRs and design, and can thus position in-depth reviews later in the process because less work will be thrown away in the case of a rejection.
I've held this view as a senior dev in a small company, a senior dev in a big compant, the co-founder CTO of a startup with 5 devs, and I continue to hold it now I'm an EM with many teams in a really big company. It's not about team size. It's about fixing the right problem in the right place. PRs just aren't that. They're useful if you have a problem with people not meeting a good standard, but they're not useful as a gate for whether or not the code works or if it's been architected in a sensible way. You need to know those things earlier (especially in a startup where speed is paramount.)
I disagree in some cases. Code is a means of communication. Sometimes it's more efficient to write code and bring it to your team than to discuss it in the abstract.
An example of this might be adding load shedding. You could spend hours talking through it, or you could say "I'm going to add a load shedder to the blah service as a proof of concept" in standup then take an hour to implement it, and have the team critique it from there.
I agree that whether they're a useful gate is debateble, but they can be a useful means of expressing an idea to be approved or rejected.