Automated code reviews via mutation testing

Years ago, I used mutation testing in a pretty big Java project. Let me tell you: You don't want so much coverage. It's usually a waste of time and makes changes to existing code very annoying. Keep mutation testing/high coverage for a small selection of _very important_ code, but don't overdo it

I read about mutation testing in a course of test oriented programming. And i am unsure to this day of what domain it's meant for.

The most strict and safety critical code i have encountered, was specifying its system behaviour in a state machine diagram of a ISO specification.

Yet it was too state based for effective unit testing. Instead it opted to assert the living hell out of the codebase and be paranoid about even the stability of the ram it wrote to.

I have a hard time imagining a domain that is both;

1. Easily to coverage test to a high enough degree that mutating testing can be used.

2. Needs pedantic testing to the point of absurdity, perhaps already using boundary value diagrams for its test-cases.

And so mutating testing feels quite odd to me, and selling it as an AI tooling is even more insane. It feels like far more effort than to just not use AI.

My arc of using AI code generation tools started by steadily adding more and more guardrails. I realized the returns were diminishing while cost, complexity, and latency climbed. Mutation testing made it obvious I had gone too far. While there is a time and place for it, I completely lost the flow and was fighting against my dev setup. I’ve since found a better balance: the most strict, opinionated dev environment, especially on types and linting, with the least code and configuration.

Fascinating idea. The price stopped me right in my tracks.

Why are you making Europeans pay ~15% more for the same product? Instant nope from me on that basis alone.

Paper from 2017 on mutation testing for code review at Google:

https://research.google/pubs/state-of-mutation-testing-at-go...