Clean Coder: The Dark Path (2017)
blog.cleancoder.com36 points by andrewjf 5 days ago
36 points by andrewjf 5 days ago
> The rules of the language insist that when you use a nullable variable, you must first check that variable for null. So if s is a String? then var l = s.length() won’t compile. ...
> The question is: Whose job is it to manage the nulls. The language? Or the programmer? ...
> And what is it that programmers are supposed to do to prevent defects? I’ll give you one guess. Here are some hints. It’s a verb. It starts with a “T”. Yeah. You got it. TEST!
> You test that your system does not emit unexpected nulls. You test that your system handles nulls at it’s inputs.
Am I reading or quoting this wrong?
Just some pros of static type checking: you can't forget to handle the null cases (how can you confirm your tests didn't forget some permutation of null variables somewhere?), it's 100% exhaustive for all edge cases and code paths across the whole project, it handholds you while refactoring (changing a field from being non-null to null later in a complex project is going to be a nightmare relying on just tests especially if you don't know the code well), it's faster than waiting for a test suite to run, it pinpoints to the line where the problem is (vs having to step through a failed test), and it provides clear, concise, and accurate documentation (instead of burying this info across test files).
And the more realistic comparison is most programmers aren't going to be writing lots of unhappy path tests for null edge cases any way so you'll be debugging via runtime errors if you're lucky.
Static typing here is so clearly better and less risky to me that I think expecting tests instead is...irresponsible? I try to be charitable but I can't take it seriously anymore if I'm honest.
The idea that tests can replace a type system (and vice versa) is a known fallacy.
Discussed here, two years before this article was written: https://www.destroyallsoftware.com/talks/ideology
Your elevator should not have automatic doors, doors are restrictive. They stop you from quickly jumping out of the elevator if you decide that you actually want to stay at the first floor.
Sure, we’ve seen some pretty gnarly accidents, and there is no reasonable situation where risking death is a sane choice.
But ask yourself: is it the elevator's job to prevent an accident? If you think so, I suggest you never leave your home again, as safety is your own concern.
Like and subscribe for other posts like “knife handles? What an idiot” and “never wear a helmet you coward”.
> “never wear a helmet you coward”.
Unironically the biggest flame-wars I ever saw on forums back in the day was on whether or not mandatory bike helmets made cycling safer or more dangerous.
Can't resist taking this bait, but I feel like the consensus is pretty much that as an individual, choosing to wear a helmet will make you safer, and as a society, mandating bike helmets (and other measures that will cause people to use transportation methods that are more dangerous to others) will make everyone less safe.
Of course, it's hard to prove. But I think you'll generally find that, if you compare the number of injuries/deaths while cycling in countries with mandatory helmets per km will be higher than it is in the Netherlands, where they are not mandatory.
Comparing the Netherlands with some of the best, if not the best bike infrastructure to other countries without said infrastructure seems very reductionist. To get anywhere near an interesting number you would have to compare the number of injuries to the total number of accidents including cyclists in countries with comparable bike infrastructure and differing helmet policies.
I can understand, I’ve had some awful bicycle accidents while not wearing a helmet and the helmet would have made no difference. My knees, hands and elbows have been through a lot but I have never hit my head. Even while skateboarding the one time I hit my head a helmet would not have helped. In hindsight I should have worn knee pads a lot more often but then I wouldn’t be able to tell if it’s gonna rain or not from the feeling in one of my knees :)
However if I had to ride on a public road with cars zooming pass me recklessly I would absolutely wear a helmet on a bicycle.
And there are countries where everyone rides a bike, and they are not usually the ones with mandatory helmets. When you hit someone with a bike you are less likely to kill them than when you hit them with a car, so more bike riding means less deaths, without even considering the effects of air pollution.
Helmets are fine for sport riding, but inconvenient if you want to ride 5 minutes to the shops on a whim. And that kind of riding is usually less intense and safer, I presume, anyway. Football has helmets, walking doesn't.
Those countries also have a higher number of brain injury cases vs countries where people wear helmets.
If you are hit by a car a helmet will do approximately nothing. However there are a lot of accidents that happen where you are not hit by a car where a helmet will help. (and even more when knee and elbow pads are what you need)
C programmers non-ironically believe this
And of course they only believe this because they don't lose a finger when they write outside of bounds or they don't fall down the shaft once a pointer was accidentally null
C programmers want program like they drive race cars or fighter jets, anything for greater speed
Did you even read the second half of the post? The author's answer to your concerns is testing. He suggests relying on tests rather than on strict type system that forces you to design everything upfront.
Personally, I can see arguments for both approaches - stricter types or more tests.
I've written python and C++. For small simply problems python is nice, and writing tests for everything is easy enough. However as the program gets larger python becomes painful to work with. Writing code is the easy part, the hard part is when you want to refactor existing code to add a new feature - if you don't have the right tests (integration tests, not unit tests - though these terms are poorly defined) you will miss some code path that then won't work when it finally is run - unless you have a good type system which catches most of the errors that can happen when refactoring.
C++ is a notoriously difficult language to write in. However when the problem demands tens of millions of lines of code I'll take it over python because of the type system. There are other languages with good type systems that are reportedly better.
Tests check the operation for a single value of the, potentially infinite, input space.
Types check the operation for many, and can be made for all possible input values.
The difference is:
None of these boxes contain bicycles because:
* With tests: we checked 3 of them and found no bicycles
* With types: all the boxes are too small to contain bicycles.
Yes I did, which doesn’t mean I buy the point at all. Testing clearly falls into “manually noticing the mistake”.
I could not agree less. The line of reasoning here is: relying on the type system to prevent error lets people writer fewer tests. Tests are the only way to assure quality. Therefore fewer tests is bad, and powerful type systems are bad too, because they cause you to act against software quality.
Furthermore, Uncle Bob sets up this weird opposition between the programmer and the type system, as if the latter is somehow fighting the former, rather than being a tool in their hand.
I think that sadly this is just the narrative of a man whose life’s work consists of convincing people that there is a silver bullet, and it is TDD.
> Every time there’s a new kind of bug, we add a language feature to prevent that kind of bug.
That's why learning more academic, 'non-practical' aspects of computer science is sometimes beneficial. Otherwise very few will naturally develop the abstract thinking that allows them to see uncaught exception and null pointer are exactly the same 'kind of bug.'
Anyway the author got it completely upside down. The stricter mental model of static typing came first (in more academic languages like Haskell and Ocaml). Then Java etc. half-assed them. Then we have Swift and Kotlin and whatever trying to un-half-ass them while keeping some terminology from Java etc. to not scare Java etc. programmers.
I suppose it's somewhat accurate to claim that Haskell and Ocaml historically preceded Java (or even Objective-C). But Java wasn't inspired by those academic languages, but C: a then widely used real-world language with only partial static types.
(Not saying Java's attempt to remedy C's problems wasn't half-assed — it was.) The trend to plug holes is primarily motivated by empirical evidence of bug classes. Not by elegance of academic research.
As Bjarne Stroustrup famously quipped:
> “There are only two kinds of languages: the ones people complain about and the ones nobody uses.”
Swift, Kotlin, Rust, C++ are attempt to become languages that everyone complains about, not Haskell or Ocaml.
As someone not familiar with Haskell and Ocaml, which parts of Java are poorly implemented?
There are many, but one particular example is the type syatem.
Explain how
Not OP, and not sure about OCaml and Haskell, but one example where Java's type system is unhelpful/incorrect is mutable subtyping.
E.g. Java assumes Cat[] is a subtype of Animal[]. But this only holds when reading from the array. The correct behavior would be:
- `readonly Cat[]` is a subtype of `Animal[]`
- `writeonly Cat[]` is a supertype of `Animal[]`
- `readwrite Cat[]` has no relationship with `Animal[]`
But Java doesn't track whether a reference is readable or writable. The runtime makes every reference read-write, but the type checker assumes every reference is read-only.
This results in both
- incorrect programs passing the type checker, e.g. when you try to write an Animal to an Animal[] (which, unbeknown to you, is actually a Cat[]), you get a runtime exception
- correct programs not passing the type checker, e.g. passing a Animal[] into an writeCatIntoArray(Cat[] output) function is a type error, even though it would be safe.
(Although all that is assuming you're actually following the Liskov substitution principle, or in other words, writing your custom subtypes to follow the subtyping laws that the type checker assumes. You could always override a method to throw UnsupportedOperationException, in which case the type checker is thrown out of the window.)
Interestingly, AFAIK Typescript makes these types both subtypes and supertypes at the same time, in the interest of not rejecting any correct programs. But that also allows even more incorrect programs.
All type checkers either permit incorrect programs, reject correct programs, or are turing complete.
Did you mean arrays instead of lists? Arrays behave as you describe (with ArrayStoreException when you write a wrong value to an array). List<> is invariant WRT its type parameter.
Another issue is that Java's initial containers were type-less and were then type generics were retro fitted as erasures.
I can't make a Mappable interface, and have my classes implement map(f). Because map(f) will necessarily return Mappable, not my class itself. So no method chaining for me.
Also null. Yeah I know it's contentious. People don't want to let go of it. Since learning to hate null, I've also lost any nuance in my ability to explain why it's bad. Because I know longer see it as 'subtly-bad' or 'might lead to bugs'. It's just plain, on-the-surface-wrong. One might as well have named it 'wrong' rather than 'null'.
'Null' is the thing which it isn't. I can write business logic that says every Person has a Name. Once you admit null into the mix, I can no longer make that simplest of statements. My autocomplete now lies to me, because person may or may not implement the method .name().
"But how will I half-arse instantiate a Person? I don't have a Name, yet I want to tell the computer I have a Person?" It makes me happy that you can't.
"I wrote a function that promises to return a Person. I was unable to return a Person. How can I tell the computer I'm returning a Person even though I'm not?" Glad that you can't.
It's not really about the implementation of Java (might be bad, I don't know). It is the specification.
- People talked about null being an issues and that is a big one.
- The entire idea of OOP extremism Java implemented was a mistake - though just a consequence of the time it was born in. Much has been written about this topic by many people.
- Lacking facilities and really design for generic programming (also related to the OOP extremism and null issue
So much more more you can find out with Google or any LLM
While I consider Uncle Bob a bad programmer, there is some merit to this article. This paragraph was particularly prescient:
>But before you run out of fingers and toes, you have created languages that contain dozens of keywords, hundreds of constraints, a tortuous syntax, and a reference manual that reads like a law book. Indeed, to become an expert in these languages, you must become a language lawyer (a term that was invented during the C++ era.)
And this was written before Swift gained bespoke syntax for async-await, actors, some SwiftUI crap, actor isolation, and maybe other things, honestly, I don't even bother to follow it anymore.
He has a point in that paragraph, but as a C++ developer I couldn't help being agitated by his passive aggressive comment:
> [...] (a term that was invented during the C++ era.)
...like it's some sort of relic, or was in 2017
I agree, but i think his point applies more to haskell than, say, kotlin. There is a balance between type strictness and productivity and if you go too far in one direction you get horribly buggy code and if you go too far in the other direction you have a language that is grindingly slow to develop in.
Another thing I dont think a lot of people appreciate either is that types have sharp diminishing returns catching the kind of bugs tests are good at catching and vice versa.
Yeah once you get into Monofunctor or 'PorcelainPile<static const volatile String&>' lawyering you know you went too far
TIL about Monofunctor.
Pretty nifty. It's for cases where your type is a container of something (as opposed to anything).
I.E. you can .map (or .Select if you're .NET-inclined) over the Chars in a String, but not some other type, because String can't hold another type.
https://hackage.haskell.org/package/mono-traversable-1.0.21....
I understand what the author says, but in my experience, "Nullable Types" and "Open/Sealed Classes" are two different subjects and...
1) For "Nullable Types", I see that it is VERY good to think about if some type can be null or not, or use a type system that does not allow nulls, so you need some "unit" type, and appropriately handle these scenarios. I think it is ok the language enforces this, it really, really helps you to avoid bugs and errors sooner.
2) For "Open/Sealed Classes", my experience says you never (or very rarely) know that a class will need to be extended later. I work with older systems. See, I don't care if you, the original coder, marked this class as "sealed", and it does not matter if you wrote tons of unit tests (like the author advocates), my customer wants (or needs) that I extend that class, so I will need to do a lot of language hacks to do it because you marked as sealed. So, IMHO, marking a class as "open" or "sealed" works for me as a hint only; it should not limit me.
sealed classes are just retrofitting sum types onto the JVM. If Kotlin could have used "enum" for it then they probably would have, like Swift and Rust did.
The main point of sealed classes is exhaustive `when` expressions:
return when (val result = something()) {
Result.Success -> // ...
Result.Failure -> // ...
}
What I read between the lines: “I have such a fragile ego that I feel offended when a tool points out a mistake I made. I feel intellectually rewarded by doing the same busywork over and over again. I don’t want to change the way I do my work at all. I feel left behind when people other than me have great ideas about language design.”
What I read between your lines: "I don't want to think about the code at all. It should only compile if it has no bugs. I don't like rapid prototyping. I feel stupid when people other than me feel they can program effectively with fewer safeguards."
What I read between your lines: "I'm sooo tired of this strong-typed wankery. I shoot and only then ask questions."
P.S.: nothing personal, just playing the game.
> Now, ask yourself why these defects happen too often. ... It is programmers who create defects – not languages.
> And what is it that programmers are supposed to do to prevent defects? ... TEST!
Unfortunately, altering people's behavior by telling/commanding/suggesting that they do so, whether or not supported by perfect reasoning, rarely if ever succeeds.
It's overwhelmingly the case that people, including programmers, do what they do in reaction to the allowances and bounds of a system and so it is far more effective to alter the system than attempt to alter the people.
Surely needing to change some class declarations is better than bugs that take all day to track down? And sure as a programmer i can consider every npe case along with all the others but if the language can take car3 of that for me, I’ll let it
"The Blub Paradox", the article.
total yikes for the entire thing. "What if a function needs to return null" or "throw an error" is not a fundamentally different concept than "what if a function needs to return a totally different type".
for me there is a clear problem in all those languages. The exception paradigma opens a second way to exit a function. This is clearly a burden for every programmer. it is also a burden for the machine. you have to have RTTI, Inconvinient stack undwindings and perhaps gerneric types. Also nullable types are a but of a letdown. first we specify a "reference" kind type to neverhave to deal with null violations, then we allow NULL to express a empty state. Better have Result return types that carry a clear message: Result and Value. Also have real Reference type with empty and value. by accessing a empty value you get back the default value. i think c# has mastered that realy nice, but far from perfect
I was recently switched from Java to C# at work.
Initially I was impressed by the null detection. Then I found out about defaults. Way worse than null.
C and Go can demand a bit of ceremony with manual error checks. Things get bad if you forget to do the checks.
Java and Checked exceptions forced error-checking, which is a little verbose, and most of the time you can't 'handle' (for any meaning other than log) them, so you just rethrow.
C# went with unchecked exceptions. But with default values, there's no need to throw! Avoid Java's messy NPE encounters by just writing the wrong value to the database.
Author argues against strong typing systems and language features to prevent classes of bugs and instead encourages developers to "writing lots of tests" for things that a type system would prevent.
The authors thesis seems to be that it's preferable to rely on the programmer who wrote bugs to write even more bugs in tests in order to have some benefit over a compiler or type system that can prevent these things from happening in the first place?
So obviously it's an opinion and he's entitled to it, but (in my own opinion) it is so so so, on-its-face, just flat out wrong, I'm concerned that that it's creating developers who believe that writing so many tests (that languages and compilers save you time (and bugs) in writing) is a valid solution to preventing null pointer defeferences.
The conversation at the time was commonly along these lines. Powerful type systems won, for the most part, and most of the discussion fell away.
There's also another strong argument against favoring tests over types, which is maintainability. If I go ahead and change a variable type from being non-nullable to nullable, I instantly get a complete list of all the places where I have to handle that, which makes it much much faster to generalize the logic. But in a dynamic language, all tests that were written by the previous developer was written at a time when this variable was assumed to never be null and uses non-null values for all test vectors, so good luck using that to find the places that need to be fixed.
On top of that, every test that could have been omitted due to a type system incurs an extra maintenance tax that you have to pay when you change the API.
I think you you're missing the point though, he did not say that adding those constraints was a bad idea per se, but they are leading to a bad path. If the path is "just add more guard-rails" then we will get to a point where we lose the "soft" part of "software" and you'll find yourself restarting each time you need to change something because at that point the language you chose was already the first wall of a bad program architecture (ease of change).
I lean on types heavily as my vehicle for change.
My UserService doesn't know that it's talking to a UserDB (ironically I learned this from Uncle Bob).
All UserService knows is it has a dependency which, if passed a UserId, will return a User (+/- whatever the failure mode is .. Future<User>? Promise<User>? ReaderT m User?)
When I change my mind about what UserService requires or what UserDB provides (which I frequently do), I immediately look at all the red underlines that my compiler & static types tell me about.
Between this and the debate about ideal method length with Ousterhout, my respect for Uncle Bob is plumbing new depths.
I get the point of the article. However, you can have both: programmers that write tests and don't override safety measures AND safety measures.
I disagree with the article, but also some of these examples are complete straw-men. In Kotlin you have nullable types, and the type checker will complain if you use it as a non-nullable type. But you can always just append !! after your expression and get exactly the same behavior as in Java and get a null pointer exception, you don't have to handle it gracefully as the author is suggesting. Tests checking that you gracefully handle nulls in a language without null types are fucking tedious and boring to write. I would take a language with null types over having to write such tests any day.
Kotlin's final-by-default is also just that - a default. In Java you can just declare your classes `final` to get the same behavior, and if you don't like final classes then go ahead and declare all of then open.
I also disagree with the author's claim that languages with many features requires you to be a "language lawyer", and that more simplistic languages are therefore better. It's of course a balance, and there are examples of languages like C++ and Haskell where the number of features become a clear distraction, but for simpler things like null types and final-by-default, the language is just helping you enforce the conventions that you would anyway need when working with a large code base. In dynamically typed languages you just have to be a "convention lawyer" instead, and you get no tool support.
Your last sentence makes a very good point. And Uncle Bob's tool is to have loads of tests.
I suppose it's all just a balance: simplicity versus expressiveness, foot guns versus inflexibility, conciseness versus ceremony, dev velocity versus performance in production.
I'm okay with shifting some of the burden of common errors from developer into the language if that improves reliability or maintainability. But Martin has a point in that no guard rails can ever prevent all bugs and it can be annoying if languages force lots of new ceremony that seems meaningless.
It is a little weird to end an article calling for less safeguards in languages by a reference to a nuclear disaster caused by overriding safeguards.
Isn't that the classic argument "Real C programmers don't write defaults!" ?
The one that companies have spent billions of dollars fixing, including creating new restrictive languages?
I mean, I get the point of tests, but if your language obviates the need for some tests, it's a win for everyone. And as for the "how much code will I need to change to propagate this null?", the type system will tell you all the places where it might have an impact; once it compiles again, you can be fairly sure that you handled it in every place.
I was rewriting a mod for Rimworld recently. As Rimworld is built on Unity, it's all some sort of C#. I heard people say it's a wrong kind of C#, but since a) I had no choice and b) I never wrote any C# before I cannot tell.
First, C# proudly declares itself strongly-typed. After writing some code in Zig (a project just before this one, also undertaken as a learning opportunity, and not yet finished), I was confused. This is what is called strong-typed? C# felt more like Python to me after Zig (and Rust). Yes there are types. No, they are not very useful in limiting expression of absurdity or helping expression of intent.
Second, test. How do you write tests for a mod that depends on an undocumented 12 year old codebase plus of half a dozen of other mods? Short answer - it's infeasible. You can maybe extract some kind of core code from your mod and test that, but that doesn't help the glue code which is easily 50-80% in any given mod.
So what's left? I have great temptation to extract that core part and rewrite it in Zig. If Unity's C#-flavor FFI would work between linux and windows, if marshalling data would not kill performance outright, if it won't scare off potential contributors (and it will of course), if, if...
I guess I wanted to say that the tests are frequently overrated and not always possible. If language itself lends a hand, even as small and wimpy as C#'s, don't reject it as some sort of abomination.
> For example, in Swift, if you declare a function to throw an exception, then by God every call to that function, all the way up the stack, must be adorned with a do-try block, or a try!, or a try?.
Funnily enough, Uncle Bob himself evangelised and popularised the solution to this. Dependency Inversion. (Not to be confused with dependency injection or IOC containers or Spring or Guice!) Your call chains must flow from concrete to abstract. Concrete is: machinery, IO, DBs, other organisation's code. Abstract is what your product owners can talk about: users, taxes, business logic.
When you get DI wrong, you end up with long, stupid call-chains where each developer tries to be helpful and 'abstract' the underlying machinery:
UserController -> UserService -> UserRepository -> PostgresConnectionPoolFactory -> PostgresConnectionPool -> PostgresConnection
Now when you simply want to call userService.isUserSubscriptionActive(user), of course anything below it can throw upward. Your business logic to check a user subscription now contains rules on what to do if a pooled connection is feeling a little flakey today. It's at this point that Uncle Bob 2017 says "I'm the developer, just let me ignore this error case".
What would Uncle Bob 2014 have said?
Pull the concrete/IO/dependency stuff up and out, and make it call the business logic:
UserController:
user? <- (UserRepository -> PostgresConnectionPoolFactory -> PostgresConnectionPool -> PostgresConnection)
// Can't find a user for whatever reason? return 404, or whatever your coding style dictates
result <- UserService.isUserSubscriptionActive(user)
return result
The second call should not permit failure. You are running pure business logic on a business entity. Trivially covered by unit tests. If isUserSubscriptionActive does 'go wrong', fix the damn code, rather than decorating your coding mistake as a checked Exception. And if it really can't be fixed, you're in 'let it crash' territory anyway.
* I took a jab at testing, and now at least one of you's thinking: "Well how do I test UserService.isUserSubscriptionActive if I don't make an IUserRepository so I can mock it?" Look at the code above: UserService is passed a User directly - no dependency on UserRepository means no need for an IUserRepository.