How good engineers write bad code at big companies

seangoedecke.com

232 points by gfysfm 6 hours ago


sfpotter - 2 hours ago

I've read a few of this guys posts now and have consistently been rubbed the wrong way by them. I think I know why now. It's not that he's wrong. His analysis is reasonable and straightforward. I think it's that the basis for his analysis is ultimately a form of nihilism, coming from someone who (maybe?) used to be an idealist but was burnt by a bad experience and must now explain why believing in anything is misguided.

My instinct after reading this article is to pull back a bit and ask some larger questions. Why is it necessary for big tech companies to act this way? Why does bad code bother engineers so much? Are they actually misguided for feeling like bad code is a catastrophe, or is it really the fault of the broader economic sphere we all inhabit? Is it actually maturity to reconcile ourselves to drift powerlessly as faceless and titanic forces sculpt our reality? So many possible questions.

area51org - 3 hours ago

Maybe it's things like 4-year tenure, or shorter tenure, or something else.

But I think it's a matter of motivation, Bob.

> The thing is, Bob, it's not that I'm lazy, it's that I just don't care. It's a problem of motivation, all right? Now if I work my ass off and Initech ships a few extra units, I don't see another dime, so where's the motivation? ... my only real motivation is not to be hassled. That, and the fear of losing my job. But you know, Bob, that will only make someone work just hard enough not to get fired.

mikert89 - 4 hours ago

what I see alot is that the syntax and overall code architecture is text book, but its the completely wrong approach that creates extremely complicated tech debt. All the code reviews will be on the syntax, and none on the big picture of the business problem, or whether the implementation is overcomplicated.

in the short run (1-2 years) there is no repercussion for this, but eventually making changes will be extremely risky and complicated. The individuals that built the software will lord over everyone else with their arcane knowledge of this big pile of junk

Herring - 4 hours ago

> Big companies know that treating engineers as fungible and moving them around destroys their ability to develop long-term expertise in a single codebase. That’s a deliberate tradeoff. They’re giving up some amount of expertise and software quality in order to gain the ability to rapidly deploy skilled engineers onto whatever the problem-of-the-month is.

And also to "keep the balance of power tilted away from engineers and towards tech company leadership." The author touched on that and forgot about it. You don't want key projects depending on a group of engineers that might get hit by a bus or unionize or demonstrate against Israel or something. Network effects and moats and the occasional lobbying/collusion mean the quality of your product is less important.

bmitch3020 - an hour ago

I'd simplify this post down to this: companies optimize the trade-off between time, cost, and quality by sacrificing quality.

It's not that the goal is to write low quality code, it's that big businesses understand the sales cycle and how to maximize profits. If they over spend on employees, that cuts into their profits or causes the product to be too expensive. And if they spend the time to write quality code rather than developing features, they lose sales. Customers don't buy quality, they buy features at a price, and quality issues (like bugs) get thrown over the wall to downstream support staff.

As much as I dislike this, knowing how unstable it makes the overall software ecosystem, companies aren't wrong for making these decisions. The companies that choose differently don't become big businesses, they either stay small, get acquired, or go out of business.

jeltz - 4 hours ago

I don't really buy that this is the main reason. A good senior engineer is for the most part able to not write bad code from day one, just at a very low speed and with the need to ask other people frequenyly. Even if you do not know the code base or domain yet there are a lot of things you can do to avoid writing bad code. Yes, as someone new you will make mistakes and misunderstand things but a lot of the bad code I have personally seen has not been caused by that. Most bad code I have seen has been caused by people rushing and not having their fundamentals in order. Like not actually doing reviews, not spending a few extra hours to think about architecture, etc. Also a big issue is that people just let the complexity of systems explode for the gain of short term projects.

I think the issue is more that engineers face unreasonable pressure to deliver short term value and that there is no respect for the craft/engineering from many managers or even engineers.

cadamsdotcom - 12 minutes ago

There’s a cost-benefit component missing from the analysis.

“Bad” code is probably “good enough for now” code that was written some time ago on a bet that doing it better would never be needed as it wouldn’t need to change.

Also, “good” code is costly especially if taking longer to build the thing causes the company to miss its market.

ianbutler - 43 minutes ago

I don't think there's an objective assessment of good code. I've been writing code for over 20 years at this point and most times I've seen what people describe as their own good code I disagree with various decisions.

Experience CAN remove pitfalls, though developers even disagree about those sometimes.

Organization, chosen abstractions, naming etc are basically personal thinking and have differed on every team I've ever been on.

When it's been good is when it's been consistent and that's taken a strong personality the team trusted to have authority.

nathancspencer - 3 hours ago

The claim this article makes about very short tenures at big tech is misleading. Because of headcount growth, the median tenure is naturally going to be short. Google grew headcount by 60% the year before 2013, so no wonder the median tenure was 1.1 years. A better statistic to use would be median tenure conditional on that the employee has already left.

pxc - 4 hours ago

I think, sadly, that's often "the job". My career has been good so far, all things considered, but I think it would probably be better if embracing that idea came more naturally to me.

One of my first strange and unpleasant realizations in transitioning from studying computer science to "working in the real world" came in a 1:1 meeting with my manager at my first job out of school. I was complaining about code quality both in the context of some of our existing codebases and some new code one of my peers (also a junior developer) had recently written. When the light bulb finally lit up in my naive little head, the question I asked my manager with a sense of horror and outrage was "... so you're saying they wrote bad code on purpose?". The painful thought was that I, too, would (or had already) found myself tasked with pushing code that I knew sucked, for reasons entirely unrelated to architecture or design or other purely "technical" constraints.

I used to fantasize about moving into a different software niche, maybe in safety critical systems, where correctness is more highly valued. But recently I'm coming to realize that the thing I crave (and miss from my school days) is the joy of the craft— something involving elegance and taste in a way that even the strictest standards of correctness doesn't necessitate.

I think for the most part, truly excellent code isn't something many businesses perceive themselves as needing (even if many practical benefits can flow from its virtues). And, probably, for many businesses, such indifference is right. So excellent code, where it exists, is probably more often "gotten away with", half-snuck in by stubborn engineers who are productive enough to burn time injecting some extra consideration and effort into their code, than it is commissioned by a business which understands that it wants good code.

01100011 - 3 hours ago

IDK, my team at a FANG has an average tenure of around 7 years and the ones less than that are new hires. I keep getting refresher grants every year. I'm sure this article rings true for some people but not me.

anyonecancode - 2 hours ago

Over my career, I've been in a big company twice. This article definitely tracks with my experience. At one company, I think management actively didn't care, and in fact my direct manager was pretty hostile to any attempts at improving our code base as it meant disruption to what was, for him, a stable little niche he had set up.

At the second, it wasn't hostility but more indifference -- yes, in theory they'd like higher quality code, but none of the systems to make this possible were set up. My team was all brand new to the company, except for two folks who'd been at the company for several years but in a completely different domain , with a manger from yet another domain. The "relative beginner" aspect he calls out was in full effect.

tyleo - 4 hours ago

I don’t think the underlying point is true: big companies don’t necessarily write bad code.

A big company is like a collection of small companies. Code quality varies depending on where you are in it.

Similarly, nothing leads me to believe small companies are any better. Some are excellent. Some are nightmare spaghetti.

can3p - 4 hours ago

The other reason is the volume of the code being produced combined with the constant product changes. An innocent change like mixing two close but still different concepts can easily poison the whole codebase and take years to undo and may even be nearly impossible to fix if it propagates to external systems outside of direct control

matt3210 - 4 hours ago

It’s always a trade off between raising the bar and making a deadline. The deadline always wins since the boss doesn’t know how to read code

orwin - 3 hours ago

I did a mistake during an early refactor a year ago (the last refactor just before the code hit production, and any new update on models would demand a db migration), and i architectured and named a data structure poorly. Sadly it was a huge refactor on many part of the code, and we had a small team and few seniors, so the PR didn't catch the mistake.

I noticed an issue with a new feature i couldn't fix in a satisfactory manner monday. I talked a lot, with the lead and the other senior early. First i started doing a shitty fix. Then i asked for a carefull review from the other senior, we discussed the issue and managed to find the origin of all the bad code. Then i asked for more time (well, i "told" more than asked tbh) and did a full refactor, correct this time (hopefully) (the deployment + migration script will run next monday).

Writing bad code happen to everyone, at every company, especially when you don't have a lot of experience and domain knowledge. The issues appear when no one catch this bad code, or when people don't have the time or the latitude to fix it before it corrupt all the surrounding code.

pmarreck - 2 hours ago

Meanwhile, I have a ton of experience, am personable, am highly technical, and can't find something for some reason despite requesting a fairly moderate salary ($180k) given being 53 and having worked in technology for decades.

austin-cheney - 4 hours ago

The short tenure is a symptom of a larger problem. The deeper problem is that very little is expected of big company software employees. Conversely those same employees tend to expect a lot in return. You can call that entitlement, poor expectation management, first world problems, and all kinds of other names.

I have not worked for a FAANG, so maybe things are different there, but I don't suspect so. People are people no matter where you put them.

Increasing compensation is not the solution. It can be a factor in a larger solution, but just increasing compensation increases employee entitlement which makes this problem worse, not better.

The best solution I have seen is risk/reward. Put people in charge of their assigned effort with real adult danger of liabilities. Likewise, award them for their successes. This is called ownership, and it works because it modifies people's behavior. The rewards and liabilities do not have to be tied to compensation. Actually, associating these rewards/liabilities to social credibility within the team/organization appears more effective because it reinforces the targeted behaviors.

I have seen this missing in all of my software career until my current employment. Conversely people in the military are pushed into this liability/reward scenario from the very beginning and its very effective. It has always been striking to see the difference in my dual career progression.

epgui - 4 hours ago

> That’s a deliberate tradeoff.

In my experience, while this line is often repeated, in practice it’s rarely really a “deliberate” tradeoff. Rather it’s mostly accidental.

hkleppe - 4 hours ago

You have to realise there is a almost full complete disconnect between engineering and business value

zkmon - 4 hours ago

The actual coding work in most non-tech big companies, is considered a low-level or dirty work and is delegated to the contractors or junior developers, who just can't bother anyone to get the information. As a result, bad code happens.

Also, the process, security, approvals and compliance could dominate so much that less than 20 lines of code changes per week could become the norm and acceptable.

pkrecker - 2 hours ago

This article repeats the idea that the tenure of SWEs at large tech companies is only 1-2 years, but I don't think this is true, and certainly not at the lower bound of one year. I am not sure about other companies but at Google, where I work, the average tenure of a SWE is over 5 years.

0xbadcafebee - 2 hours ago

I've been at big companies for 12 out of the last 20 years, never a FAANG, just "average" big companies. The rest of the time I've spent at startups and medium-sized companies, and sometimes a startup-in-a-big-company.

I have met maybe 5 good engineers in my whole career. The size of the company did not matter. The reason is, the only thing that exists in our world today that can make you a good tech engineer, is yourself.

When you hear the word "engineer", you might imagine a professional who has done studies, passed exams, has certificates, maybe even apprenticed. They know a specific body of knowledge (which is maintained by some organization), they're held liable for their work. They are masters of their domain and they don't step outside of it.

But not if they're in tech! Then an 'engineer' can be a high school graduate or a PhD. Both can make the same amount of money, and have the same lack of real-world experience and job skills. They will both regularly apply technology they've never been trained on, never learning more than the least possible information to get a proof of concept working (and then that immediately becomes a production service). There's often no record of the decisions they made, no formal design process, no architectural review, no standards listed, no testing required, no risk analysis performed, no security/safety/reliability checklist performed. And they often are dealing directly with PII, with absolutely no thought to how to manage it. And they often have far more access than they should have, leak critical credentials everywhere, don't manage the software supply chain properly, don't even pin versions or even test rollbacks, etc. I have seen all of this at every single company I've worked for.

In any other 'engineering' profession, this would be illegal. Hell, it's sometimes illegal just to change a breaker in a subpanel in your home without pulling a permit, because doing it wrong has consequences. Think of all the times your personal financial records, health records, sensitive data, social security numbers, etc, have been leaked, just in the last year or two. 9 times out of 10 those happened because nobody cared enough to prevent it. But these things shouldn't be optional. There should be some kind of mandatory thing in place to force people to ensure this doesn't happen. And some kind of mandatory minimum requirements about what people know, what they're allowed to work with, and how. None of that applies in tech, yet we still call it engineering.

n4r9 - 5 hours ago

The referenced article Pure and Impure Engineering was discussed a few months back here: https://news.ycombinator.com/item?id=45165753

hereme888 - 4 hours ago

I'm just gonna drop this funny roast song here. Hope it's heard lightheartedly:

https://suno.com/song/d6d77518-16ca-455f-ade1-0e8d08fc4b0b

mtnygard - 3 hours ago

Middle management gets reorged almost as frequently as the engineers. So they have little to no incentive for long term viability of the code either.

- 3 hours ago
[deleted]
nrhrjrjrjtntbt - 32 minutes ago

Another reason for short tenure is to get uplevelled more quickly than is possible internally. I.e. it is easy to get to level X as a candidate than via promotion.

lambersley - 3 hours ago

Remember the Stanford Prison Experiment; "bad company corrupts good people."

stuxnet79 - 4 hours ago

Maybe I have it wrong but the very essence of "engineering" is managing the constraints of (1) providing an acceptable solution to a problem (2) within some fixed parameters of time and cost.

The code may look "bad" in a vacuum but if it yielded a successful outcome then the engineer was able to achieve his/her goal for the business.

The stories shared in this article are exactly what you'd expect from big tech. These are some of the most successful firms in the history of capitalism. As an engineer you are just grist in the mill. If you want to reliably produce "good" code then IMO become an artist. And no ... working at a research facility or non-profit wont save you.

lapcat - 4 hours ago

> They are almost certainly working to a deadline, or to a series of overlapping deadlines for different projects.

I think this is crucial. Even old hands working on their area of expertise can be compromised by deadlines.

- 3 hours ago
[deleted]
znpy - 2 hours ago

This overall makes sense.

In my experience at a FAANG working on one of the core services for both internal and external customers, essentially two kind of people crank out great code:

1. "rock stars": they joined the company at 25 and they're still there at 35+. they're allowed pretty much everything (eg: no RTO, work from home country) and they know many codebases across the services very deep. they aren't really motivated to go look elsewhere, their grass is already one of the greenest.

2. people with kids. the company pays enough. they aren't really interested in switching job, rather they want to provide for their family. they're good, and maybe every now and then will push through for a promo in order to face new challenges in life (another child coming or some kind of new financial burden).

i'm not saying either one is inherently good or bad.

but yeah. in such large companies you end up working in on a very large codebase that interacts with other very large codebases. all the codebases are proprietary and you're lucky if you can use some libraries that come from the outside world (that have not been heavily lobo^H^H^H^H customized - the libraries i mean).

you do what you can, you do your best, but you're essentially a relative beginner.

mberning - 3 hours ago

It is only briefly touched on in the article but most of the “best” engineers spend almost no time coding or engineering. I’ve worked at multiple Fortune 500 companies and many weeks I would be lucky to spend 4-8 hours coding. Often I would just work on things that interest me after hours or on the weekend since it would be unlikely to be bothered. Unless some other unfortunate soul happens to see you are online.

huqedato - 4 hours ago

...and also bad engineers write bad code at small companies.

innocentoldguy - 4 hours ago

I worked for a company writing Elixir code several years ago. Prior to my arrival, the ignorant architect had deployed Elixir in a way that broke the BEAM (which he viewed as "old and deprecated"). Furthermore, one of the "staff" engineers—instead of using private functions as they're intended—created a pattern of SomePublicModule and SomePublicModule.Private, where he placed all the "private" functions in the SomePublicModule.Private module as public functions so that he could "test them."

I tried almost in vain to fix these two ridiculous decisions, but the company refused to let code fixes through the review process if they touched "well-established, stable code that has been thoroughly tested." After being there for a couple of years, the only thing I was able to fight through and fix was the BEAM issue, which ultimately cost me my job.

My point in all this is that, at least sometimes, it isn't good engineers writing silly code, but rather a combination of incompetent/ignorant engineers making stupid decisions, and company policies that prevent these terrible decisions from ever being fixed, so good engineers have no choice but to write bad code to compensate for the other bad code that was already cemented in place.

jeffbee - 2 hours ago

There's quite a bit of cope involved in this discussion, right? As in "I didn't get hired but at least I, a noble artisan, am not compromising my beautiful style"?

elisbce - 2 hours ago

1. In Silicon Valley, people are not bounded by non-compete clauses and can come and go at will. So fungibility is a top priority for any tech company. The only way to do that is to make sure expertise is shared across the team and not monopolized by one or a few old-timers.

2. Eng teams that have mostly old-timers tend to get stale and slow in changes. This is bad for products that need rapid evolution or new ideas to break status quo. New engineers have way more incentives to make changes to prove themselves and collect credits, while old-timers tend to play safe and stay on the side of stability.

3. Bad coders, not new coders, write bad code.

samdoesnothing - 4 hours ago

I think it's more that optimizing your hiring process for leetcode savants selects developers who prioritize algorithmic practice over everything else. They also deprioritize character over raw technical skill. But it turns out you need well rounded developers who are able to work with others, communicate well, and have taste. If your hiring process deprioritizes that, don't be surprised when the software produced is shite.