Git at any scale
cursor.com188 points by meetpateltech 2 days ago
188 points by meetpateltech 2 days ago
It’s hard to overstate the reputation of the author of this post. Everything good about GitHub’s internal systems seemed to have his name all over it (I realize that today this statement hits different than a few years ago). Our times at GH didn’t overlap much but hearing the fact that he’s working at cursor increases my estimation of their engineering org by leaps and bounds.
Is this enough to counteract the reputation of his new boss, Elon Musk?
<edit>: there is lots of demand for a GitHub alternative due to poor uptime, poorly implemented components (CI), and Microsoft stewardship. Switching, especially at this early stage requires a huge component of trust.
[flagged]
Did you really create an account for this?
its literally elon getting mad and making an account calling people cock suckers
tired of this
Just like with climate change, the problems Elon creates don’t go away just because people are reminded of them.
He still pushes his agenda, he still has lots of followers who believe his nonsense
There's a reason you say that with a throwaway account. Even you know it's shameful.
Well, there's going to be a lot more of it. You can't be an outright ultra-right wing racist and expect that it doesn't cause brand damage. It never made business sense for Musk to act the way he does, but he couldn't help himself. Do not expect people to just be ok with it.
Took me some thinking to figure out where this design is not optimal.
The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already.
Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services).
For performance the objects need to be cached locally and requests routed to nodes where the objects are cached locally.
> What about consensus? Elections? Which server is the primary for a given repository? It also doesn't matter! There's no state and no consensus here. Any server can be the primary. All updates to the write-ahead log are synchronized with an atomic compare-and-swap (CAS) operation on S3, so it's always safe for any instance of a repository to receive a push.
Again reminded of what an amaizing piece of engineering S3 is (99.999999999% - 11 nines of durability) [1]
1: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu...
> The productivity cost of five minutes of downtime in your CI system is hard to quantify in dollars, but it is, by any measure, a humongous amount.
Not sure the last month GH had fewer than five minutes of downtime - but I think it's been a while
There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a proprietary product, not an algorithm.
https://cursor.com/en-US/blog/git-at-any-scale to avoid automatic translations (couldn't find a button to switch)
The language selection is in the footer. Interestingly, the URL for English is the original post without en-US.
It's time to look back on S3's 20-year history
> The "fan-out" is synchronized with a classic consensus algorithm called 3PC (three-phase commit) so that a push is only accepted if a majority of the nodes acknowledge it.
Doesn't 3PC require all nodes to agree, not just a majority?
I would think so - outside this sentence the author indicates they're talking about consensus across all members.
IE - the interactive diagram and note about latency being bounded by the slowest replica
The key part: they decided to use the unchanged git codebase as their building block. Likely assuming that the people who wrote the Linux kernel probably knew what they were doing. That explains some features that look bizarre from the generic distributed-database perspective. I think the author has had this argument many times, because he reiterates it several times in the post. The rest follows from it.
That was a very well written and interesting article. I really enjoyed it. Some good background for people telling GitHub to "just scale".
I don’t see a button to switch to English and the German text is a pain to read, comparable to Reddit auto translate..
The button is apparently in the footer: Another comment earlier has the english-only link: https://news.ycombinator.com/item?id=49376960
Remember, don't put large objects in cnt, it makes it a real pain later.
cnt?
Continuity system of Cursor. Brilliant name, just like the octo"cat"! Elon will approve.
WAL on S3 is the new normal.
how does git lock the repo on extreme concurrency? let's say 1000 devs writing to the same repository? eventually you'll have saturation of the resource locking everyone out.
When I've worked at big companies they had bespoke SCM that resolved this. But i'm curious about mid size teams that are big enough to overwhelm git but not so that they can rewrite it.
If those 1000 devs write to different branches then you only take a light lock on the branch, usually that is not the problem.
The problems start when there is a lot of changes on one branch and usually the first band-aid is to put a commit queue in place. The few very big companies that need even more than that probably aren't going to put their code on a new git hosting provider tomorrow.
that sounds like a practical solution once you reach that size. yeah you're right i think only the refs are locked, the rest of the objects can all be submitted concurrently, so it would be high contention refs/branches that would be a pain point.
It doesn't matter how many technical blog posts they put out, I will still never host my code with Elon Musk.