Tldraw pauses external contributions due to AI slop
github.com133 points by pranav_rajs 12 hours ago
133 points by pranav_rajs 12 hours ago
We've enjoyed a certain period (at least a couple of decades) of global, anonymous collaboration that seems to be ending. Trust in the individual is going to become more important in many areas of life, from open-source to journalism and job interviews.
I've been trying to manifest Web of Trust coming back to help people navigate towards content that's created by humans.
A system where I can mark other people as trusted and see who they trust, so when I navigate to a web page or in this case, a Github pull request, my WoT would tell me if this is a trusted person according to my network.
You need a very complex weighing and revocation mechanism because once one bad player is in your web of trust they can become a node along which both other bad players and good players alike can join.
Then I can see who added that bad player and cut off everyone who trusted them (or decrease the trust level if the system allows that).
Build a tree, cut the tree at the first link, now you get rid of all of them. Will have some collateral damage though, but maybe safe to assume actually "good players" can rejoin at another maybe more stable leaf
It's a web, not a tree... so this is really not that simple.
Yeah, that's the problem, and my suggestion is to change it from a web to a tree instead, to solve that issue.
> global, anonymous collaboration that seems to be ending. Trust in the individual is going to become more important in many areas of life
I don't think it's coming to an end. It's getting more difficult, yes, but not impossible. Currently I'm working on a game, and since I'm not an artist, I pay artists to create the art. The person I'm working closest with I have basically no idea who they are, except their name, email and the country they live in. Otherwise it's basically "they send me a draft > I review/provide feedback > Iterate until done > I send them money", and both of us know basically nothing of the other.
I agree that trust in the individual is becoming more important, but it's always been one of the most important thing for collaborations or anything that involves other human beings. We've tried to move that trust to other system, but seems instead we're only able to move the trust to the people building and maintaining those systems, instead of getting rid of it completely.
Maybe, "trust" is just here to stay, and we all be better off as soon as we start to realize this, and reconnect with the people around us and connect with the people on the other side of the world.
Your tone is disagreement, but it's not clear why?
There is an individual who you trust to do good work, and who works well with you. They're not anonymous. Addressing the topic of this thread, you know (or should know) that it is not AI slop.
That is a significant amount of knowledge and trust in an individual, and the very point I thought the GP was making.
trust in trust.. as programmer would say
the web brought instant infinite 'data', we used to have limits, limits that would kinda ensure the reality of what is communicated.. we should go back to that it's efficient
Seems like reading the code is now the real work. AI writes PRs instantly but reviewing them still takes time. Everything flipped. Expect more projects to follow - maintainers can just use ai themselves without needing external contributions.
Understanding (not necessarily reading) always was the real work. AI makes people less productive because it's speeding up the thing that wasn't hard (generating code), while generating additional burden on the thing that was hard (understanding the code).
There are many cases in which I already understand the code before it is written. In these cases AI writing the code is pure gain. I do not need to spend 30 minutes learning how to hold the bazel rule. I do not need to spend 30 minutes to write client boilerplate. List goes on. All broad claims about AI's effects on productivity have counterexamples. It is situational. I think most competent engineers quietly using AI understand this.
In these cases AI writing the code is pure gain.
no, it isn't. unless the generated code is just a few lines long, and all you are doing is effectively autocompletion, you have to go through the generated code with a fine toothed comb to be sure it actually does what you think it should do and there are no typos. if you don't, you are fooling yourself.
> There are many cases in which I already understand the code before it is written. In these cases AI writing the code is pure gain.
That's only true if the LLM understands the code in the same way you do - that is, it shares your expectations about architecture and structure. In my experience, once the architecture or design of an application diverges from the average path extracted from training data, performance seriously degrades.
You wind up with the LLM creating duplicate functions to do things that are already handled in code, or using different libraries than your code already does.
> There are many cases in which I already understand the code before it is written.
Typing speed is your bottleneck?
The problem is, even if all that is true, it says very little about the distribution of AI-generated pull requests to GitHub projects. So far, from what I’ve seen, those are overwhelmingly not done by competent engineers, but by randos who just submit a massive pile of crap and expect you to hurry up and merge it already. It might be rational to auto-close all PRs on GitHub even if tons of engineers are quietly using AI to deliver value.
I mean we did copy/paste before this? Also create-react-app is basically that. And probably better than a stochastic AI generating it.
As not all codebases are well-written, I have found useful once to get an LLM to produce code that does X, essentially distilling from a codebase that does XYZ. I found that reviewing the code the LLM producced, after feeding the original codebase in the context, was easier than going through the (not very well-written) codebase myself. Of course this was just the starting point, there was a ton of things the LLM "misunderstood", and then there was a ton of manual work, but it is an (admittedly rarer) example for me where "AI-generated" code is easier to read than code written by (those) humans, and it was actually useful having that at that point.
Very much disagree. When I type code I don't just press keys, I read, think, organize .. and the interplay between acting, seeing, watching, reevaluating was the fun part. There's a part of you that disappear if you only review the result of a generator. That's why it's less interesting imo
It makes a great code reading tool if you use it mindfully. For instance, you can check the integrity of your tests by having it fuzz the implementation and ensure the tests fail and then git checkout to get clean again.
AI makes people less productive because it’s speeding up the thing that was hard: training AI for better future AI.
The productivity gets siphoned to the AI companies owning the AI.
Reviewing code is much less of a burden if I can trust the author to also be invested in the output and have all the context they need to make it correct. That's true for my team / tldraw's core contributors but not for external contributors or drive-by accounts. This is nothing new and has up to now been worth the hassle for the benefits of contribution: new perspectives, other motivations, relationships with new programmers. It's just the scale of the problem and the risk that the repo gets overwhelmed by "claude fix this issue that I haven't even read" PRs.
this is precisely why i refuse to use AI to generate code at all. i'd have to not only read it but internalize it and understand it in a way as if i had written it myself. at that point it is easier to actually write the code myself.
for prototypes and throwaway stuff where only the results count, it may be ok. but not for code that goes into a larger project. especially not FOSS projects where the review depends on volunteers.
In the civic tech hacknight community I'm part of, it's hard to collaborate the same now, at least when people are using AI. Mostly because now code often feels so disposable and fast. It's like the pace layers have changed
It's been proposed that we start collaborating in specs, and just keep regenerating the code like it's CI, to get back to the feeling of collaboration without holding back on the energy and speed of agent coding
> Mostly because now code often feels so disposable and fast
I really like this thought. We used to take pride in elegant solutions and architectural designs. Now, in the era of shipping fast and AI, this has been disregarded. Redundancy is everywhere, spaghetti is normalized. AI code has always been unsettling for me and I think this is why.
Think 1 <pick currency> shops, now that factories have fully taken over.
I see a future where those that survive are doing mostly architecture work, and a few druids are hired by AI companies.
Clowns will just use LLMs to post slop comments in the spec discussions.
I actually think Ada has good potential as an AI adjacent language because the syntax is optimised for readability (I personally find it very readable too.)
Using a coding agent over days on a personal project. It has made me think
1. These llms are smart and dumb at the same time. They make a phenomenal contribution in such a short time and also do a really dumb change that no one asked for. They break working code in irrational ways. I’ve been asking them to add so many tests for all the functions I care about. This acts as a first guard rail when they trip over themselves. Excessive tests.
2. Having a compiler like Rust’s helps to catch all sorts of mines that the llms are happy to leave.
3. The LLMs don’t have a proper working memory. Their context is often cluttered. I find that curating that context (what is being done, what was tried, what is the technical goal, specific requests etc) in concise yet “relevant for the time” manner helps to get them to not mess up.
Perhaps important open source projects that choose to accept AI generated PRs can have such excessive test suites, and run the PRs through them first as a idiotic filter before manually reviewing what the change does.
This is probably true, and while I expect productivity to go up, I also expect "FOSS maintainer burnout" to skyrocket in the coming years.
Everyone knows reading code is one-hundredth as fun as writing it, and while we have to accept some amount of reading as the "eating your vegetables" part of the job, FOSS project maintainers are often in a precarious enough position as it is re: job satisfaction. I think having to dramatically increase the proportion of reading to writing, while knowing full well that a bunch of what they are reading was created by some bozo with a CC subscription and little understanding of what they were doing, will lead to a bunch of them walking away.
i have fun reading code, but the fun comes from knowing a human did this. if i find errors i get the satisfaction of teaching that human become a better developer by helping them realize the error and avoid it in the future. if the code is the contribution of a volunteer to a project of mine, even more so. that all goes out the window with AI generated code.
Not to worry! Microslop probably has a product in the works to replace disgruntled open-source maintainers with agreeable, high-review-throughput agentic systems.
Generally speaking, the value of these contributions was determined by "proof of work". Time and effort are precious to a human hence its a somewhat self-regulating system preventing huge amounts of low quality contributions being generated. This is now gone. Isn't that an interesting problem to fix?
That's interesting; another project stopped letting users directly open issues: https://news.ycombinator.com/item?id=46460319
Check Ghostty "CONTRIBUTING.md#ai-assistance-notice"
"The Ghostty project allows AI-assisted code contributions, which must be properly disclosed in the pull request."
Mitchell Hashimoto (2025-12-30): "Slop drives me crazy and it feels like 95+% of bug reports, but man, AI code analysis is getting really good. There are users out there reporting bugs that don't know ANYTHING about our stack, but are great AI drivers and producing some high quality issue reports.
This person (linked below) was experiencing Ghostty crashes and took it upon themselves to use AI to write a python script that can decode our crash files, match them up with our dsym files, and analyze the codebase for attempting to find the root cause, and extracted that into an Agent Skill.
They then came into Discord, warned us they don't know Zig at all, don't know macOS dev at all, don't know terminals at all, and that they used AI, but that they thought critically about the issues and believed they were real and asked if we'd accept them. I took a look at one, was impressed, and said send them all.
This fixed 4 real crashing cases that I was able to manually verify and write a fix for from someone who -- on paper -- had no fucking clue what they were talking about. And yet, they drove an AI with expert skill.
I want to call out that in addition to driving AI with expert skill, they navigated the terrain with expert skill as well. They didn't just toss slop up on our repo. They came to Discord as a human, reached out as a human, and talked to other humans about what they've done. They were careful and thoughtful about the process.
People like this give me hope for what is possible. But it really, really depends on high quality people like this. Most today -- to continue the analogy -- are unfortunately driving like a teenager who has only driven toy go-karts. Examples: https://github.com/ghostty-org/ghostty/discussions?discussio... " ( https://x.com/mitchellh/status/2006114026191769924 )
You conveniently left off the follow up.
> @zeroxBigBoss: .. It's not all AI, I have experience with Zig and MacOS, ..
> @mitchellh: I appreciate it! And my bad on the experience, I must have misunderstood or misremembered your messages
Use xcancel. For the very least to see an entire thread.
Apart from the external person turning out having experience with zig and macos (but not on developing terminals and rendering stuff), this is a good imo example of what ai can be used well for: writing one-off code/tools for which it is enough that it is just working (even if not perfectly), but one does not really care about maintaining, because it is meant to be used only on a specific occasion/context. In this case, the external person was smart enough to use AI to identify the problems and not to produce "fixes" to send as a PR.
Imo, an issue is that the majority of people who submit AI slop as PRs have different motivations than this person (developing a PR portfolio whatever that may mean), or are much less competent and eager to do actual work themselves (which AI use can worsen).
> With luck, GitHub will soon roll out management features that let us open things back up.
I wouldn't bet on it
SlopHub
Curl project has had this issue https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...
Book publishers have stopped accepting unsolicited submissions for the same reason.
You need a literary agent for just about all of them
> An open pull request represents a commitment from maintainers: that the contribution will be reviewed carefully and considered seriously for inclusion.
This has always been the problem with github culture.
On the Linux and GCC mailing lists, a posted patch does not represent any kind of commitment whatsoever from the maintainers. That's how it should be.
The fact that github puts the number of open PR requests at the very top of every single page related to a project, in an extremely prominent position, is the sort of manipulative "driving engagement" nonsense you'd expect from social media, not serious engineering tools.
The fact that you have to pay github money in order to permanently turn off pull requests or issues (I mean turn off, not automatically close with a bot) is another one of these. BTW codeberg lets any project disable these things.
I have an old open-source project that I archived on GitHub (because I do not maintain it anymore). Once a user opened an issue with a completely unrelated project of mine (same user account than the archived one), posting some AI slop with step-by-step click instructions how to unarchive the project and enable issues etc. He spammed the same text to two different email addresses he found from my Github page and the git history. I banned that user immediately from opening issues on that said project, closed the issue and ignored him. Just to receive another outrageous email why I did not comply with his request, and how I would dare to ban him from opening further issues. I swear, the entitlement sometimes on GitHub is unbearable.
They invited AI in by creating a comprehensive list of instructions for AI agents - in the README, in a context.md, and even as yarn scripts. What did they expect?
Hey, Steve from tldraw here. We use AI tools to develop tldraw. The tools are not the problem, they're just changing the fundamentals (e.g. a well-formed PR is no longer a sign of thoughtful engagement, a large PR shows more effort than a small PR, etc) and accelerating other latent issues in contribution.
About the README etc: we ship an SDK and a lot of people use our source code as docs or a prototyping environment. I think a lot about agents as consumers of the codebase and I want help them navigate the monorepo quickly. That said, I'm not sure if the CONTEXT.md system I made for tldraw is actually that useful... new models are good at finding their way around and I also worry that we don't update them enough. I've found that bad directions are worse than no directions over time.
This is my experience as well. I work with AI agents a lot, they are very useful. What's not useful is some passer-by telling the AI "implement <my favorite feature>" and then sending that as a PR. I could have written a sentence to the LLM too if I wanted to, you aren't really giving me or the project any value by doing that.
Now that writing the code is the easy part, we're just going to transition to having very few contributors, who are needed for their architectural skills, product vision, reasoned thinking, etc, rather than pure code-writing.
The CONTEXT.md file was created 5 months ago, and the contribution policy changed today. I would interpret that as a good-faith attempt to work with AI agents, which with some experience, didn't work as well as they hoped.
Wouldn't that be for their usage? It's presence doesn't implicitly mean they want incomplete PRs submitted to their repository constantly.
I still find it useful to vibe code in a private fork. For example with yt-dlp its now super easy to add a website with Claude for personal use, but that doesn't mean it's appropriate to open a PR.
Didn't take long before the quality went downhill.
Skynet was evil and impressive in The Terminator. Skynet 3.0 in reallife sucks - the AI slop annoys the hell out of me. I now need a browser extension that filters away ALL AI.
A LinkedIn comment I made on an adjacent topic:
> If the job market is unfavourable to juniors, become senior.
That requires networking with a depth deep enough that other professionals are willing to critique your work.
So... open-source contributions, I guess?
This increases pressure on senior developers who are the current maintainers of open-source packages at the same time that AI is stealing the attention economy that previously rewarded open-source work.
Seems like we need something like blockchain gas on open-source PRs to reduce spam, incentivize open-source maintainers, and enable others to signal their support for suggestions while also putting money where their mouth is.
> If the job market is unfavourable to juniors, become senior.
Don't love your job, job your love.
At first I aggressively banned anyone that submitted slop to my projects.
Then I just took my hosting private. I can’t be arsed to put in the effort when they don’t.
> <BROWN AND WHITE DRAWING OF AN ASSHOLE> claude added the Task issue type 4 hours ago
is this satire?
I have a GitHub action that labels and tags issues automatically. It also standardizes the issue title. I love this script and would recommend it to anyone. https://github.com/tldraw/tldraw/blob/ce745d1ecc1236633d2bf6...