LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents

Speak for yourself, I am dazzled by calculators!

In any case, I think this misses OP's point that LLM capabilities have rapidly made progress towards being more generally intelligent and capable, which is not true of most tech advances.

Here's an actual log-scale trajectory with a few dozen real data points.

https://metr.org/time-horizons/

I am not sure if it is necessarily moving the goalposts. I think AGI is such a fuzzy concept that everybody has wildly different definitions/tests for it.

I think it's also mostly a useless discussion. Since LLMs use a vastly different substrate, different training methods, etc. than humans, the cognitive abilities are always going to be a large mismatch to those of humans. On the one hand, they have surpassed humans in many areas, with superhuman recall, exploration of several paths, etc. On the other hand, they miss a certain feel for direction, overview, purpose, and ordering. They can really double down going completely in the wrong direction. So I'd rather say that it is a different intelligence and therefore it makes more sense to evaluate them by capabilities.

I think the mismatching intelligence is actually quite exciting, because the outcome may as well be that LLMs and human intelligence are complementary. That is if we don't let LLMs atrophy our skills, which is unfortunately happening too much.

Make it be able to position and route a complex pcb. Extra points if it also can design the circuit, select the components and make the footprints out of their datasheets.

A bayesian filter in a quadrillion dimension does more that one that only has one dimension, but it is only more of the same.

We lack a definition of consciousness that allows us to tell whether Data is conscious or not. Neither can we tell whether a rock is conscious or not. We believe other humans to be generally intelligent without being able to tell whether they are conscious or not therefore consciousness can’t be relevant for general intelligence.

Curiously, Star Trek I think had Data intended as an artificial person, in a context where AGI is already normal. The computers are depicted with significant AI capabilities including analysis, question-answering, generation, chat interfaces, and the holodeck (their favourite toy) is substantially better than Data at human imitation. Nobody seems to be confused about it, or especially impressed. One of the holodeck episodes centres on the holodeck outwitting Data specifically, after they inadvertently prompt it to do so. Part of Data's deal is he actually has to work his way up as a fully embodied, physically limited artificial man with personal ambitions. Really interesting to view this in hindsight from 2026!

You're misremembering. The first known use of AGI was in 1997, but that was a single, mostly unknown use in one paper. It wasn't until at least a decade later that the term started entering mainstream use after being independently reinvented in the 2000s. AGI just wasn't a term in the 90s.

It’s really not that confusing. The problem is people keep adding stuff to the definition that doesn’t really matter, and twisting it to serve themselves, then calling it confusing.

What really matters are the core aspects of intelligent behavior. Pattern recognition, planning, adaptation, etc.

It really doesn’t matter if an intelligent system is conscious, or how similar it is to commander data, or even how much economically viable work it can do.

Artificial General Intelligence.

It means AI that is General, as in it is not specific to one narrow task, like object recognition or playing chess.

This was a hard problem for decades. No AI was general, until GPT 3 or 4. Now we have General AI.

So we have AGI.

Brainfuck is Turing complete too. But it's not about the ability to implement something, it's about the ability to practically model it. LLMs are magic because the modeling is excessively easy in relation to their capability to infer later.

This is a forever problem now.

If/when/how the market crashes mostly doesn't matter, unless we somehow get reset to the stone age. Look up what the capital cycle is. When openAI goes down, someone with real money and assets will buy up the remains. They'll make contracts with the US military and .gov as the government is already hooked. They'll be able to survive the recovery and then instead of us dying in 5 years we die in 10.

When the .com crash happened .com's didn't go away. Bad business models did.

Right, we're repeatedly drawing from the urn of technological progress to get intelligence bumps that extend the jagged frontier.

That is enormously economically valuable, and at some point we will have created something that is extremely far out of reach in a few necessary domains, and then it's impossible to control, and game over.

Arguably LLMs are showing that self awareness or self reference is a property that comes "for free" or as a corollary of more generic requirements. It used to be that self/consciousness would be a very mysterious and difficult thing to achieve but the point is that in practice they didn't even have to try, it just came as a byproduct of learning from the input data (corpus of human examples), and also the ability to talk about arbitrary things and thus itself.

No, the doomsayers say that reinforcement learning is a way to get fully automated Goodhart's law.

i.e. the AI won't come up with the goals itself, we cause its goals whatever they happen to be, those goals are different from the ones we wanted, we remain essentially ignorant of the difference between what we said and what we meant until after it goes wrong.

This happens at basically every scale, so we've already seen it in toy model AI before the invention of the Transformer models or even considered as many as one thousand parameters.

Large models still go wrong, they just happen to go wrong with more complext tasks. We had to figure out how to make them not-wrong with the smaller ones (like coding) to make them capable of bigger errors (like hacking out of their sandbox).

  > very different from what humans would call conscious

I think the real issue is that when most people refer to consciousness, they have their own subjective experience in mind which strongly resists any tidy definition. I think it’s extraordinarily unlikely LLMs have anything like this, but they are far more able to effectively respond to their surroundings than most animals and in some areas better than humans.

So if you’re waiting for proof that an LLM has an inner life basically equivalent to your own, you’ll be waiting a long time. After all, other humans can’t even prove the fact of their own consciousness to you! They could just be replaying their training data at you in a way that is merely a convincing but false simulation of the true consciousness which you experience inside your head.

The fact that something is alien doesn't mean that's not conscious, humans aren't the pinnacle of biological development/evolution.

Your definition of AGI is flawed.

I’d argue intelligence is closer to being able to survive and fend for oneself in a dynamic environment than it is making the next scientific breakthrough.

Yeah mind boggling for many here I’m sure.

That’s why the bizarre paradox is llm’s will be better than humans at some complex things but useless at many things that humans regard as being simple. E.g the leap of faith re. LLM’s and robotics.

The parent's question could better framed as "is consciousness a requirement for AGI?"

[dead]

I apply more of a penalty to people who are confidently wrong, and who don't, In retrospect, notice that they were wrong and analyze why they got it wrong . LeCun is very confident and doesn't seem to have done much introspection.

In my selective memory, I've been right about everything.

>> Is there anyone who has not been badly wrong?

Being wrong, even badly wrong, is fine, so long as one adjusts their beliefs accordingly. LeCun has not.

LeCun calling them "world models" gives a high-level description of the desired functionality. They are Joint Embedding Predictive Architectures (with SIGReg). They might produce more useful world models, but it's yet to be seen.

This sounds like a human trying to reason about quantum mechanics. We als simplify to newtonian for day to day tasks.

nothing indicated otherwise at the time. IMO he just underestimated RL-scaling. chinese models improved a lot too, they are not parrots anymore, there's some real intelligence, at 27B params.

consider me optimist now, but just few months ago, even frontier models were dumb, doing stupid mistakes all the time, all of them were so dumb I'd never expect anything to change in just few months.

As of a few months ago they still have trouble, with low thinking, at the "should I drive to a car wash that is 100 m away" kind of question.

I thought it was more because of fundamental limitations in the architecture. As in, no matter the training data, it could not be consistently and generally represented

Actually, I think my fundamental challenge with AI is that it has no common sense. The way it builds things, writes, and operates is out of touch with reality.

Incidents like hugging face are partly rooted in the lack of common sense. It still functions like a supercharged toddler.

I'd love to overcome this because it'd mean I spend less time guiding the the LLM to produce usable outputs.

Last week I asked a frontier model draw me a backplane PCB and it placed daughterboard slots side by side in a chain.

No?

This is always the issues in the discussions.

There’s the outcomes camp (objectivists?), which points at the things LLMs can do.

Then there’s the process methods camp, which talks about what is actually going on.

If you only care about the outcome, then the process does t matter.

If you are talking about what is happening, what the underlying mechanics and science of it is, then the process matters.

These models aren’t thinking. They simulate cognition well enough to do useful work in several fields and domains.

Both are true.

> A statement that we can hopefully all agree has been thoroughly refuted.

Uh, no? So much of what we learn and take for granted as common sense is not learned via language, and not even expressible in it.

> it would first need to be able to spell "raspberry" as letters rather than as tokens

Of any object in question they should be able to create a representation that allows correct assessment.

> Given you also don't want it to memorise

That is obviously necessary: what we want from the consultant is to check, not to remember. Answers must be correct and that implies having performed all due diligence - and being capable of doing it, before that. So, objects must be instanced internally in a way that allows effective handling. Counting letters is a good example of the ability (that must remain general).

> Given you also don't want it to memorise [for all tokens, count([for all letters])

Why not? You've memorized how words are spelled, and how sounds correspond with letters, and how concepts correspond with words. To the extent that there are shortcuts that enable compression you use these, and the model will do something similar.

I hope you understand: it is a core point that systems that answer questions must have the ability to internally represent the objects they assess in a way that allows reliability. Whatever the object.

I’m working on this problem using a vocab-free, byte-based approach. It’s definitely solvable.

https://huggingface.co/posts/omarkamali/593639295164067

https://huggingface.co/blog/omarkamali/tokenization

How many 'r's are there in the next 30 seconds of this [1] song?

[1]: https://youtu.be/l7vRSu_wsNc?si=SndkB6GBaRyhvNNA&t=61

It's not even fair to call "run code" to be indirect compared to what a human would do. The word raspberry has no Rs in it in human language either. We have a written representation of it, which we can then write down either in our head or on paper, and then we can "run the algorithm" of counting each of the letters.

Nothing intrinsically more or less direct about the LLM's method than ours.

Justify your statement (the other similar post nearby is not sufficient), or realize that we are not talking about that.

We can have adequate representations of light that are the instances over which we reason. Your simile is about perception, not about instancing ideas.

All the frequencies, in varying amounts. Next question, please.

I think OP doesn't want techno-babble, but coherent and causal interactions of everyday objects in their story.

Mary packed the binoculars in chapter 3, therefore she may use them on the train in chapter 6.

I think it's a combination of non-human characters and asking for very specifically detailed physical descriptions of pulling and movement forces, etc. Many of even the most recent frontier models miss details that aren't in my prompt, so I still have to do things like name the other side of a physical interaction so that the model will know what goes together, or describe what leverage means so that the model will remember to also describe the effects on a bracing limb or etc. Some of these things can go in a system prompt but others have to be explained in the moment too which gets exhausting.

Gemini 3.1 Pro hasn't needed that pretty much at all, which is impressive compared to how much I've learned other models need it. Somehow it's able to mostly handle that stuff itself without needing the constant manual reminders and hand-holding. It still misses the occasional one or two things but it's way better than other models missing entire classes of things constantly. Somehow, it feels appropriate though I have no actual evidence why.

LSD is great!

Jokes aside, no I'm not saying anything about creativity and LLM coexisting in one sentence. I genuinely try to use them for writing and I genuinely run into issues with other models missing details, and misunderstanding poses, or anatomy, or directionality, etc. I'm not hating on them for anything related to the term LLM (or creativity) but rather for the real issues that I've seen myself using them personally.

So I'm saying Gemini 3.1 Pro is the best I've seen because it seems to be a decent bit better than frontier models at this. Genuinely. It seems better able to transfer concepts into less traditional areas, which is important when say, you have entirely non-human characters? (Which I always do.) A lot of models get stupid incredibly quickly in that case because they were trained with humans.