If math is more than proof, we need to better celebrate the rest of it
terrytao.wordpress.com342 points by num42 a day ago
342 points by num42 a day ago
I’m reminded of that famous debate between Poincaré and Hilbert at the International Congress of Mathematicians in Paris in 1900. It was then that everyone decided to follow Hilbert’s path, and proof came to be valued more than intuition. I think modern math at school and at applied university kind of lost this intuitive part.
I try to teach my students that mathematics is, first and foremost, a very precise language of communication. It’s sometimes amusing to ask those who don’t like math to do without it entirely, just to see how much harder it becomes to describe the things around them.
Second thing I tell them, formulas are the essence of mechanisms in their purest form. And in this form, they’re much easier to grasp and mentally manipulate. It always amused me, after taking a mechanics course, to imagine that for any formula, you could visualize a mechanism or process that implements it.
And third thing, I suppose, the ability to verify one’s own statements as proof. Although, of course, mathematicians would probably tear me apart here for my heresy:sorry, I’m not a mathematician, but an engineer. You can make mistakes by using incorrect assumptions, but at some point, analysis itself will show you that you were mistaken. There’s a wonderful book, How to Prove It by Daniel Velleman, which provides an introduction to proof for the uninitiated like me. I really enjoyed it.
As a math prof, I care(d) much more about proof than intuition, not because proof is more important, but exactly because intuition is (I'm a bit Chesterton-ish here haha). You cannot do proof without intuition hence, if you emphasize proof, intuition will take care of itself. Whereas if you emphasize intuition, students won't have any idea of what a rigorous proof should be.
As someone who mostly only applies math, that strikes me as a peculiarly academic take. Intuition is more important for me because it’s what enables me to know what methods are most applicable to whatever practical problem I’m trying to solve. The proof’s purpose is to verify my intuition. It’s just a means to an end. I only take the time to do my own when I can’t confirm what I need from a textbook or paper.
> As someone who mostly only applies math, that strikes me as a peculiarly academic take.
Yeah I was talking strictly about preparing students to become pure mathematicians. No opinion here on other goals.
If what you teach is proofs, then wheat you will filter for are students who live proofs.
And if your job is to train people to become mathematicians, that is absolutely what you should be doing.
The idea the proofs are the heart and soul of mathematics is an unfortunate unforced error, and will lead to the death of the professions now that machines are better at making proofs.
Love is more important than breathing. It is and it isn't.
What good is an end you can't reach, or worse, you can reach but it's wrong?
>You cannot do proof without intuition hence, if you emphasize proof, intuition will take care of itself.
This isn't always the case. Our algebra (or analysis) course focused a lot on proofs for the exam. The result was that a lot of people learned the proofs by heart.
I think intuition is hard to test in a way that feels 'fair'.
You can do it - I doubt you could have got a first when I was at Oxford just by learning and understanding the material, but you should probably have been able to get an upper second. The final part of every question virtually always involved insight, but you'd obviously then have to prove what that insight helped you understand.
If you give people questions like those, there is the risk of complaints about the university not having been taught the material for the exams I guess, or you might find that nobody can answer those harder intuition parts. Certainly most students at Oxford couldn't answer that many of them - you needed to answer about three 'final' parts out of about ten questions say in each three hour exam to get a first and perhaps about 20 percent of students got firsts?
An aside, but tests should be that difficult. Otherwise you aren’t getting any signal at the high end. Even in the face of grade inflation that signal can be translated into grad school recommendations (in the uk case, getting all 10 might even garner a “really not bad.”)
Same here, but I didn't memorize the proofs, I tried to internalize their logic, so I could reconstruct them on demand by just thinking systematically. It did work for me pretty well on my real analysis final exam IIRC (27 years later).
Analogous to the Archimedean Property - there is no approach to teaching mathematics so intrinsically good that it cannot be done poorly enough to yield arbitrarily bad results.
Surely this implies these LLM generated proofs require the LLMs to have mathematical intuition . . . and honestly I don't think many people believe that, and rightly so, certainly not in the way Poincaré was on about.
Maybe it's been done, but I'd like to see an LLM recreate Euclid from questioning without having seen it during training.
> Surely this implies these LLM generated proofs require the LLMs to have mathematical intuition . . . and honestly I don't think many people believe that
Yes, I believe that, it's part of what I was implying (I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs)
Appreciate the clarification, even if I disagree!
I think we differ on what "mathematical intuition" is then. I've seen people that do well in undergrad math degrees simply by massively memorising things and learning how to join them up to some level of degrees-of-separation, but seemingly completely fail to understand, for ezample, why even calculus is how it is. Because they are able to regurgitate the results and "produce proofs" this is never questioned.
The Euclid example also shows my bias towards spatial intuition of mathematical concepts (which is deeply unfashionable) but also exposes exactly where at least current LLMs break down; they do the symbol based pattern matching version, but they cannot leap outside of that, at least today.
This feels very related to the issues re: the presence or absence of world models in LLMs. Insofar as they have world models (or "intuitions"), these would seem to have to be primarily verbal-linguistic (or symbolic, when using math). LLM world models are not likely (currently) very spatial, in contrast to e.g. V-JEPA-2 models, which likely do have some basic spatial models (and perhaps "intuitions").
Yes, I think the augmentation of LLMs with (hopefully eventually higher dimensional) world models will prove very interesting for all this.
> I've seen people that do well in undergrad math degrees simply by massively memorising things and learning how to join them up to some level of degrees-of-separation, but seemingly completely fail to understand, for ezample, why even calculus is how it is. Because they are able to regurgitate the results and "produce proofs" this is never questioned.
If you want to catch them, surely you can find proofs they aren't able to produce.
That's easy: basically all the spatial ones.
I used to be a game dev, and one of the interview questions someone came up with consisted of working out the surface area of a variant of Menger sponge to some given level of depth. The bifurcation for people that could do this vs those that couldn't was incredible, and did not follow obvious trends for academic achievement. (The same interview also included the gem "How wide is a pointer?" which also catches a frightening number of people).
I think it's actually the opposite. Intuition is the only thing LLMs can do, which is why they are prone to hallucinating when they can't validate their intuition against reality.
There is an idea that human intuition, expertise, and critical thinking are largely pattern recognition. When you encounter a situation, your brain gives you a plausible starting point, based on what it has experienced before. You then continue with explicit reasoning, which is slow and inefficient, and try to validate your ideas. The more relevant the patterns you have learned are to the situation, the more likely you reach a useful conclusion.
LLMs are largely the same, except that they cannot learn from experience in normal usage. And except that they experience the world only through symbolic data, while the human brain has access to plenty of sensory data.
My hunch (or intuition, hah!) is that intuition is an instinctive mental shortcut required to navigate large problem spaces that can’t entirely fit into our heads.
Maybe LLMs do not need intuition because they can scale their “cognitive capacity” with hardware and brute force their way through these problem spaces.
> Maybe LLMs do not need intuition because they can scale their “cognitive capacity” with hardware and brute force their way through these problem spaces.
My view is that is certainly true of smaller LLMs but becomes less true as they scale up.
To quote the parent bananaflag in a sub-comment:
> I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs
I think as the sort of spare space adjacent to pure language processing in LLMs grows the probability of the sort of reasoning bananaflag is getting at (or spatial reasoning, or anything else) emerging in that space grows enormously.
One of the questions for AI development over the coming months or years is going to be if deliberately cultivating the architecture of those sub models for specific reasoning types beats any emergent reasoning mechanisms or not.
Hmm we may be talking of two different interpretations of intuition here. I agree that LLM weights contain representations of abstract concepts, as a lot of prior research has shown. This surely includes Mathematical concepts.
But to me that is analogous to what human brains do, and a bit different from intuition. I think of intuition as “heuristics”, typically developed through experience, that may link seemingly unrelated concepts via vague, hard-to-define associations, but which let us make mental leaps (or shortcuts) while reasoning. (Maybe analogous to System 1 / 2 thinking.)
On the other hand, LLMs can do both: build “intuition” from patterns in data AND brute force a huge amount of potentially unrelated concepts. This gets fuzzier when we realize that even these “concepts” themselves are gleaned from patterns in data! But my point is we necessarily have to take shortcuts to scale, whereas machines can scale with hardware.
This is of course a layman theory! But it could explain why these models are progressing so fast.
Yes, in my case "intuition" comes a lot from visualizing things spatially, manipulating them, and being able to capture their properties in equations/proofs, and it's that which is (currently) conspicuously missing when dealing with LLMs. (And may yet appear with world models).
With the alternate view of intuition that many of you are describing it is clear LLMs are somewhat either there or heading there now.
This is an intriguing observation! LLMs were famously bad at spatial reasoning, until Astra which apparently has a huge improvement. I wonder if that has any bearing on the recent jump in Mathematical performance?
One thing that struck me from Dario's last podcast with Dwarkesh was that he said training LLMs on a diverse set of tasks does not make them better just at those tasks, but they get better at unrelated and other tasks overall. What you described could be a concrete example of how that dynamic works!
>Surely this implies these LLM generated proofs require the LLMs to have mathematical intuition
Well yeah they do, obviously.
LLMs have LLM intuition, not human intuition. (See the movie Her.)
LLM cannot reinvent Euclid from scratch, but a larger system including LLM might.
It does not imply that. He is talking about how people do math. Intuition is what you use when deciding what to try and how to think about things.
Proof is the rigorous outcome.
LLM running probabilistic loop is different kind of process.
The parent comment literally said "You cannot do proof without intuition".
Therefore, according to that logic, an entity producing proofs must have intuition.
Edit to add: the parent commenter has now confirmed my interpretation of their statement.
Your unstated major premise here is that their intent was to make a universal statement about how proofs work and not just talking to humans about how they teach humans.
That premise seems unlikely to be correct.
Why? The entire subject of conversation is triggered by things which are not humans producing proofs.
If it's possible for a machine to produce a proof without intuition then clearly a human could also do it too. (And in fact I'd argue I've seen many people like that, simply very good at pattern matching over memorised items).
Because regardless of the point TFA is making, that interpretation makes less sense for the specific comment. It doesn’t fit with the immediate context, which was a response to a thoughtful comment about how humans do math. And it requires assuming a math professor doesn’t understand a very basic and obvious thing about their area of expertise.
That doesn’t really read as good faith engagement in the discussion. At best, it reads as being so AI pilled that you can’t even fathom that others might want to have a little side discussion about something other than AI.
You realize the commenter has now confirmed my interpretation was right?
What is up with this whole sub thread of obvious hole digging?
That math prof was talking about his stance in discussion between mathematicians long before AI.
Plus, I studied math, I am from that environment. His description matches how math is done by people.
People who are good at pattern matching and memorize are, frankly, shit mathematicians. They are find in fun culture around math, but rarely in actual math. They cant really do it as science.
As a math professor, I care much more about the key idea, heuristics, and motivation than the proof. With the others in place the proof is clear, something an AI or a student can do.
Well, it's knowing when to push and when to not. You probably have an intuition for, I don't know, abstract algebra objects (I don't know your field of specialty :P), without needing to symbolically manipulate all of it, but you developed a deep intuition for them through many proofs and attempts at proofs with them.
Im glad you brought up abstract algebra—that was the one class in my math undergrad that I never developed an intuition for. I learned to do the proofs by pushing symbols around and putting bars on top of them but I never felt like I understood what was happening.
That's leaning into engineering, away from math. Heuristics aren't always accurate. Math history before proof is the history of delusion. Idea, heuristics, and motivation aren't nearly enough for correctness outside of a sandbox.
To agree:
In my experience, proof is the gym reps that allows you to harness strong intuition elsewhere.
In practice as an engineer, intuition is far more useful, eg, being able to “feel” when something is off in our reasoning — but proofs are where I train those same sensibilities on “harder” problems, (eg) details about how to model identity, equality, and equivalence in a formal model.
Engineering is a religion based on faith. Math is the god you follow :-)
Not really: engineering is explicitly empirical compared to other fields — and mathematics serves as ontology for that experience.
There’s not faith involved.
There could be meta-faith, if you insert engineering into
Mathematics, physics, chemistry, astronomy, march in one front [lockstep]. Whichever lags behind is drawn after. Whichever hastens ahead helps on the others...
--Karl Schwarzschild
The problem or nonproblem before (elite) software engineers were pointed at rather bespoke conjectures, depending on one's specific denomination, was that the frontier mathematicians got too far ahead of the others to effectively drag them along (hence Tao's recent fundraising attempt using his one-off compressed-sensing work)
There is also the Experience<->Understanding "wave equation" if you will, codified by the popular engineers' joke about how mistakes/bugs mediate the two
Imho what academia+industry really need are GLM-wielding plumbers cheap but capable enough to find these abstraction leaks between silos. One taxes these plumbers so brutally that their clients can get by on basic tokens (morally speaking, so as not to drive demand in the farflung silos of billions bottles and babes)
There were interdisciplinarian buzzwords but these did not live outside the grant proposal, and probably won't survive better under the reign of Pangram
Similarly, programming is also a precise language of communication. Initially, we focused on direct machine behavior but every abstraction above the hardware (including assembly) has been to make that behavior legible to humans.
Developed notations and shared procedural abstractions have made thinking about computation more intentionally human and source control has established a protocol for conversing with other humans in the language of a program and changes to that program.
The moment just now feels like a neglecting of the idea of communication being central. If the program is a compile target but not sufficiently legible or if the conversation moves too quickly for us to keep up then we retain the effects of computation but loose its meaning as communication. We loose the understanding and the ability to develop and evolve further shared abstractions.
Open source programs could be more like motivated explanations of computation. For open source to survive, maybe we should start to make the distinction between free product distribution and programming as communication and community building.
Programming was never about communication. It was always about making the machine do the thing we want. Back in the day, a good game programmer knew which time intervals had writable video memory and which CPU cycles drew which scanlines, and spent more time rearranging the code to hit these timings than to write the actual algorithm. Later programmers (I hesitate to call them good) learned everything there is to learn about OS internals and wrote theoretically nonsensical and invalid code that still worked thanks to those internals, to save CPU cycles and especially memory use. And the next generation of programmers took the principles of late binding and abstraction to the logical extreme and created architectures that cannot be described in words anymore, only in diagrams - but are crazy good for code reuse, traceability and A/B testing.
> Open source programs could be more like motivated explanations of computation.
It is already that. Every time a method/function is created, a structure is defined, a variable is added, a file is created or renamed,… It’s all for the purpose of human communication. The computer only need binary in a single file.
But people feels like they should be able to jumpninto curl code without any understanding of networking, or linux code with no knowlede of computer architecture. Few code are meant for total beginners.
> There’s a wonderful book, How to Prove It by Daniel Velleman
the name sounds familiar but i don't think i have read that one, i did enjoy "introduction to mathematical reasoning" by eccles.
personally my relationship with mathematical proofs has been complicated. it took some work to understand basic proofs (dedekind cuts, ideas vs. instructions with mathematical notation), but all of the theory of computation proofs, which supposedly are difficult for many, were completely intuitively easy for me.
i think mathematicians are facing a similar confusion as computer programmers. the medium used to require precise thinking and the simple act of reading, writing and composing it was a mechanism for thinking and learning. in the llm era, the question is: should there be a new mechanism and if so, what should it look like?
People often hate math because it was not explained to them correctly, usually by people who are good mathematicians but know close to nothing about teaching.
It was so infuriating to see everyone in the class absolutely fail on a specific subject and the "teacher" assumed that everyone must be stupid then. No self reflection, no questioning himself why he is not getting gaussian distribution in marks, just straight Fs.
> usually by people who are good mathematicians but know close to nothing about teaching.
I higly doubt that. Maybe in university level courses. Most people’s only experience with mathematics is an elementary or high school teacher who were probably themselves at best mediocre at the subject. Simply because of selection factors. Those who are good at math are encouraged to go into STEM. There will be of course exceptions everywhere, but that is not what “usually” happens.
And thats just about being good at maths the school subject, which is distinct from being “ good mathematicians” the science / research topic. Mathematicians are few and far between, simply because it is a specialist subject. There just aren’t enough of them to go around for them to be the formative experience around math for most people.
I think the problem is partly circular. Most people do not like maths. This includes most primary school teachers - in places I know primary school teachers are not subject specialists so just reflect the population of those with the required level of education in terms of their attitude to maths.
If you do not enjoy a subject, any subject, you cannot make it fun for those you teach. In the case of maths specifically its pretty bad: https://worrydream.com/refs/Lockhart_2002_-_A_Mathematician%...
My daughter hated maths when I took her out of school at the age of nine. A few years later she was very good at it and enjoying maths and STEM subjects. When she went to a sixth form college[1] she liked it well enough to pick it as one of her A levels[2].
Another response to math that makes me sad: "I must be too stupid to understand this," "my brain is too small for this," etc. Different people say it for different reasons, but it's almost always in response to a hand-wavey explanation that doesn't makes sense to anyone not already in the know. Math is so much more about humility and skepticism than it is prodigy.
This is tricky, because, in fact, hard math having an intelligence floor is one of the nastier realities of the human condition. Anyone who is even quite intelligent but has really pursued the rigorous stuff, unless they are in fact a prodigy, eventually realizes they have an abstraction ceiling (and this term is a common one thrown around in people studying mathematics, because intelligence denial is so obviously false when you do hit your abstraction ceiling).
Most people are correct that they lack the intelligence / mind for a lot of hard math (even epsilon-delta proofs are enough to eliminate the majority of the population, no matter how good a teacher you are, and these are just basic undergrad calc).
And yeah, sure, people have different kinds of intelligence and such, but there is still a g-factor, and people of low intelligence almost universally can't do hard math, whereas most people who can do e.g. advanced undergrad math can generally do almost all other advanced undergrad fields reasonably well. The world isn't fair here.
> Anyone who is even quite intelligent but has really pursued the rigorous stuff, unless they are in fact a prodigy, eventually realizes they have an abstraction ceiling
Eh. I'm a math PhD who fled academia because it was too much for me. But I have never encountered this term "abstraction ceiling" nor did I succumb to it. I simply ran out of motivation to pursue higher math, especially when following through on learning and research became more and more labor. (It was always labor; but it was a labor I used to love.) I am far from a prodigy.
> even epsilon-delta proofs are enough to eliminate the majority of the population, no matter how good a teacher you are
Disagree. It's a notoriously hard subject to teach, and with all the demands placed on e-d in so little time in your average curriculum, it doesn't require appeals to IQ to explain its infamy. With enough motivation and practice, the quantifier alternation is comprehensible to any sound mind. What your average mind (and student) lacks is exposure to formalism, abstraction, and how these things tie in with what they are familiar with, which is symbolic manipulation. With the exception of geometric proofs (another educational bugbear), they have little context for what formalism is or why it matters.
> Eh. I'm a math PhD who fled academia because it was too much for me. But I have never encountered this term "abstraction ceiling" nor did I succumb to it.
This sounds a lot like you may have in fact succumbed to your abstraction ceiling, because in practice, the ceiling manifests as not as it being impossible for you to learn something, but that it would take you years and inordinate effort to master what you notice others mastering easily in just a fraction of the time. You may have not heard the exact term (comes from Douglas Hofstadter), and you may be talking about just the academic busywork, but I find it hard to believe you never encountered discussions about this kind of stuff. I would also politely suggest that unless you are Terry Tao posting under some kind of alt, you most certainly do have an abstraction ceiling (or your own mathematical limits) too.
> It's a notoriously hard subject to teach, and with all the demands placed on e-d in so little time in your average curriculum, it doesn't require appeals to IQ to explain its infamy. With enough motivation and practice, the quantifier alternation is comprehensible to any sound mind
The latter statement is obviously false, but regardless, intelligence explains some of the difficulty, and much other difficulties far more parsimoniously than "everyone could just learn any math if they just tried hard enough and had good enough teachers". E-d is merely an obvious and generally familiar example, and nothing I said really relies on this very specific aspect of maths, obviously. We also shouldn't pretend your (almost certainly false) view of math and intelligence isn't also often harmful to struggling students in its own way.
> Since then I've had the chance, in the world of mathematics that bid me welcome, to meet quite a number of people, both among my "elders" and among young people in my general age group, who were much more brilliant, much more "gifted" than I was. I admired the facility with which they picked up, as if at play, new ideas, juggling them as if familiar with them from the cradle - while for myself I felt clumsy. even oafish, wandering painfully up a arduous track, like a dumb ox faced with an amorphous mountain of things that I had to learn ( so I was assured), things I felt incapable of understanding the essentials or following through to the end.
(Alexander Grothendieck, Recoltes et Semailles)
Amazing that he managed to keep going after hitting his abstract ceiling in graduate school.
You clearly don't understand the meaning of the term. Grothendieck was almost certainly wrong about his gifts here, and even if not, your mathematical ability and output isn't fully explained by your ability ceiling.
Honestly, the pushback on this post is utterly baffling. Clearly the human mind has limits on what it can comprehend and the rate at which it can learn difficult things. Clearly these limits differ among individuals and are related to intelligence broadly.
Huge proportions of the population struggle to ever even grasp simple fractions, and not for a lack of effort from them or society. Fourth-year undergraduate mathematics is another beast entirely. Pretending the world is otherwise is pure fantasy and also plainly harmful, to the world and people that are unfairly pushed beyond their capabilities.
> Grothendieck was almost certainly wrong about his gifts here, and even if not, your mathematical ability and output isn't fully explained by your ability ceiling
lol, see, it's unfalsifiable. No true abstraction ceiling.