Ten advances in mathematics and theoretical computer science
openai.com621 points by milkshakes 3 days ago
621 points by milkshakes 3 days ago
Replace philosophers for mathematicians and Douglas Adams was spot on again.
Whilst current models can't 'intuit' and come up with conjectures, they can certainly disprove some of them very quickly through the kind of grind that humans can't do. I suppose there really are some mathematicians out there today, whose last few years of study, have just been up-ended by this.
--
"Yes we are," insisted Majikthise. "We are quite definitely here as representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons, and we want this machine off, and we want it off now!"
"What's the problem?" said Lunkwill.
"I'll tell you what the problem is mate," said Majikthise, "demarcation, that's the problem!"
"We demand," yelled Vroomfondel, "that demarcation may or may not be the problem!"
"You just let the machines get on with the adding up," warned Majikthise, "and we'll take care of the eternal verities thank you very much. You want to check your legal position you do mate. Under law the Quest for Ultimate Truth is quite clearly the inalienable prerogative of your working thinkers. Any bloody machine goes and actually finds it and we're straight out of a job aren't we? I mean what's the use of our sitting up half the night arguing that there may or may not be a God if this machine only goes and gives us his bleeding phone number the next morning?"
From a mathematician who was intimately familiar with some of these problems [0]
>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.
They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.
[0] https://nitter.poast.org/henryquantum/status/208362369543662...
I think it might be like waveform collapse, but very high dimensional.
> I think it might be like waveform collapse, but very high dimensional.
The waveform collapse is natively "very high dimensional", not sure why the "but" part of the sentence belongs here.
Can you expand on this?
Not OP, but my interpretation is this.
Quantum waveform collapse has been proposed as explanation for consciousness, allowing to explain how an entity can have free will and yet obey rigid physical laws: https://en.wikipedia.org/wiki/Consciousness_causes_collapse
I think OP is suggesting a similar thing happened in the LLM, implying it gained consciousness despite following a well-defined compute process.
No I just meant WFC as an algorithm for solving problems.
https://github.com/mxgmn/WaveFunctionCollapse
It's one of the coolest things I've ever seen.
I guess I've been pronouncing it wrong though. It's wave function collapse, not waveform.
I haven't studied it properly yet but it kind of looks like how sudokus work. You have a bunch of plausible options, and the various possible worlds overlap with each other but not completely and then you eliminate the things that are not possible until the actual possibility remains.
I think thinking works similarly, and I wouldn't be surprised if artificial thinking also worked similarly.
>Whilst current models can't 'intuit
That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.
Some of them...
The two places were seeing lots of movement are:
* Updates to lower/upper bounds. In many cases, these kinds of problems are the deep-math equivalent of calculating more digits of pi. Yes, if you throw time at it you'll break the record, but it may not be terribly worthwhile.
* Finding counter examples which disprove conjectures. This is really useful, and helps offset some positivity bias on the human side, often bringing together known tools from distant silos.
If you read the list of ten results, almost all fall into one of these buckets.
It is unfair to dismiss contributions to decades old open problems as equivalent to calculating more digits of pi. It missed the mark by a lot—as does the two bucket simplifaction.
Five (maybe six?) of the results are improvements on bounds. These kinds of problems tend to have some initial advances, and then stall out as the complexity of the bound skyrockets... until some grad student is bored enough to push the boundary. The big-O complexity of matrix multiplication is a good example of how this works: yeah, it's a useful problem, but the solutions are galactic algorithms, and increasingly convoluted.
As someone with a PhD in combinatorics, I believe that I'm qualified to say that, yes, there are problems as useless as calculating more digits of pi.
Your inverted logic does not hold. The fact that such useless problems for bounds exist does not mean that improving bounds is useless. 9 fields medals in the last twenty years, including the one to Terrence Tao, were for improvements on bounds. 3 of the 4 medals in 2022 were for bounds; 2 of these medals were in combinatorics.
I stated that boring bounds improvement problems exist, not that all bounds problems are boring... Sigh.
I agree boring problems exist; bounds may have a fare share of them. None of the bounds problems in this set are even close to this category; many of them are closer to the type of contributions that in the past got recognized by special awards. Your initial replies were misleading.
As someone with a PhD in combinatorics, you're aware that it takes only one counterexample to invalidate a conjecture.
There's nowhere else to move the goalposts. You've already stashed them in the far corner of the parking garage down the street from the stadium. If you go any farther you'll leave the school grounds entirely.
"It's just brute-forcing the search space."
It can move to any place within the search space but it can't move outside of it and it can't move in between the 'pixels'. Human thought can, as human thought has created the search space.
it could also be that they try every possible approach that has been proposed by humans. it seems that was the case for the non sofic group example. humans are not able to do the same at that scale. it's unfortunate that we don't know what's happening behind the hood with these models, and that's a huge danger also for the rest of us without access to them.
> "matrices"
Saying that AI is "matrices" is like saying human cognition is "neurons." Maybe true at some level, but it's a low-level implementation detail. The important part of a language model is the function that maps tokens to contextual embeddings. You could compute this function using analog computing, biological neurons, or any other substrate.
It isn't an implementation for neurons, unless you believe in a designing god.
Matrices are an implementation detail in reconstructing the surface of human knowledge. It's a complex surface, but it's a regurgitation.
What's the argument here? It's not about the implementation method, it is about the behavior that emerges from it. You could calculate the next token by hand on paper if you had enough time.
It's incredible how the “Chinese Room” argument, as well as its counterarguments, is still incredibly pertinent despite being now more than 40 years old. Scientific American published many articles on this topic in the Eighties; they seem as fresh as ever.
> Whilst current models can't 'intuit' and come up with conjectures
I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.
> they can certainly disprove some of them very quickly through the kind of grind that humans can't do
Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.
Ahh, but you missed the continuation, where they get to the heart of the matter: money.
"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"
DT: Might I make an observation at this point?
MT: You keep out of this metal nose.
VF: We demand that that machine not be allowed to think about this problem!
DT: If I might make an observation…
MT: We’ll go on strike!
VF: That’s right. You’ll have a national philosopher’s strike on your hands.
DT: Who will that inconvenience?
MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!
DT: [Booming] If I might make an observation …
“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”
Fook glanced impatiently at his watch.
“How long?” he said.
“Seven and a half million years,” said Deep Thought.
Lunkwill and Fook blinked at each other.
“Seven and a half million years!” they cried in chorus.
“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”
The two philosophers gaped at him.
“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”
“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”
So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”
> Whilst current models can't 'intuit' and come up with conjectures
People keep saying this. Why?
Surely the AI can complete the prompt “Generate new research questions based on these observations”?
When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.
I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
I have news for you. All humans do is also filling gaps with combinations of known facts and results - in new ways. "Everything is a Remix" is a good watch on youtube that explains this. Picasso might look like he has an invented personal style, but his style is a combination of different little details he took from others and mixed in a new way. Mozart the same. No music artist could ever create music in a vacuum. Everyone, for every art and science, the same. I know many are trying to cling to the last hope of human specialness, that "thing" that AI can never get to.
It's a convex hull of information that is reflective and spans outside of itself and combines in a new way, when you shine two known rays of light together from the inside.
Now it gets better. AI can be orders of magnitude more creative than any human could ever hope for, because his convex hull of information is orders of magnitude larger, and the possibilities for new combinations are equally larger.
> "Everything is a Remix" is a good watch on youtube that explains this
Not completely. Novelty used to be a major thing, when Humans did it. Another important criteria used to be if that new thing makes sense at all. Here the language model has a problem, as it doesn't have the means to evaluate this criteria.
Neural nets can extrapolate past their training data, and there is no reason to think LLMs don’t inherit this capability.
The extent to which they are able to do this is the more interesting question!
The extrapolation can also be a learned skill, especially in math. How many papers took result X, extended it to Y using known building blocks, and applied to Z.
By the way, convex hull permits extrapolating past the training data. LLM won't invent a new word that could not be defined by a sequence of known words. Just if it's meaningless and fully random/hallucinated, the new knowledge won't work with other known information blocks (breaks convexity).
Is that actually true though? I think it is an analogy, and as an analogy it seems quite risky because “convex hull” and “linear combination” are technical terms that might give the recipient the impression that it is a technical argument.
I think this is only "statistically" true in the sense that training is based on facts and not non-facts (except maybe with the ingestion of flat-earthers literature ;-). The existence of hallucinations in a bare transformer shows that the convex hull is not about information but about text, so the limit may more be "possible linear combinations of text", which allows for much extrapolation and counterfactuals. True creativity may be one reinforcement learning mid-training goal away that rewards novelty over correctness.
All arguments like this boil down to semantics at a certain point, but yes large language models can “intuit” because they can generalize between examples. The issue then becomes how you pack new examples into context.
Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.
Define insanely complex and deep in a way that isn't illiterate hand waving.
Most humans are dumber than a box of rocks. Here in Seattle we had one of many light rail-related fuckups where they had to replace part of the line with buses. People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
Chatgpt was smarter than the average person a while ago
> People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
This does not demonstrate a lack of intelligence. It demonstrates laziness and a lack of interest in spreading apart. Or just lack of consideration (or even malice) on the part of those at the back of the wad.
> Chatgpt was smarter than the average person a while ago
This is an absurd claim that fundamentally misunderstands what it means to be "smart". Reasoning that would get you to this conclusion would equally well apply to Google's search engine over a decade ago.
I’m not talking about the actions we take or how we might perform at certain tasks, I’m talking about how our brains actually work. My point is that we have no idea how I’m able to imagine an apple and see it in my mind’s eye. It’s basically biological magic to us at this point.
There are processes at work there that we don’t even have the language to describe.
Not only that, but we do it with a processor that is basically required to operate in a narrow temperature band below 40C, using a mere 86 billion neurons (although the equivalence with either machine-learning "neurons" or LLM parameters is not at all clear) operating on a few dozen watts; and with this we operate many other systems besides language processing. It's not clear that our reasoning process requires language, either.
(86 billion is the number ChatGPT, ironically enough, has given me a couple of times. I remember hearing for a long time that it was estimated to be somewhere in the ballpark of 100 billion. This is not my field of study.)
I recently read a blogpost from a human neuroscience student who came across this question (https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie...), and she looked up the study behind this number (https://ora.ox.ac.uk/objects/uuid:1f559b3b-97fd-48c2-b2ac-29...), and it seems that the actual current state of knowledge is that the number is somewhere in the range 60-100 billion, with probably some of that variation being biological and some coming from uncertainty in the measurement techniques, hard to tell, because all the data comes from nine brains.
For now, caring about a topic (or having somebody you trust care for you) still gets you better information than asking an LLM.
LLMs traverse an assembled surface of human knowledge.
You can't find things on a map that aren't there, but maybe you can draw a route nobody used before.
Because people have internalized an inaccurate model of LLMs as "stochastic parrots" that was incorrect at the time of formulation and is also significantly outdated
Well imho, it's a bit of a fundamental problem for a certain aspect of the meaning of "to intuit".
Since we're quoting Douglas Adams in this thread, I'll mention something I posted a while back, with his writings as example. After Douglas Adams passed away, somebody was tasked to "finish" The Hitchhiker's Guide to the Galaxy :
> And Another Thing... is the sixth and final novel in The Hitchhiker's Guide to the Galaxy series. Written by Eoin Colfer with the blessing of Douglas Adams' widow Jane Belson
Being a rather big fan, I immediately bought and read this novel and I have to say that Eoin Colfer did a really really great job, nailing the tone, humour and writing style of Douglas Adams.
IMVHO, he did about as good as anyone could reasonably expect someone to do, when given this task. It was big shoes to fill, and I was impressed.
But it just also wasn't good enough, in a weird way that I found hard to put my finger on at first.
The thing is that Colfer was doing the tone of voice, even came up with somewhat new jokes perfectly in the style of, etc etc. And for the sake of argument let's say he was able to get "arbitrarily close".
But there was always one thing he couldn't do: Actually make something new happen, make a new kind of joke, do a real plot twist, a big reveal, stuff like that. Because then it would deviate from Douglas Adams' work too much.
However, if Douglas Adams was still alive, this limitation would not apply to him: he could make a new kind of joke, do a plot twist, big reveal, and it would become canon.
This the best "good faith" argument I can present for how LLMs lack "intuition", in some sense. Now "intuition" is not a very exactly defined term, but I'm arguing that the thing I'm describing here, is at least a part of intuition, that an LLM fundamentally can't reach (until they start getting their own volition, which I would prefer they didn't).
To address your question:
> Surely the AI can complete the prompt “Generate new research questions based on these observations”?
Yes I imagine it could do that very well. But it would still need a human to decide if the research questions are "relevant" or "within scope" of what the human wants (a.k.a. their volition). Without that filter, the research would just bloom out exponentially, with more and more questions nobody was asking.
And yes, up to some point that "blooming" behaviour is a useful aspect of research, the exploratory aspect/phase, but at some point you need to get back to the "synthesis" aspect/phase, to distill all the explorations back to "stuff that matters". And just like Douglas Adams vs Eoin Colfer, only the human who wants to know something, can decide to widen or change the domain of that synthesis, but if the LLM were to decide this (outside of exploratory phase), it would actually be considered the wrong answer.
And this is not at all to say you can't do research with LLMs, obviously you can. But this is just a thing they can't do, on a real philosophical level. I'm also not saying you can't work around this limitation, you probably can, I'm just saying it exists.
> People keep saying this. Why?
For the same reason that you can't draw a 15 of Diamonds from a regular card deck.
People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results.
The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? How about experiment driven domains like biology?
We will get much better at manipulation and better at people “writing” things to justify their own feelings.
What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)
Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.
Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)
My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
Not new about LLMs. Targeted ads / big data is this.
Another degree of capability, yes. But we have been trending here for a long time.
I'm surprised no one has mentioned Cambridge Analytica.
What about them?
Cambridge Analytica gathered data to build targeted profiles and used these profiles for political advertising without informed consent: https://en.wikipedia.org/wiki/Facebook%E2%80%93Cambridge_Ana...
Yes, but there is little evidence this had a meaningful effect on votes.
Don't think that it matters if it was effective.
There's a reason attempted murder is a crime even if it was unsuccessful
It does matter though. If you want to murder someone by hitting them with a plushie, you're not going to get charged with attempted murder because it's not possible that that would ever work. There must be justification that the choice will have the intended effect.
We should not be gung-ho to give the government more power to regulate speech.