The literal meaning of ramen is a noodle dish served in broth. The kabbalistic interpretation of ramen is the divine act of creating intelligence. It’s almost too overt:
ra-men - Ra is the Ancient Egyptian father of all creation, and his relationship to men is that he created them. He created all living things, and the special thing about humans is intelligence. So ramen refers to the creation of intelligence. It is no coincidence1 that in English the phrase use your noodle refers to thinking. Ramen even looks like a brain, for Ra’s sake - the similarity has achieved meme status in Japan.
The definitive movie about ramen is Tampopo. It is a phenomenal movie for many reasons, one of which is that it’s a bit of a Rorschach test - you can view it as a romantic comedy, as a tragedy, as a Western, as an episode of a home renovation show, etc. It’s kind of similar to life - people can view their own life as a tragedy, a comedy, a horror… Which makes an unexamined life reality TV, I guess.
But back to Tampopo - I watch it as a heist movie. The MacGuffin (object of desire) is the perfect bowl of ramen. The metaphor the movie uses for the MacGuffin is a ladder. Tampopo says that she’s found her ladder - it’s making ramen, and that she’s lucky, because some people never find theirs. As an aside, one of my best friends really likes quoting the “Chaos is a ladder” quote from Game of Thrones when he gets drunk. Most people understand this phrase as “chaos creates opportunities for unscrupulous people”, but my friend’s demonstrated life preference is that chaos is his ladder. He actively climbs it to create more and more chaos. Interesting guy.
But back to Tampopo, again, for real this time - a large portion of the movie is devoted to assembling a motley crew of people who will go on the ramen heist ranging from a homeless gourmet sensei (formerly an OB-GYN) to a private chauffeur of an eccentric millionaire, to a brawly contractor. There is a plan, there are flaws in that plan that are overcome in real-time, and then there is the score. All of the staples of the heist movie are there.
There is A LOT that happens in Tampopo apart from the heist, see, for example, this delightful scene that’s completely unconnected to the rest of the movie plot-wise:
But the majority of the movie focuses on stealing the secrets of the perfect bowl of ramen.
So that makes Tampopo a perfect metaphor for life for me. I view life as a heist, the object of which is to steal the mysteries of the Universe, the first of which being the creation of intelligence, which, as we already know, is the meaning of ramen. Today is the eve of the Lunar New Year. And when the proprietor of Gatosano Ramen invited me to attend a Lunar New Year party tonight, I knew I had no choice but to go. Gato sano means healthy cat, ramen means the creation of intelligence, 2023 is the year of the Cat, ergo, by attending, I am going to celebrate a healthy year for creating artificial intelligence.
As I was writing this post, a random kitten rolled up on me, climbed into my lap, and started chewing my shirt. I miss the days when omens had subtlety, but what can you do, the Internet has ruined our attention span.
The term technological singularity means that at some point we (meaning humans specifically) will create superhuman intelligence, and it’s impossible for us to predict what will happen after that. The term was popularized2 by Vernor Vinge, one of the all-time greatest sci-fi writers, in his 1993 essay The Coming Technological Singularity: How to Survive in the Post-Human Era. I HIGHLY encourage you to read the whole thing, it is one of the most concise and lucid discussions of what superhuman intelligence actually means. I will throw a couple of quotes here, but again, read the whole damn thing, it’s short, prescient and extremely well-written.
From the human point of view this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control. Developments that before were thought might only happen in "a million years" (if ever) will likely happen in the next century.
But as time passes, we should see more symptoms. The dilemma felt by science fiction writers will be perceived in other creative endeavors. (I have heard thoughtful comic book writers worry about how to have spectacular effects when everything visible can be produced by the technically commonplace.) We will see automation replacing higher and higher level jobs. We have tools right now (symbolic math programs, cad/cam) that release us from most low-level drudgery. Or put another way: The work that is truly productive is the domain of a steadily smaller and more elite fraction of humanity. In the coming of the Singularity, we are seeing the predictions of _true_ technological unemployment finally come true.
And what of the arrival of the Singularity itself? What can be said of its actual appearance? Since it involves an intellectual runaway, it will probably occur faster than any technical revolution seen so far. The precipitating event will likely be unexpected -- perhaps even to the researchers involved. ("But all our previous models were catatonic! We were just tweaking some parameters....") If networking is widespread enough (into ubiquitous embedded systems), it may seem as if our artifacts as a whole had suddenly wakened.
And what happens a month or two (or a day or two) after that? I have only analogies to point to: The rise of humankind. We will be in the Post-Human era.
All of the above was written in 1993 and has aged REMARKABLY well. The middle passage is playing out with Generative AI in front of our eyes right now. So when does Vinge predict the technological singularity will happen? 30 years from 1993 or… Oops.
Granted, Vinge provides an interval. He says that he’d be surprised if the singularity arrived before 2005 or after 2030. With impeccable foresight, however, he makes allowances for the true computational power of neurons not being fully understood, which might push the timeline further out. As I’ve discussed at length, I believe that people severy underestimate individual neurons, so even if we don’t hit the singularity before 2030, I’d give him a lot of points for predictive power.
All of the above is just a meandering introduction to the very commonplace notion that the beginning of the year is a good time to take stock of things. The things in question naturally are our trials and tribulations in our heist of a technological singularity.
I have listed three possible paths to superhuman intelligence: BCIs, AI, and genetic approaches (engineering, embryo selection, etc.). Vinge lists four, the fourth one being the Internet “waking up” and achieving consciousness. I do not take this possibility too seriously, so I will not discuss it here.
Genetics are mostly out of the race. While we know that we can boost intelligence using an array of tools, the iteration cycle is SO DAMN SLOW. It takes 9 months to make a human, it takes another 18 years for them to become an adult. To make meaningful changes in human intelligence through genetics, we would need decades, which the other approaches likely won’t give us. We don’t have any tools at all that can change the genetic code of adults to increase their intelligence, nor are we close to having any.
I have written about BCIs before, but my overall take is that it is mostly a backup approach. The BCIs iteration cycle is not quite as slow as the genetics one, but it’s still measured in years. It takes years of animal testing of a new approach to even begin any testing in humans. And we are still many (my guess is 5-6) orders of magnitude (in terms of the number of neurons connected) off from technology that would be truly transformative to intelligence/the human experience, aka hivemind-enabling tech. Let’s say each generation improves the state of the art by one order of magnitude and takes 10 years from inception to maturity. That would give us a timeline of 50-60 years. Granted, there may be some overlap between the iterations, so let's cut that time in half. 25-30 years still seems too slow to get to superhuman compared to how quickly AI is developing. Metaculus is currently predicting the arrival of AGI to happen in 2039. Granted, betting on a singularity is a bit silly because what the hell are you going to collect at that point but Metaculus is still better than any individual prediction. Summing up, the only scenario in which I see BCIs winning the race to superintelligence is if we hit serious roadblocks in AI. Which is quite possible! See more on that below.
Finally, we can get to what this article is, which is State of AI, early 2023 edition. This is a big-picture article, so I won’t focus on things like DALL-E vs Stable Diffusion or GPT-3 vs Claude, but rather on where we are on the road to superhuman intelligence as we enter the new year.
There are two major currents happening in AI. The first one is in Perception, which includes computer vision, Large Language Models, generative art3, etc. Given a large dataset, we are getting REALLY close to a human-level understanding of that dataset. The dataset might be visual, textual, auditory, etc. We are mostly limited by getting enough data in many cases. Anyway, it’s easy enough to imagine that we are on the precipice of having AI models that are human-level accountants, diagnosticians, lawyers, artists, software engineers, etc.
But what about superhuman-level? That’s A LOT trickier. The way these models get to human-level capabilities is by reading everything humans have written on a subject. But where will you get superhuman training data? Unfortunately, we lack a corpus of things written by superhumans. For example, we might get AI lawyer models that are superhuman in the sense that they produce legal briefs faster and cheaper by many orders of magnitude than humans (as soon as 2023) but, at present, there is no obvious path for us to make AI lawyers superhuman in the sense that they will argue better than ANY human lawyer. I guess near-perfect recall of every single court case in recorded history will help, but I doubt it will lead to such a leap that will make ALL human lawyers irrelevant. Just most of them.
Don’t get me wrong, if we get to the point of commoditized human-level intelligence, it will still be transformative, but it’s not quite a technological singularity, it’s just the approach to it.
To sum up this bit, I expect 2023 to be the year where Perception models lead to human-level performance at a fraction of the cost/time across many different industries. I expect a gradual slowing down of the improvement in performance in base models (I expect a smaller jump from GPT-3 to GPT-4 than the one from GPT-2 to GPT-3) as we approach the limits of our training data (both in terms of the amount of human-level data that we have and the complete lack of superhuman-level training data).
The second current in AI involves Action. The technical term for this current is Reinforcement Learning, and the gist of it is that if we can clearly define a goal (win at go, fold a protein, make coffee) and we can make an environment where an AI agent can rapidly train in achieving that goal, we can get superhuman results on that goal. For now, they have mostly been limited to virtual-space activities, like games or protein folding, but we are rapidly moving into the physical world.
We can also make agents with many superhuman capabilities. For example, we could (should?) make an agent that can fold proteins better than you AND beat you in chess. My guess is we are a few (5-10) years away from having robots that can perform many tasks in the physical world at a human level using only the spoken word as instructions, including things that they haven’t done before, but that share some similarities with tasks they’ve been trained on.
We are closer than that to having AI agents do very specific tasks at a superhuman level. The exact timing depends on the task, as some tasks are easier to model than others, but the gist of it is there is a bit of a tradeoff between specificity and performance in these agents.
Overall, if I were to guess, I’d guess that much like 2020-2022 have been banner years for Perception-related AI, 2023-2025 will be banner years for Action-related AI. A really promising development is that robots can learn from other robots with different architectures, so I would expect the field to be able to accelerate quite rapidly.
Where we are lacking in our journey to superhuman intelligence is independent agency. We can make perception models that don’t want anything, that just respond when we ask them things. We can make RL agents that want to be good at a specific task. But we cannot yet make agents that just exist in the world, have some desires, but are also able to create their own higher-level desires.
Humans start life with some genetically preprogrammed desires - food, sleep, sex, snake aversion, etc. But there are also desires that arise from the human experience, like painting or trying to discover new planets. You can make an argument that those are just intermediate goals for genetically preprogrammed desires like social acceptance, but the end result is that left to their own devices, humans will engage in an enormous variety of different activities. They will make weasel coffee, juggle, play the piano, make movies about ramen, write blog posts, just sit there and think about recursion, etc. etc. etc.
We are not good at making agents with such diverse emergent behaviors (which is what we think about when we hear the word agency). Even in virtual environments, while we make agents that will engage in some exploration and experimentation to achieve a certain goal, we haven’t made agents that ever got fundamentally creative. A great example is that we can make an agent that can learn how to mine diamonds in Minecraft, but we can’t make an agent that without being explicitly told to builds a CPU in Minecraft.
An important reason for that lack of creativity is that our agents only ever see one goal. Even agents who can play both Pong and Minecraft, only ever experience one goal at a time. When they play Pong, they want to win at Pong, when they play Minecraft, they want to get those diamonds.
Creativity is a product of both curiosity AND a combination of different desires. To make agents that can invent new disciplines altogether, we need to work on agents that have different desires at the same time, and have mechanisms to balance them and alternate between behaviors that achieve them. In nerd terms, we need to think about agents as dynamical systems. While there are a few people who think about the human brain as a dynamical system, notably Chris Eliasmith in his book How to Build a Brain, to my knowledge, none of the big AI shops think about agents in the terms of dynamical systems.
Is such creativity/agency necessary for superhuman intelligence? Can’t we just make a general superhuman intelligence with a single desire (or with no desires)? I’m not sure, I don’t think anybody knows the answer to that question. But if we started to run into the limits of existing approaches, dynamical systems is where I would turn to for ideas. I don’t know if any moves will be made in this direction in 2023, but I will be watching it carefully.
Because, well, you know.
Though, like most good things in life (you know - computers, mutually assured destruction, cellular automata) it was originally coined by John von Neumann.
You might make the case that the word generative implies going beyond mere perception, but it should be clear from the following passages what I mean through contrast.