The Most Valuable Problem in AI

Throughout the brief but exciting history of computing, you could generally only throw a limited amount of compute at a problem. You had a problem, you had a program, the program ran, used some compute, and you got the result. Now, this wasn’t true for all problems - there were some optimization programs that you could run for any amount of steps, sure, but generally, that’s how things worked.

With Machine Learning this changed - somewhat. Now you can use a neural net to throw large and arbitrary amounts of compute at most problems - provided you shape the problem into something a neural net can work with. But in practice, for almost any problem there comes a point where it doesn’t make sense to keep training the neural net and humans need to step in and change something.

Now, we are on the precipice of a new (and almost certainly the last) era in compute. We are going to be able to throw arbitrary amounts of compute at ANY problem. You can already kind of see how it works if you play around with any of the AI Agents, here’s a screenshot of AgentGPT:

It breaks down a task, any task, into subtasks and then uses the available tools to complete the task. It more or less has its own OODA loop. It executes this loop for a while and then just stops mid-execution when it runs out of the compute you allocate for the task.

Now, right now, there is a shortage of GPUs and some other boring market dynamics in play so we can’t really throw truly ARBITRARY amounts of compute at problems as consumers - there are API quotas, GPU quotas, all kinds of stuff. But the beauty of capitalism is that when we want a lot of something, a lot of that shall be built, so the situation won’t stay the same forever. The endgame looks like this:

1. There is a hard physical limit on how efficiently we can convert energy into computation called Landauer’s principle. We are currently racing towards it and in the best case will reach it by 2080 but more likely, we will just slow down at improving on how close we are to it sometime before then.

2. The price of energy is not decreasing. It’s going up in nominal terms but roughly in line with inflation. Now, this might change if we get fusion or something, but the base case is that the price of energy in real terms is roughly flat.

   

3. That means that barring some unforeseen advances in energy production or our understanding of physics, we are going to run into a pretty straightforward limit of dollars per unit of compute (FLOPS/$ or the like).

4. So, we’re going to live in a world where we can throw arbitrary amounts of money at ANY intellectual problem.

This isn’t a novel idea - that’s approximately how the human world works in Permutation City by Greg Egan (or at the beginning of it, anyway). But we are rapidly approaching that world, the world where we tell AIs:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5.

Now, most household tasks are too trivial to warrant a budget, they’ll be paid for either by ads, a cheap monthly subscription bundle, or even through your own electricity bill if the task is simple enough to complete locally. But for any serious intellectual/creative tasks - the tasks that will remain valuable, there will likely always be a compute budget set.

So in a way, computing history will have had 3 eras:

1. You pay someone to write a program to accomplish a particular task.

2. You pay someone to translate your particular task into something an LLM can understand (this is where we are today).

3. You give an AI agent a task and a budget and that’s it.

Generally stated, the problem we need to solve to get to era #3 is this:

Given any unbounded¹ problem X and compute budget Y, what’s the most efficient way to spend Y on compute to achieve the best result on X?

This… Is a hard problem. But my assertion here is that whoever solves this problem the best will build the most valuable company of the century.

The current generation AI Agents just kind of punt on this problem and say “just run this loop until you run out of compute” but this by no means guarantees the best result. In fact, the AI agents of today frequently don’t produce any result - just some intermediate steps of a “thinking” process. What needs to happen for sure is when you ask an AI agent:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5.

It answers with a basic report and when you ask it:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5,000.

It answers with a better report. Now, naturally, if you want to dig into its process, you can do so, but ultimately people want results, not process.

Many other things also need to exist to solve this problem! Estimators of whether a problem is better solved by code or a neural net given a certain compute budget. A database of all the tools and APIs the agent can use or subcontract to in the world. A way to pick the best result out of all the intermediate results, etc.

Is it possible that the solutions for this problem will be domain-specific? That one company will be the best at spend 100$ to make a video game and another will be the best at spend 100$ to write a novel? For sure, that’s possible. But if I were to guess, I’d guess that a large part of the solution will generalize, and besides, if you’re shooting at the most ambitious target - you pick the most ambitious goal, which is the general case.

1. Unbounded problems in this case just means that there’s no perfect answer that you can achieve with a known compute budget. “2+2=?” is a bounded problem, “make me the best possible movie about superintelligent rats” is an unbounded problem.