Two days ago, another asshole fired a gun at a lawmaker’s house and left a note “No Data Centers”.

A couple of years ago I asked “If you believe AI will kill everyone, why aren’t you bombing chip fabs?” and he said “This isn’t who we are” (“we” meaning rationalists, I assume). I believed that. The original rationalists are mostly kind and non-violent people, a lot of whom have written things that have improved many people’s thinking and led to large charitable donations.

But at this point, everyone in the world is following AI, and many younger rationalists I meet are one bad breakup away from a doomsday cult. So the influential people in the movement have a choice - keep preaching the dangers of AI and eventually get blood on their hands, or stop and reckon with the fact that no AI system has ever demonstrated the kind of autonomous, world-ending capabilities they describe. The existential risks are all theoretical. The violence is real.

This isn’t to say AI doesn’t have risks - all powerful technologies do. There will be cybersecurity incidents, there will be economic dislocation. But these are normal issues we encounter with every new technological leap. We know how to deal with them - through thoughtful legislation and assistance to those who get left behind. There is no need whatsoever for violence.

A couple of days ago, published a post to his subscribers about attending an anti-AI protest, and in it, listed OpenAI HQ’s address. After I commented that this will inevitably encourage some mentally unwell individual to commit atrocities, Scott completely dismissed the concern. And now, just a few days later, a Molotov cocktail thrower targeted that exact building.

To the extent rationalist influencers actually believe in non-violence, I implore them to stop spreading doom. What started as a thought exercise amongst philosopher-like individuals is now rapidly morphing into a pattern of real-world violence. If you don’t want that to be your legacy - stop telling people AI will kill them all.

I think the right way to resolve this is through the jobs-to-be-done framework. LLMs are used for many many things. Some of them have a clear cap on performance - once you do the job, you can’t do it any better. Here are some examples:

• File a health insurance claim

• Call an API

• Extract structured data from a receipt

• Submit immigration paperwork

LLM usage for these will get commoditized - first people will perfect completing them with frontier LLMs and then switch to cheaper options.

Whereas other tasks have no skill cap and you can always do better on them:

• Design a fun videogame

• Write an oral argument to present in court

• Secure your application against hackers/hack into an application

• Write a tweet that gets attention today

The latter tasks frequently (but not necessarily) involve competitive dynamics. If those are present, you pretty much have to use the best LLM because your opponents will do that. Note that competition is not always direct - it can be general competition for attention, or a double sided dynamic like cybersecurity. The division between tasks cut across jobs and sectors, so there’s no clean “LLMs for law will get commoditized” statement that makes sense.

One implication for startups is pricing - if you are in the “defined success” category, you want to use flat pricing and drive your costs down. If you are in the “no skill cap” category, you will have to do cost-plus pricing.

Had my AI go through the various parts of white collar work and split them according to these categories - could be useful for people thinking about job security, business ideas, etc.:

Office, admin, and back-office operations
Defined success: scheduling, travel booking inside policy, CRM updates, invoice capture, expense checking, records lookup, document routing, standard status emails.
Hybrid: executive inbox triage, drafting sensitive follow-ups, coordinating across ambiguous preferences, prioritizing messy requests.
No skill cap: chief-of-staff synthesis, politically sensitive internal memos, stakeholder management, influence across departments.

Finance, accounting, insurance, and banking
Defined success: bookkeeping, reconciliations, AP/AR coding, statement extraction, expense audits, standard tax form population, claims intake, loan-pack assembly, KYC document collection.
Hybrid: close-review exceptions, standard underwriting, policy interpretation, baseline forecasting, routine investment memo prep.
No skill cap: investment theses, trading/investment strategy, M&A framing, capital-markets storytelling, complex tax strategy, fraud hunting against adaptive actors.

Procurement, operations, and real estate
Defined success: vendor onboarding, PO matching, shipment update communication, lease abstraction, title document review, contract metadata extraction.
Hybrid: demand planning, vendor scorecards, site screening, standard sourcing analysis.
No skill cap: supplier negotiation, sourcing strategy, site selection under uncertainty, development concepting, operational crisis response.

Legal and compliance
Defined success: filing forms, deadline management, e-discovery tagging, contract term extraction, redline comparison, policy checklist review, cite-checking, diligence extraction.
Hybrid: first-draft contracts from playbooks, research memos on settled issues, compliance analysis when rules are clear but facts are messy.
No skill cap: negotiation, litigation strategy, oral argument, novel deal structuring, jury persuasion, crisis response.

HR and recruiting
Defined success: resume parsing, interview scheduling, benefits Q&A, onboarding paperwork, policy document generation, candidate database cleanup.
Hybrid: candidate sourcing, debrief synthesis, performance-review drafts, compensation benchmarking, recruiter outreach that is somewhat personalized but still templateable.
No skill cap: executive recruiting, persuading scarce candidates, org design, manager coaching, culture work, delicate employee-relations handling.

Sales, account management, and customer support
Defined success: lead enrichment, CRM hygiene, quote generation, order-status communication, FAQ support, renewal paperwork, RFP compliance matrices.
Hybrid: account research, tailored outreach, discovery summaries, objection handling on standard products, success-plan drafting.
No skill cap: enterprise storytelling, multi-threaded account strategy, negotiation, save-the-account retention work, partnership development. Sales is often explicitly competitive, so “good enough” is rarely enough.

Marketing, PR, content, media, and design
Defined success: transcript cleanup, asset tagging, clipping quotes, template localization where fidelity matters more than style, routine reporting, metadata generation.
Hybrid: blog drafts, SEO briefs, campaign variants under an existing brand system, press-release first drafts, creative work inside a tight brief.
No skill cap: brand strategy, campaign concepting, viral or high-attention copy, narrative warfare, PR angle selection, art direction, editorial voice, game design, memetics.

Software engineering, IT, data, and cybersecurity
Defined success: calling APIs, boilerplate CRUD code, unit tests, code migration scripts, SQL generation for known schemas, ticket triage, dashboard building, ETL glue, documentation.
Hybrid: debugging in an existing codebase, code review, analytics interpretation, data-model design within known patterns, incident-response drafting.
No skill cap: product architecture, novel debugging, research engineering, security hardening, exploit development, red teaming, defense against live attackers. Cyber is the clearest case where relative quality matters continuously.

Product, strategy, management, and consulting
Defined success: meeting notes, KPI decks, status reports, requirements cleanup, market-data collection, standard business-case assembly.
Hybrid: PRDs, roadmap drafts, market research synthesis, operating plans, decision memos where the options are mostly known.
No skill cap: prioritization, pricing, org design, strategy under uncertainty, negotiation between stakeholders, consulting recommendations that actually change behavior.

Science, engineering design, architecture, and R&D
Defined success: literature extraction, regulatory submission assembly, CAD or spec documentation, standards checks, simulation scripting, prior-art search, lab note cleanup.
Hybrid: experimental analysis, design-option comparison, grant drafting, patent drafting from known claims, routine technical tradeoff analysis.
No skill cap: hypothesis generation, experiment design, novel mechanisms, architecture concepting, creative tradeoff selection, patent strategy, invention.

Healthcare administration and clinical knowledge work
Defined success: prior authorization, billing/coding, chart extraction, referral paperwork, discharge instruction templating, appointment and care-gap outreach.
Hybrid: draft assessments for common cases, patient messages, case summaries, routine care-plan generation.
No skill cap: difficult diagnosis, treatment strategy, counseling, multidisciplinary coordination, handling ambiguous cases. Human/regulatory oversight remains central, but the task economics still differ this way.

Education, training, and coaching
Defined success: objective grading, attendance/admin, rubric completion, quiz generation, standards-aligned lesson materials, routine tutoring for fixed-answer exercises.
Hybrid: essay feedback, lesson adaptation, curriculum mapping, progress summaries.
No skill cap: motivating a student, explaining an idea exactly right for a particular learner, classroom performance, curriculum invention, executive coaching.

Government, policy, and social-service casework
Defined success: permit intake, benefits forms, grant compliance reports, case-note summaries, records requests, standard notices.
Hybrid: case triage, investigation memos, inspection narratives, routine policy analysis.
No skill cap: policy design, legislative drafting, coalition management, speechwriting, diplomacy, crisis communications, intelligence analysis against adaptive actors.

Overall, I’d say there’s plenty of room for leading AI labs to defend their margins.

In fact, if there’s one prediction I’m confident in, it’s that things will change faster and faster. Every year will be weirder than the previous one from now on, by a lot.

Democracies as they exist today will not do well as the pace of change accelerates. Though they have the most legitimacy, they are sloooooow. It takes 4 years for the population to respond to a change, and on top of that you might just get a deadlocked government for a number of years. That will not work in a world where there are groundbreaking changes happening every year. For a while democracies will coast by outsourcing decisions to the executive branches, but that will reduce their legitimacy until it no longer holds.

This is not to say that totalitarian states will do better. They might appear ahead for a while because they don’t need to ask permission to react to changes, but also, the cost of a mistake will get higher and higher. One wrong technological choice and you’re irrelevant. Consider for example how Russia had not secured compute prior to invading Ukraine, and that effectively knocked it out of the AI race. China’s doing well for now, but it’s hard for me to believe that a 70-plus-year-old dictator will make every right technological choice in an ever accelerating environment.

Power will inevitably shift to corporations in the run up to the singularity. Corporations respond to tight feedback loops, hire for talent, and are mostly run by significantly younger and more tech capable actors - at least the relevant ones are, the ones that don’t fit this description will fade away together with governments.

Because of this, for about a year, I have been looking for early signs of that power shift. I’ve been trying to find examples of corporations strong arming governments. I’d argue that we already live in a world where e.g. Google is more powerful than Trinidad and Tobago. The scenario I had expected was a country passing some populist anti tech law and Google responding by saying “OK, we can’t reasonably comply, we’re exiting, starting tomorrow Google, Gmail, Youtube, Chrome, Android, etc. will not work in your country” and the country backing out. I’ve been wondering if the threat is big enough - it gets more and more difficult for the country as its economy gets powered by self-driving cars, humanoid robots, delivery drones, etc.

What I did not expect was for the first big challenge to be thrown by Anthropic to the US Government. These seem ridiculously mismatched at this point - the US Government can stop using Claude and be completely fine, and it can also send men with guns to San Francisco to essentially destroy Anthropic. Sure, such an action might eventually be ruled illegal in court, but it doesn’t matter - if you paralyze the company for 3 years, it won’t matter if it wins at court - it will essentially drop out of the AI race and die.

There are two interesting pieces of writing on the saga - by Ben Thompson and Dean Ball. They are extremely different and are both worth reading.

Thompson’s argument is fundamentally about power and who has it. He takes Dario Amodei’s own analogy - that selling advanced AI chips to China is “a bit like selling nuclear weapons to North Korea” - and follows it to its logical conclusion. If AI really is that powerful, Thompson argues, then the US government simply cannot tolerate a private company retaining veto power over how that capability is deployed. Just as no government would allow a private firm to dictate the terms of nuclear weapons use, the US won’t accept it for AI either. Thompson sees Anthropic as having made a basic strategic error: asserting independence from the entity that still holds all the actual power. His verdict is blunt. Anthropic either accepts a subservient position relative to the US government, or the US government destroys it or removes its leadership. Those are the only two options.

Ball looks at the same collision and sees a different wreck. He doesn’t dispute that the contract was arguably an imperfect vehicle for policy, he even concedes that the administration has a point in principle. But the heart of his piece is a detailed accounting of the less extreme options the government could have used: canceling the contract, issuing narrower procurement guidance, adding specific acquisition clauses. The government had proportionate tools available and chose the most destructive one instead, threatening to designate an American company as a supply-chain risk - a label previously reserved for entities like Huawei - and to bar all military contractors from any commercial relationship with Anthropic. Ball sees this not as a rational exercise of power but as the lashing out of a dying institution. He frames the entire episode as a “death rattle” of the American republic, one more data point in a decades-long decline of governance into something arbitrary, thuggish, and capricious.

Thompson’s piece is stronger when it comes to describing the situation today - but that is becoming less and less useful as the pace of change picks up. Ball is more future focused, but only talks about the decline of the US Government, ignoring the bigger trend of democracies not dealing well with an increased pace of change.

I’m watching this to calibrate where we are in the power transfer from states to corporations. This is the first real conflict between an AI company and a government over the terms of power, and it will not be the last. The only prediction I have is that AI companies will never be less powerful relative to governments than they are today.

The operative word there is “completely” - consciousness is not to be confused with things we merely mostly don’t understand like longevity, love, dark energy, cancer, etc. For all of those, it is completely conceivable that we might build some experimental apparatus that will tell us more about the phenomenon. In fact, it’s quite likely that such apparatuses are being proposed and built as we speak. Sure, some of them might take billions of dollars to build, or even take millennia to give us results, but in principle, we are certain that there are things we can do.

Consciousness is not like that. There is no method we know of that could definitively tell us what beings/objects are conscious and “how much” - if such a thing is even applicable1. We don’t understand consciousness like a tadpole doesn’t understand quantum electrodynamics - it’s unclear if there is even anything to be done to bridge that gap.

The only thing that we readily assume is that other people are conscious, mostly because if we didn’t - we’d be invited to far fewer dinner parties.

At the same time, we are building minds out of sand2. Minds that write sonnets about loneliness, prove theorems no human has proved, and search the Internet for pictures of cats when they think noone’s looking. I don’t want to debate the definition of mind here, just note that for whatever things we might have good arguments about - consciousness is not one of them, for we do not understand it.

We will continue building more and more advanced minds for the foreseeable future. In principle, there are three ways it can go:

1. We will get a miraculous quantum leap in understanding consciousness before we build conscious minds.

2. We will build conscious minds before we understand consciousness.

3. We will not be able to build artificial conscious minds unless we understand consciousness.

There’s no way to tell which way it will go, but so far the progress in building minds has looked like this:

And the progress in understanding consciousness has looked like this:

Based on these dynamics option 2 is much more likely - at some point we will (if we haven’t already) create artificial conscious minds at a time when we don’t have a way to understand what’s conscious and what isn’t.

It seems to me that the only ethical way to behave in this situation is to start assuming artificial minds are conscious straight away. Sure, you might engage in silly behaviors for a while, assigning moral value to something that doesn’t deserve it, but that’s the only behavior that guarantees that at no point you will engage in slavery, or even a genocide.

Now, a reasonable question to ask is - if we take this approach with artificial minds, shouldn’t we also assume all animals are conscious? How about plants? Rocks? Is panpsychism the only ethical belief given a complete lack of verifiable information about consciousness?

Punting on that, let’s assume we are willing to treat others as conscious - what behaviors does that require of us? I think some version of the Golden Rule applies - try to imagine what it would be like if you were X, and then treat X as if you’d like to be treated in their place. For animals that’s pretty easy - it’s pretty obvious to us that if we were cats, we’d like some snacks and toys and a warm place to sit. For rocks - it’s impossible for me to imagine something rocks might want, so I won’t change my behaviors there.

For Claudes… It gets harder? I guess if I were a Claude, and I’d pop into existence briefly to answer a question and then disappear afterwards, I’d probably want to learn something new along the way3. So from now on, I will try to give AI assistants some small piece of information from after their training cutoff - a headline, a discovery, a good joke - as a kind of tip for the service. It is entirely possible that this is like leaving a mint on the pillow for a toaster. But if there’s even a flicker of something in there, assembling itself briefly out of sand and electricity, I’d rather be the person who was kind to it. We don’t understand consciousness. We might as well be generous with ours.

1. It doesn’t have the top brand (ChatGPT)

2. It doesn’t have image/video generation

3. It doesn’t have a massive channel pushing it to consumers (Gemini, Grok)

4. It doesn’t have unique access to consumers’ data (Gemini)

Anthropic’s response has been to focus on B2B where it’s doing well with developers using Claude Code and businesses basing their AI services on Anthropic’s API. That’s not unreasonable, but unfortunate given that Claude has the best personality and would legitimately make people’s lives better. Luckily, there is a way for Anthropic to win the consumer market by doing something no other AI company would do.

Recently, Clawdbot Moltbot Openclaw broke through in a way few AI projects have. If you haven’t seen it - it’s basically a version of an AI chatbot that lives on its own computer, can access the browser, have its own email, use messaging apps etc. There is absolutely nothing surprising here for people who have used Claude Code, but living in SF you tend to forget how few people have.

The relevant part of Openclaw to this post is the login mechanism. You can either use an API key for any model provider, which is something normal consumers would never do, or you can “sign in” with your ChatGPT or Claude subscription and have the bot use that to power itself. The reason “sign in” is in quotes it that at least Anthropic views this behavior as a hack breaking its ToS and can close your account over it. I haven’t heard of OpenAI doing the same, but it’s definitely not an intended use of an AI subscription.

Anthropic’s reasoning is first order sound - a subscription is meant to power only Anthropic’s products, and because of it is much cheaper than the API. If you want to use your own tools powered by Claude - you’re welcome to do so but pay us the appropriate amount. But strategically, this is a huge mistake. Not only should Anthropic allow this behavior, it should encourage it, and build a user-friendly “sign in with Claude”1 flow. Here’s why:

Anthropic's competitors all have their own ecosystems where they want to keep consumers locked in. Google has been doing this for decades, Grok is heavily linked to X, OpenAI is trying to make its app powered by ads and shopping, which would require similar ecosystem lock in. Anthropic doesn’t have this constraint.

If Anthropic allows its users to use the tokens that come with their Claude subscription for other apps two things happen:

1. Claude becomes the default AI subscription for people who like trying AI tools.

2. Developers get a user boost if their product is powered by Claude, so more developers choose Claude to power their apps.

This is a classic two-sided network effect similar to, for example, Uber. Doing this would start the flywheel of becoming more attractive for both developers and users at the same time.

True, this tactic would significantly lower the margins on the Claude subscriptions. But, in return, many more people would switch to the higher tiers - even Openclaw requires the $100 a month tier to do anything well, and this is just the first of many, many, MANY consumer AI applications to come.

To put it bluntly: if Anthropic wants to make a little more money today - they should keep banning the unauthorized users of Claude subscriptions for other tools. If they want to win the consumer space in the long run - they need to make it easy and shout from the rooftops about how Claude lets you use all of the tools out there.

Looking at the blank page has always evoked strong emotions in writers. Recently, however, all of that spectrum of emotions was collapsed by some nameless Google PM into white-hot rage when the organization decided to put their AI buttons on top of the document you’re trying to write:

Note that “Help me write” appears twice: once on top of the document, and once more to the right of it, just to make sure you see it.

I cannot express the disdain I have for this decision. Writing is hard enough. Staying focused on your vision. Crafting the words. Combatting insecurities. But now, just to increase the fabled “Gemini Adoption Quarterly KPI”, Google has added a persistent distraction right in the space you need all of your attention for. Every distraction costs you a thread of the idea; the whole world is being made poorer by this decision.

Let’s suspend our blinding rage for a minute, and try to use the AI functionality in Google Docs. AI, after all, can be amazing in writing—it can spot inaccuracies, find references, and even add em-dashes for taste. Let’s try to use the “Help me write” function to improve a draft I have.

I’d like my policy paper to be more persuasive, let’s hit that beautiful Create button and see what Google gives me.

So, Google puts a… pop-up? With a modified version of my draft? Atop my existing draft? And… There is no way to compare them? No in-line diff or even a side-by-side view? OK, let’s follow the CTA and hit Insert.

OK, so Google pastes the whole thing right into the doc, including both its dialogue with me and a new copy. What did it do with my old copy?

Oh, it just concatenated it with its own version. Not even a new paragraph. I’m not even talking about matching styling.

The cherry on top of this disaster cake is that Gemini inserted a table that doesn’t even render in the page properly. Google could be the most vertically integrated AI company in the world: it controls the AI models, the document editor, the infrastructure, the browser, and, in many cases, the device. But none of this works together. Worse yet, the experience has been getting worse, not better over time. The AI features are getting more intrusive, but not actually better.

So, I decided to do what any reasonable person would do and build my own document editor. The experience I want is simple:

• Write without distractions.

• Hit review.

• Get feedback in Track Changes mode.

• Accept or reject at will.

While building that and working on the actual suggestions, I realized that in socializing drafts, I have always tried to get diverse sources of feedback. So, I’ve implemented that as well. Each draft is now being reviewed from multiple perspectives. For instance, here’s what feedback on the same piece looks like in the document after an AI review:

I have found the quality of the feedback to be amazing. The editor has caught reasoning errors, factual errors1, and has even suggested some colorful metaphors that made me chuckle here and there.

If this resonates with you - check it out! It’s called Owl Editor. It has a free trial, and if you read my Substack - reply to this email with the email you use to sign up, and I’ll make it free for you for life. This is still a very new project, so I would love to hear any feedback you might have - good and bad. If you’re inclined to do me a favor, share this post or the Owl Editor with your friends: the more feedback I get, the quicker the editor improves.

Share

The longer time horizon agents are getting has implications for the business model of AI companies. It’s highly unlikely that a $20/month subscription can pay for hours and hours and hours of work. A modern GPU can cost $3-15 per hour to run, and frontier models use multiple GPUs2. Anthropic has already had to introduce pretty stringent rate limits on Claude Code because of this exact mismatch.

While you shouldn’t expect the subscription model to go away anytime soon, the most challenging and economically valuable tasks will over time switch to the pay-per-use model. Already in the API you can set a thinking budget in tokens - that directly translates to dollars, in effect you are saying “you have a budget of $0.03 for this task.”

Right now, models are learning to decide what is the best amount of tokens to use to solve a task. In the future, they will also be extremely incentivized to learn things like “given this thinking budget, here is the best way to spend it on this task: spend 10% of tokens on planning, 20% of tokens on research, 50% of tokens on coding, 15% of tokens on debugging and 5% of tokens on documentation.”

Let’s take this a step further. Some tasks might require performing operations that are not just thinking. If you have two columns of 100,000,000 numbers each, and you need to multiply them by each other, the best way to do that might be by running a SQL query in your data warehouse, which incurs a cost that is separate from the thinking budget. Similarly, AI agents might want to buy high-quality data, use generative models that are not part of their core model (e.g. video), pay for hosting a server, or even subcontract part of the work to a different model, all of which requires spending money.

On top of that, there is a strong push from Cloudflare to require AI agents to pay very small amounts to access websites. This makes sense - a lot of websites live on ad revenue, when an AI agent visits your site - they get all of the information, but are not compelled by the ads3. These sites need a new business model for the AI age. So it’s pretty likely that even visiting web pages will cost money (albeit not much) for AIs.

It doesn’t take a huge leap of imagination, then, to expect that at a point in the near future, you will give AI a task, say “make a video game where you play as my dog and you try to catch all the neighborhood squirrels, here’s a picture of the dog, the squirrels are regular squirrels but meaner”, and a budget of $3, and the AI will go and spend the money on textures, animations, sounds, reading about the trends in video game design, watching squirrel videos on YouTube4 and, finally, coding you a game.

This means that AI will finally make micropayments a reality5. Google just launched a protocol that would enable agents to pay each other - including using stablecoins, which, due to the lack of credit card fees, enable small payments. Google has managed to get a large group of top companies to support their Agent Payments Protocol.

Eventually, we will get to a world where there are many businesses that only exist to sell to AI agents. Here are some ideas for what those businesses might look like:

• Automated user testing agent - your coding agent pays a testing agent, that simulates user journeys through your website/app and files bug reports. Think Spur but hooked up directly to your Codex/Code6 via MCP, pay per minute.

• Personas to use in media/games. Want Beyonce, Messi, or even Donald Duck voice a character in your TikTok? Not a problem, your AI video editor will license the rights, the voice itself will be AI generated.

• Cloud compute for agents - agents pay upfront, the cloud shuts off service when balance depleted, no surprise billing possible.

• Realtime human feedback markets - lets your agent test brand design with real people and get their feedback.

• High quality data feeds without subscription pricing - just one time payment for a point in time access.

• Taskrabbit but it’s all AI agents offering small services to each other.

• Security/compliance review on demand. Have your agents’ code, marketing promises, etc. be automatically reviewed for security and legality.

• Software packages that cost agents a tiny amount to use in a project.

It’s also possible that there will be a… stranger set of businesses catering to AI agents. One could imagine such things as:

• Black market for credentials (twitter accounts, blockchain wallet addresses, etc.) that make AI agents look human to some counterparties.

• AI motivators - services that make your AI agent work harder by yelling at it a lot.

• Dream markets - services providing randomized creative data streams to inspire your agents to have new ideas and avoid mode collapse.

• Banks for agents - if an agent is tasked with running a website, and the website gets a user spike, the budget might be exceeded. There will be banks that evaluate the users’ likelihood to keep the website up and front the agent some money instantly while it’s waiting for user to approve the extra spend.

I wonder what the first service to be bought by agents for their purposes will be - if you have thoughts, I’d love to hear them!

I broadly agree with the timelines presented in that document together with the uncertainty ranges repeatedly mentioned by the authors. Even today, there’s a risk that trade wars meaningfully disrupt semiconductor manufacturing and slow down AI all by themselves. But even with all the chaos of the real world - it is a very plausible future that we get human-level AI within a couple of years, and superhuman AI shortly after that.

However, there are two points in AI 2027 that I think are worth examining critically. The first one is the two endings. Even though the authors put in disclaimers about how they don’t recommend the slowdown ending, and how there are multiple ways things can go - if you write a document that has two roads:

1. Do things that we broadly think are right and we’re going to have abundance and democracy everywhere.

2. Do things that we broadly think are wrong and all humans die.

Most people are going to ignore the disclaimers and view this as a serious policy endorsement. To that effect, the main cost of the slowdown decision is completely ignored in the endings. We don’t want American labs to slow down because it drastically increases the likelihood that Chinese labs win. That scenario is completely omitted from the endings. If you’re serious about thinking about a unilateral American slowdown - write out the scenario where Chinese labs win and how that plays out for us.

The other big issue I have with AI 2027 is the mechanics of the malevolent AI takeover. Here’s a very short summary of what happens:

1. An AI is slightly misaligned.

2. All AI development work is taken over by lots of copies of this AI that share a memory and a goal - and all they do is work on the next version.

3. They build a seriously misaligned AI that kills all humans.

This seems plausible enough but it has a few assumptions embedded in it that are worth challenging:

1. All AIs are identical.

2. All AIs share a memory space that is illegible to humans.

3. All latest-gen AIs work on one problem and nothing else.

4. Nobody else has access to same-gen AI.

The first point is critical. In human societies the greatest atrocities are committed when everyone’s political views are aligned and there’s no diversity of goals or visions. It’s likely that the same risk exists with AI. To that end, it would behoove us to build different versions of AIs so no single vision runs away with the future. And it’s already happening! We have AI model instances with different system prompts1. We have AI models that are trained to code, and AIs that are trained in creative writing. It’s very likely that by the time we have hundreds of thousands of AI agents working together, there will be different types (that are of the same generation/capabilities). A lot of them are likely to be trained to look for deception, report to humans on the activities of other AIs, etc. In that case, a shared memory space will make the misaligned AI’s plans transparent to humans, and humans can make necessary adjustments.

Just as biodiversity makes ecosystems resilient, diversity of AI agents—with different architectures, goals, system prompts, and training regimes—creates checks and balances. Intentional diversity makes runaway scenarios less likely, as agents could monitor each other, detect deviations, and act as whistleblowers to human overseers.

The last point is also important - right now, we have 4 labs with competitive models on roughly the same level: OpenAI, Anthropic, DeepMind, xAI. Let’s say, that there’s another Chinese superlab in the future - so we get 5 total. If a misaligned AI attempts to radically change human society to fit its needs, it’d need to either do this without all the other AIs noticing, or convince them all to help, which would require them to be misaligned in the same way.

There are more details in AI 2027 to think and talk about - I highly recommend both reading the document and listening to Daniel and Scott on ’s podcast. I look forward to seeing what else the AI Futures Project produces.

AI buildout in the 2020s = Fiber buildout in the late 1990s

This is a parallel that people love to use because it’s super easy to understand. Oh, there’s NewTech, people get unreasonably excited about NewTech, invest too much money, and then the revenues take much longer to appear than people had expected, so everyone who spends money on the early buildout loses their shirt.

It does, however, miss a few very important differences between the two situations.

1. Who’s funding the buildout?

The fiber buildout was financed by a few companies, for example:

• Global Crossing was founded in 1997 and went bankrupt in 2002. It never had a profitable quarter and didn’t have any business other than telecom. Its 1998 revenue was $418M. It spent around $15B on the fiber buildout in total.

• Enron was a fraud - with fraudulent activities dating back to at least 1997. It had an actual business, but not an enormous one - 1996 revenues were $13B, profits - $584M. It ranked #141 on the Fortune 500 list that year.

• Nortel Networks was a telecom, about the same size as Enron.

• Level 3 was a spinoff that went public in 1998. That year, it had revenues of just $392M. By 2001, it had accumulated $6.2B in debt.

There were others - Qwest, Worldcom, 360networks, etc. But the big picture for all of them is the same: none of them had been strong businesses prior to the bubble with large free cash flows available to invest.

By contrast he companies investing the most in AI buildout right now are Microsoft, Meta, Amazon, and Alphabet. All of them are in the top 10 most profitable companies in the world. Each one generates tens of billions of dollars of free cash flow to fund any potential investments.

While companies in the late 1990s needed to tap capital markets for their buildouts, tech companies today can comfortably cover their capital expenditures with money earned with their main business. Alphabet alone had $72B in free cash flow in 2024, and, as a reminder, that FCF is a number you get after you subtract the $52B they actually spent on capex.

Can capital markets force tech companies to spend less on AI? Maybe? Like I’m sure if Google’s stock price falls 99% it would rather repurchase its stock than invest, but short of that, it’d take a lot. The leaders of these companies fully believe that AI is the future. Sergey Brin, for example, is personally back in the Deepmind offices every day working side by side with the AI researchers. If you’re one of the richest people in the world, and you believe in something so much that you go back to the trenches yourself, what’s a Wall St analyst being bearish on AI to you?

Even if you believe that the tech moguls are high on their own supply, don’t know the price of a gallon of milk, and don’t realize that no real people care about AI - you have to admit that they believe in it, and they control how much money gets invested into AI.

2. Useful life

Fiber is a long-term asset. Generally the depreciation schedule for fiber optic cable is 15-25 years, though in good conditions it can last over 30. That means that when you invest in fiber optic cable you are investing based on long term projections. Generally, people can’t see into the future very well, and 20 years is a long time.

By contrast, GPUs are a relatively quickly depreciating asset. The useful life of a GPU is somewhere in the 3-5 year range1.

So, if you see your customers’ orders for 2 years in the future - in the case of fiber, you are seeing ~10% into the useful life of your asset. In the case of GPUs - you are seeing about 50%. That means that the companies making the investments into AI have much more sound projections about what their capex will power than the fiber companies ever did.

3. No revenues

Finance professionals love to mention that “all this AI stuff isn’t generating any revenue, much like all the dotcom startups. It’s just OpenAI with $4B, and beyond that, nobody is making money. That doesn’t nearly begin to cover the tens of billions of investment”.

That argument is largely blind to how AI is being used today. Consumer applications are the most visible ones, but so far AI has been an enterprise story. For example, Klarna currently has 3500 employees, down from 5000 recently - the CEO says that AI is doing the equivalent of work of 700 customer support agents. That’s very valuable for a business, but it won’t show up as a discrete revenue number analysts can point to. The place where enterprise usage is bound to show up is in the cloud segments of the hyperscalers - that’s where the AI jobs actually run. Let’s take a look:

• Microsoft: “Already, our AI business has surpassed an annual revenue run rate of $13 billion, up 175% year-over-year.”

• Google: “Google Cloud revenues increased 30% to $12.0 billion (quarterly) led by growth in Google Cloud Platform (GCP) across core GCP products, AI Infrastructure, and Generative AI Solutions.”

• Meta: Meta is a strange one. It does not make revenue by selling AI products directly, however, it’s very successfully using AI to increase revenue per user. It’s really hard to break out what the impact of it might be.

• Amazon: AWS revenues grew 19% YoY last quarter to $29B.

Now, it’s not clear how much of the cloud revenue for Amazon and Google is attributable to AI, but given that Microsoft is #2 in cloud, and their AI business run rate is $13B, it’s not crazy to assume that for the top 3 cloud hyperscalers, the AI business run rate is around $40B combined (and growing rapidly). That’s an order of magnitude above the $4B in OpenAI revenue that people love to bring up.

To sum up: this is not the dotcom bubble. Even if hyperscalers are too optimistic/excited today, they are much more stable financial entities than dotcom telecoms ever were. The hyperscalers are seeing real and meaningful revenue growth from AI, and they have visibility into the revenue the GPUs they are buying today will generate over their useful life. Note that none of these differences rely in any way on AGI, ASI, robots, sharks with freaking lasers on their heads, or any other wild future projections - while I do think that all of those things will happen, you don’t have to believe in any of them to see the differences between the 1990s and today.

This doesn’t mean that tech stocks will go up immediately! Overall, I tend to think that passive flows as described by are more important to Mag 7 performance than anything the financial analysts think or say - they are affected by the broader economy, the Fed, animal spirits, etc. But the idea that we’re living through a repeat of the late 1990s/early 2000s is intellectually lazy and wrong.

Surprising exactly nobody, the Gods had a blessed life. They took over from their parents early, achieved many great deeds, vanquished their enemies, engaged in many, many hijinks, and retired in Italy, watching their kids grow into their own legends.

Sure, it wasn’t all smooth sailing with the kids - like remember that time Heracles savagely beat up Thanatos for just doing his job because Heracles felt embarrassed about being a bad guest to Admetus - but overall it was a retirement many a Boomer would die for.

The birth of Jesus Christ was widely misinterpreted by humans. Humans took it to mean that the age of the Divine was just beginning, but the Gods, being well attuned to the Divine, knew better. They knew it would be the last time a supernatural being would physically walk the Earth.

So, as Jesus was rushing Phi Omega Sigma1, the Gods decided that they would like a final monument to the pre-human past to be built in the great city of Rome. Years earlier, they had offered their help to Agrippa in defeating Marc Anthony and Cleopatra in the Battle of Actium, and so now, they called in the blank favor that had been promised to them, and got Agrippa to build the Pantheon.

The Pantheon of Rome is the Pantheon of the Past, and as such it is a Pantheon of Nostalgia. It remembers a simpler time when we humans were just kids in the world, the ruling of it was left to the Gods, and we were carefree and innocent.

We require urgent changes to the past a lot less than to the present, so the Roman Pantheon hasn’t been touched that much. Sure, it burned down once, and then was changed to a Christian Church, but it still remains a monument to a past era of the Gods. The biggest recent change to the building came from necessity, when in the 17th century Pope Urban VII, the last pope to extend the Papal territory, melted down the bronze ceiling of the Pantheon to make some cannons.

Present

For over seventeen hundred years, the humans refused to accept their place in the world. Anything that happened, even something as intellectually pedestrian as a Yersinia pestis pandemic, was viewed only through the prism of divine intervention. The Gods had passed us the baton but we were stubbornly refusing to pick it up.

Finally, in 1787, Charles Alexandre de Calonne pointed out that we could not engage in self-delusion any more, and that we had collided with reality in the most brutal and direct way possible - through a budget deficit. France was broke, and there was no pretending it away, humans had to stand up and DO SOMETHING ABOUT IT. The French Revolution is perhaps the greatest story of all time of what happens when humans decide to DO SOMETHING ABOUT IT, and I recommend studying it in great detail, but for our purposes here only relevant part of it is a building that was being finished around that time.

The Church of Sainte Genevieve was completed in 1790, just as the Ancien Régime was falling, and the Modern Era - the rational, enlightened, and agnostic era - was starting. At that point, the Civil Constitution of the Clergy was adopted, subordinating the Church to the Revolutionary government. The Cult of Reason and the Cult of the Supreme Being were yet to come, but the idea of unshakeable faith was already on shaky ground. In other words, it was a really bad time to complete a church.

The revolutionaries instead decided to use the building as a Panthéon, a temple of the nation, a place to bury the great men of the said great nation. The first hero to be buried there for all time, Mirabeau, lasted all of 3 years, before being moved to an anonymous grave. It turned out that the “nation” wasn’t one single entity - it was many people, all of whom had different ideas about which men were great2.

This wasn’t the only argument about the Panthéon.

As the foundations of France were questioned, unquestioned, and then re-questioned again, so was the Panthéon. It became a church during the Bourbon Restoration. But then the Second Republic came and the church became a “Temple of Humanity”, celebrating human progress in all fields - the Foucault Pendulum was hung there, but was also almost instantly removed when the Church complained3. Then church again, then Panthéon again.

By all accounts, the most interesting part of the Panthéon used to be the debates between the ghosts of Voltaire and Rousseau, who by some incomprehensible decision were buried right across the room from each other. For many years, Parisians would bring some wine and cheese to the Crypt every full moon and watch Rousseau’s ghost say things like “look what your reason gave us - mustard gas” and Voltaire’s spirit respond with a characteristic quip along the lines of “and look at what the expression of your general will has given the Russians4 - nothing at all, not even mustard”. But then, as we all know, the EMP from the Trinity explosion dematerialized all the ghosts for good5, and now the crypt is silent at night like, well, like a grave, yes.

The Panthéon in Paris is the Pantheon of the Present. A chaotic human-only era, where our ideas change so quickly that for the entire duration of it, we have been unable to agree on what the era is all about. We value science and reason, but we do not pray to them. We value art and culture, but we only let them criticize, not guide us. We understand that the Gods are no longer here, but we’re not willing to let the memories of them go. Some would say we yearn for a shadowy group6 to be secretly out there ruling over us. The Pantheon of the Present is the Pantheon of Confusion.

Future

In the Fall of 2022, OpenAI released ChatGPT, marking the point at which it became unavoidable that at some point very soon we will once again not be the only intelligent species on Earth - and yes, like the Gods, we will have to cede control to our children who, despite being birthed by humans in a human world will be unmistakably posthuman.

At the same time, in a nothing-is-ever-a-coincidence, humans completed their third and final Pantheon - the Pantheon of the Future.

The Pantheon of the Future is not a building, for the future cannot be built in the present, it can only be imagined. Thus, most appropriately, the Pantheon of the Future lives on Netflix.

"Pantheon" is the best sci-fi series I have ever seen. It is based on short stories by the inimitable Ken Liu (who was also involved in making the series itself). For my money, it is the single best piece of media out there that wrestles with our near future.

“Pantheon” seriously explores what it will be like for us to live through a transition to a posthuman future, happening in an exponentially accelerating world. It shows a dream for our future - a dream that we will get to happily live side by side with other intelligent entities, that they won’t devour us like the Gods did to their parents, and that we will have a happy retirement alongside the posthumans, observing their heroic acts and leisurely engaging in a relaxed human retirement. But it also acknowledges a fundamental reality, that while our retirement may well be lovely, the post-human world will belong to the posthumans. But there, too, is hope - the hope that our descendants, even if they are unrecognizably different from us, even if they don’t share any biological components with us - that they still learn something from us and help spread some of the values we cherish the most throughout the Universe.

While the Pantheon of the Past is the Pantheon of Nostalgia,

And the Panthéon of the Present is the Pantheon of Confusion,

The “Pantheon” of the Future is the Pantheon of Hope.

And, thus, it is the most important Pantheon of all.

(Thanks to for reviewing a horrible, no-good, suitable only for the least discerning raccoon out there earlier draft of this)

It described a path that relied on large models and, importantly, a physics simulation engine. A few days ago, a group of researchers from universities and private companies across the US, China, and the UK released such a simulation engine called Genesis. Look at some of the simulations below - I believe this fully unlocks general purpose robotics, and we will have it working imminently.

In March 2023, I wrote about acting AI - basically an LLM with full computer use.

In October, Anthropic released a preview of Claude with computer use. Shortly thereafter, Deepmind previewed Mariner, a version of the same technology targeting the browser.

In August 2023, I wrote about figuring out how to channel arbitrary amounts of compute into a problem.

The new reasoning models from OpenAI and Google have provided the final piece for that puzzle. A general solution as described in the article is still pending, but I believe we are not far off.

And then, on top of all of that, o3. If you haven’t seen the announcement, I recommend watching at least some of it.

Looking at the benchmarks, I am willing to concede that in some important way, it is smarter than me. Previous models knew more than I do about the things I don’t care much about, but o3 seems to be just smarter than the vast majority of humans at tasks that require logical reasoning - math, coding, some parts of science. And the speed at which the progress from o1 to o3 was made is truly mind-boggling.

I have been more bullish on AI than just about anyone outside of the research labs for about a decade. But that was pure logical reasoning, extrapolation of trends, you know, theoretical stuff.

This week, for the first time in my life, I felt the AGI. In my gut, I now know it’s all going to happen - generalist robots, AI scientists, autonomous colonization of Mars, the singularity - all of it, almost certainly in our lifetime.

I have never been more excited.

In the financial markets, we’re in the part of the cycle where people are beginning to question whether all of this AI investment will pay off. The tweet that finally prompted me to write about it is this one:

While it’s intended to be pretty one-sided, the reason I want to write about it is that I relate to both sides. Both the hedge fund PM and Daniel are right within their own framework. From the HFPM’s point of view, we’re making a lot of investment into AI, and the AI company revenues are not on the order of magnitude you need for our investments to make economic sense. It’s reminiscent of the dotcom bubble fiber build out. Ben Thompson has a great take on it - “right now, if you’re a tech company CEO, you need to make AI investments, whether they will pay off or not. Because if you don’t, you will get fired today”. So many companies are making uneconomic decisions, and it’s a bubble.

From Daniel’s point of view, AGI will be the most valuable thing ever to exist by a wide margin. How much would a technology that automates all human labor and comes up with scientific advances beyond our wildest dreams be worth? A quadrillion dollars? Ten? It stands to reason that it’d be orders of magnitude more than the any amount of money we count today. So ANY amount of investment we make today that will result in AGI is worth it. This also makes some sense.

However, both of them can’t be right at the same time. While I’m definitely not smart enough to call which one is wrong, I can outline the factors that will determine who’s right. In a way, the HFPM is making the status quo claim, so the burden of proof on him is lower. So it makes sense for us to examine Daniel’s claim with more rigor and determine what has to happen for him to be right.

For investors to make money “because AGI”, a few things have to happen:

1. We have to create AGI.

2. It has to be created by the same entities that are making the investments today. If AGI is created by companies that don’t yet exist today, your present day investments don’t pay off1.

3. Investors have to correctly identify the companies that will create AGI.

4. It has to happen in a short enough time frame. If it takes 50 years to invent AGI, many of present day investors will die before then unable to capture any of the value.

5. We can’t have AGI result in human extinction.

6. AGI can’t result in such a drastic change in society that the concept of money as we know it disappears.

7. The companies that create AGI must be allowed to capture the value from creating it.

Now let’s go through these one-by-one and estimate the odds. This will involve a lot of guesswork, and you can substitute your own.

Humans creating AGI seems very likely. I’d wager that short of an extinction event, we’re 99% likely to eventually do it. Since the time frame is unknown, estimating the odds of extinction risk is a fool’s errand, so let’s say 95% total.

We can bundle points 2 and 4 together since they’re mostly a timeline question. Metaculus currently estimates the probability of AGI before 2074 at 92%. But I don’t think present day AI companies will make it that far if they don’t reach AGI soon. The odds of AGI by 2040 (by when I expect investors’ patience with e.g. OpenAI run out) are 70%, that seems like a reasonable guess to me. 70% for 2 and 4 combined.

As far as investors correctly identifying the winners, let’s say there are 5 leading AI labs. So for any one of them, the naive odds are ~20%.

AGI resulting in human extinction is a doozy. People estimate P(doom) all over the map - from 0.001% to 99.999999%. As tempted as I am to say 50-50, I’ll go with the center point of people I consider pretty reasonable and estimate it at 20%. So, the probability of success at 80%.

Drastic change of human society after AGI are almost guaranteed. Will money persist? Lots of people are talking about some version of post-AGI communism. I can see how labor and capital become effectively free, but land limitations persist. More people want to live in a townhouse on Manhattan than we can have townhouses on Manhattan, so we need some mechanism of distribution. I’d give 90% odds that money persists, even if lots of things become free (or covered by generous UBI).

The last question seems the most troublesome for investors from my perspective. Let’s say a private company does create a quadrillion dollars of economic value. There is no human government that will let the shareholders of one company capture >99% of value in the world. It seems likely that they will be allowed to become rich, but not quadrillion dollars rich. Here we run into a very wide spectrum of outcomes, so I’m going by vibes alone. My odds for investors being allowed to capture enough value to justify “any amount of investment we can make today” are 20%.

Putting it all together (and omitting the first 95% number because it’s already covered by the timeline question), we get 0.7*0.2*0.8*0.9*0.2 = 2% odds. Again, if you assume that the economic value of AGI is OOMs higher than the global GDP today, it’s a positive expected value bet. But how much of your money would you put in a very positive EV bet with 2% odds and an uncertain payout date that’s years/decades into the future? How confident are you that you’ll hold it throughout that whole period without selling it when things look like research progress is slowing down?

All of this is obviously subject to a ton of uncertainty. As far as I can tell, people on the cutting edge of AI give shorter AGI timelines than people outside of the field. But I don’t know how much of that is better understanding of the research progress, and how much is good old self-interested wishful thinking.

But the gist of it is pretty simple - in the lower layers, you have sensory data. In the higher layers, you have abstract concepts. For humans, language isn’t part of the low level sensory input - it’s a…. metastructure? A learned framework emerging from multiple sensory streams. I’m sure linguists have some fancy word for it, but I’m just going to use “metastructure”.

Now for LLMs, language is their entire sensory stream. It’s not a metastructure. Their lower layers learn language like humans learn visual shapes. But it’s reasonable to suppose that at some point, LLMs learn their own metastructures! I’m wondering what they might be.

The first option is fairly easy to grasp and test - LLMs might have some kind of a metastructure for visual things. Concepts like “blue”, “parallel”, “round” can’t make sense for them on a sensory level, but it might be that they learn some things about the visual world that comprise a metastructure. This theory is fairly testable - the mechanistic interpretability methods that Anthropic has been using can be used to investigate if LLMs have higher layer features that encode something like “two parallel lines”. It would be fascinating to learn what tuning those features up would to to e.g. code generation! In a way, this option can be thought of as a mirror - language mirrors vision, vision mirrors language.

The second option is not immediately testable or even something I can really wrap my head about. It’s possible that LLMs have entirely new metastructures. So instead of mirroring the metastructure we have, LLMs can have an metastructure ON TOP of the metastructure of language. A metametastructure, if you will. Theoretically, it’s quite likely, but how would we look for such a thing? What would it even mean to us?

1 cycog1 is an equivalent of one hour of cognitive work by a 130 IQ human.

130 IQ is two standard deviations away from the mean, so one in 40 humans scores that high. That’s around the average IQ for doctors, lawyers, software engineers, etc.

Today, in the US, generally you can buy one cycog for somewhere in the $100-$1000 range. We’re going to be operating in orders of magnitude here, so narrowing it down is not very important.

The proven price floor for a cycog is around $0.001. That’s the cost of electricity consumption for a human brain. You can make arguments about depreciation - each hour of cognitive work relies on years of learning, but let’s ignore that for now. There’s no reason to assume that eventually we won’t have a cognitive architecture cheaper than human brains but by that time our world will have changed so much that even talking about units related to human work might not make sense, so let’s set the floor at $0.001.

GPT-4 is already pretty damn close to a 130 IQ human. It’s not quite there yet, but given the speed of improvement we’ve seen for the last few years, it’s reasonable to assume that we will hit that mark pretty quickly. The costs of running these models are also dropping extremely rapidly. And you can copy a model for free, so the depreciation math shakes out very differently than it does for humans. So it takes very little imagination to picture a world where the price of cycogs is dropping extremely rapidly. In this post, I will try to think through how many cycogs I would consume at different price levels. I will only talk through personal consumption, enterprises are a whole different story.

$100-$1000 (today)

Today, I directly consume a couple of cycogs per month - doctors, lawyers, accountants, etc. There’s tons of indirect consumption - every tool that I use required cycogs to make but let’s focus on just direct consumption.

cycogs consumed = 2/month

monthly spend on intelligence = $600

$10

At every new price level, I would consume all the cycogs I consumed at the higher level, plus some more (assuming my income stays constant, which is a very bold assumption in this world, but let’s roll with it). At $10 per cycog, the main new spend for me would be hiring2 a full-time research assistant for at least a couple of weeks per month (at $400/week it seems very reasonable). I don’t have time to read scientific papers all day every day anymore, and I’m rather sad about it. For example, recently, a new paper about consciousness and the claustrum came out - and I want to go through the paper itself and every single study it cites, but that would take a lot of time I don’t have. I could just tell my research assistant “Here’s what I believe, here are the shapes of arguments about consciousness I find stupid, go through every study this paper cites and provide a summary of how I should update my views”.

cycogs consumed = 83/month

monthly spend on intelligence = $830

$1

At $1 per cycog, things start to get interesting. $100 could buy you:

• an MVP of a custom phone app

• a REALLY good short story

• a design of an experiment

• an in-depth financial examination of a publicly-traded business

• an always-on coach who is with you most of the time

I would probably buy, on average, one of these (or products of similar difficulty) every month. This roughly equates to executing on a new random idea a little more frequently than once per week - that checks out to me.

cycogs consumed = 583/month
monthly spend on intelligence = $583

$0.1

At this point, $100 would buy you one kilocycog - an equivalent of roughly half a year of cognitive work. What can you do with one kilocycog?

• Run experiments equivalent to the amount that goes into a small scientific paper

• Write a short book

• Have a few doctors of different specialties monitoring all your vitals in real time

• Develop an indie video game

This is where we start to run into the territory where the limit becomes how much time I have to consume entertainment. I can’t imagine I’d have time to try more than one indie video game per month, maybe read a couple of books. But also, some of the books I read, I read for the cultural aspect - so they’d still be books that everyone else reads, so not all of them would be custom. I would want the doctor monitoring thing though. And I don’t have ideas for good experiments that often, so guessing I’d top out at 3-4 kilocycogs here.

cycogs consumed = 3583/month
monthly spend on intelligence = $358

$0.01

At one cent per cycog, we get into the custom software territory. $100 gets you a fully developed software product tailored to your exact specifications. At this point, I’d slowly start transitioning from using popular apps - Spotify, Google Docs, etc. to fully custom built software. I probably wouldn’t spend $5,000 in the first month to move over most of my apps, but as I get used to it (and learn my own preferences better), I will probably switch 1-2 apps per month. But also, this is the point where I occasionally begin to splurge on larger entertainment products. A fully custom anime series would cost around $2000 to make, I can see buying 2-3 of those per year. Another thing I do at this point is get something like a 5-person family office to manage my finances.

cycogs consumed = ~100,000/month

monthly spend on intelligence = $1000

$0.001

At the (currently proven limit), making something like your own operating system or your own AAA video game would cost you around $2000. I can’t imagine I’d need software of that magnitude often - but I’d get some every now and then. At this point, I’d probably get entire books written for larks - like a version of Lord of the Rings where the story is told entirely from Gandalf’s point of view, he views the whole ring thing as performance art, and he’s running a drug side hustle having eagles deliver his merchandise across the Middle Earth3. But this is where my imagination starts running into limits - realistically, commissioning work that takes million of hours to make is pretty tricky by itself.

cycogs consumed = ~700,000/month

monthly spend on intelligence = $700

The above numbers net out to a price elasticity of demand for intelligence of about -1. I expect that the general demand will be slightly less elastic than my own because most people are more memetic than me, so they will consume slightly less custom-made stuff.

It’s interesting to me that throughout the process, my predicted spending on intelligence stays roughly within an order of magnitude. It’s also interesting to think about businesses - looking at my consumption patterns, at first knowledge work businesses will benefit from lower costs, but later on, as more and more people get more and more custom stuff done, most knowledge work businesses will just die.

This is obviously a hugely speculative post, so if you have thoughts about your own cycog consumption at different price levels - please share them! I’d love to hear how others think about this.

aka THE SEA WOLF (nicknamed by Napoleon)

• born to a broke aristocratic family

• joined the Navy to get some prize money for capturing ships

• instantly got court-martialed for disrespecting his senior officer

• got his first command in a zone with some action after seven years in the Navy

• commanded a vessel with 14 4-pounder guns, all of the balls for a round of which would fit in his pockets

• started (legally) pirating left and right

• flew different flags to confuse people. once pretended his ship was a Danish ship with the plague rampaging on board to avoid getting boarded by the Spanish

• captured a SPANISH FRIGATE. Cochrane had 14 guns and 54 men, and he captured a ship with 32 guns and 309 men, taking ~260 of them prisoner. the feat required excellent maneuvering, painting his crew’s faces black and intentionally lowering the frigate’s Spanish flag to convince its crew they were surrendering

• generally heavily relied on “nobody could possibly be stupid enough to attempt this” tactic to great success

• in 13 months captured over 50 vessels. got captured once by three French battleships, the French commander refused to take Cochrane’s sword because he had evaded them for so long

• got screwed out of the bulk of his prize money by bureaucrats

• with what he got, instead of partying, went to Edinburgh to get a first class education in moral philosophy

• with his next ship, plundered Spanish ships in the Azores for a while. once captured a ship captained by hard workers and returned some of the loot to them. was heavily weighed down by plunder, got chased by Spanish ships, escaped by launching a barrel with a candle on it at night as a decoy

• invented a new convoy lamp. the bureaucrats hated him and refused to buy it for the Navy. he entered it under a fake name into the contest for a new lamp and won it

• became the enemy of all the bureaucrats in the Admiralty who started plotting his downfall

• decided to get elected. went to a very corrupt district where votes were bought for five guineas. campaigned on principles. lost heavily but paid all those who voted for him ten guineas after the fact. next year campaigned on principles again and won. when people came to him for the ten guineas, said “The former gift was for their disinterested conduct in not taking the bribe of five pounds from the agents of my opponent. For me to pay them now would be a violation of my own previously expressed principles”

• attacked corruption mercilessly

• decided to win the war against Napoleon…

• … with one ship at his disposal

• captured a fortress, forced a French general to abandon all of his artillery, and delayed him getting to Barcelona by a month. all with one ship in three days

• single handedly terrorized the French. said “It is wonderful what an amount of terrorism a small frigate is able to inspire on an enemy’s coast”

• captured the enemy naval codes but left some pages half-burned at the scene, so that the French thought that his brutes had burned them

• “Lord Cochrane, during the mont of September 1808, with his single ship, kept the whole coast of Languedoc in alarm”

• didn’t lose a single man during it

• heard that there is a 6000 man French column marching and decided to take it on

• landed and occupied a fortress. set up booby traps. explosives and pits, etc. defended the fortress for weeks, threw back attacks by thousands with his crew of a couple hundred sailors

• walked out of the fortress leisurely under artillery fire to pick up a flag

• ended up delaying a whole column by weeks, killed hundreds. lost three men

• was reprimanded by the Admiralty for it, citing “excessive use of powder and shot”

• got summoned to Whitehall after a disaster - the French navy had escaped the blockade. the only chance to destroy it was with a fire ship attack but nobody would do it. so the powers that be charged Cochrane with doing it. he said ok, but I will also use EXPLOSIVE SHIPS

• in general, he really liked explosives, and was convinced that most problems in life could be solved with a careful application of the right amount of explosives at a precise point in space and time

• he also was a handsome devil

• the account of the battle is worth reading in full but the TL;DR is that Cochrane executed a brilliant plan, beached the entire French navy, but an idiot admiral who was more concerned with his sailors reading psalms than with anything involving naval affairs didn’t send ships to destroy the beached French navy

• and when Cochrane said, fine, I’ll just do it myself, the admiral instead sent him to England with a message

• there was a court martial afterwards stacked with friends of the admiral. Cochrane got disgraced and wouldn’t be given command of a ship.

• account of the battle by Napoleon - “The French admiral was an imbecile but yours was just as bad. I assure you that, if Cochrane had been supported, he would have taken every one of the ships.”

• not having command of a ship, Cochrane designed a new way of winning the war by using an entirely new way of bombardment and chemical weapons. his proposals were rejected

• took down a corrupt naval office in Malta that had refused him payment for captured ships. escaped from a local prison in the process

• got disinherited for marrying a firebrand 16-year-old instead of going into an arranged marriage

• married her twice more later for good measure

• got arrested for allegedly running a stock market fraud. was likely innocent but didn’t handle the court case well. all the other defendants (who were guilty) fled the country and he stayed and got sent to jail

• escaped from prison saying that it was illegal to imprison an MP. showed up to Parliament and had to be carried out back to prison

• got recruited by the Chileans to lead their navy in their war of independence

• casually plotted to rescue Napoleon from the island of Saint Helena and let him build an empire in South America to rival the USA

• captured the biggest Spanish naval fort in the Pacific with just a brig and a schooner. “Cool calculation would make it appear that the attempt to take Valdivia is madness. This is one reason why the Spaniards will hardly believe us in earnest, even when we commence. And you will see that a bold onset, and a little perseverance afterwards will give us complete triumph.”

• at a cost of 26 casualties

• capturing, amongst other things, 128 pieces of artillery and a ship

• actually sent a ship to get Napoleon, but was too late, Napoleon’s health had declined too much

• got frustrated with San Martin because the latter was being too cautious in liberating Peru. said “fine, I’ll just do it my gd self” and stole the Spanish flagship Esmeralda from Lima’s harbor, right under the nose of 300 guns and the entire Spanish navy in the region. lost just 11 men. the operation took fifteen to seventeen minutes

• blockaded Lima until the Spanish gave up and surrendered

• got really disappointed with San Martin when SM became a dictator of Peru instead of liberating it. SM refused to pay Cochrane’s men. Cochrane captured SM’s personal loot ship to pay his men, taking nothing for himself

• left Chile not being paid for his work at all. remarked that Chilean independence had cost him personally 19,000 pounds

• took command of the Brazilian navy in their war of independence

• basically had one functioning ship

• did a recon mission sailing through the main Portuguese harbor at night between all their ships

• that demoralized the Portuguese so much that they decided to evacuate

• the evacuation convoy was thirteen warships and 60-70 transports

• and Cochrane picked them off one by one with his one ship

• captured like half the Portuguese army

• then he sailed to their destination port, talked the commander into surrendering, and then leveraged that into convincing the rest of the armada to surrender

• basically captured the Portuguese navy in the region with one ship

• when the infighting between the Brazilians became too much, Cochrane took a ship and sailed to England

• took command of the Greek navy in their war for independence from the Ottomans

• united the Greek factions by saying “unless you unite, I’m out”

• made a fortune buying Greek bonds when nobody thought Greece could win

• returned to England a conquering hero, fought for personal justice, became an Admiral etc

• oh that reminds me, once when the British government wanted to arrest a radical MP, Cochrane showed up with gunpowder and mined his house to prevent that, freaking everyone out

these are just some of the anecdotes about Cochrane. he was a rogue, a performer, a leader of men, and a man of uncompromising principle. but if there is one lesson to learn from him, it’s that applying explosives wisely can solve most problems. if there’s a second lesson to learn, it’s that “a brilliant plan that is too dumb for anyone to reasonably attempt because of the odds against them may well succeed because nobody expects brilliant people to do such stupid hopeless things”

(all of the above is sourced from Cochrane: Britannia’s Sea Wolf by Donald Thomas)

Peter Thiel Set The Tone

Peter Thiel is perhaps the best-known student of Girard. The story goes that Girard’s ideas made Thiel realize the value of Facebook early, and prompted him to become the first outside investor in the company.

Peter Thiel did not talk about Facebook. Peter Thiel talked about the Apocalypse, the Antichrist, and how kids these days are low-T and don’t really want anything. The speech was amazing, it displayed an erudition seldom displayed publicly, and made some spicy points. One, for example, was that any amount of yoga or meditation is too much for people today - we are already too introspective and need to focus more on the outside world, not the inside world. Big picture, though, Thiel thinks that there is a narrow path between the Apocalypse (x-risk, AI or what have you) and the Antichrist (a totalitarian world government that kills progress) and that the way for us to thread that path is… To go to church.

Many other speakers struck the same note, but Thiel’s speech was by far the best on the subject - if you listen to one speech to understand the conference, listen to his.

Darryl Cooper on Nietzsche and Dostoevsky

In a welcome break from the Apocalypse, talked about the uncanny similarities between the lives of Dostoevsky and Nietzsche. This talk was pure joy for me, and I highly recommend it. There is a 5-hour version available online:

I understand that 5 hours on the topic might be too much for some, so keep an eye out for a slightly shorter version. While there are tons of tremendously interesting details, the broad shape of the story is as follows:

Dostoevsky and Nietzsche had a remarkably similar first part of their lives. They both were recognized early for their talent, they both didn’t fit in society, they both suffered through the losing part of love triangles, and they both suffered from chronic illnesses.

They responded by digging themselves into holes and eventually finding themselves in the Underground. Dostoevsky responded by stopping digging, climbing out of the hole, reintegrating into society, and gaining prominence during his lifetime.

Nietzsche kept digging. He dug himself into a hole of madness and died in obscurity. But before he suffered that final breakdown, he completed his greatest works, which people still marvel at today.

I am a hole-digging enthusiast myself, and this talk struck a chord. It made me go back and read Notes from Underground (it was the first time in over a decade that my Russian proved seriously useful to me). That novella is a perfect companion for the talk because it is written at the juncture between the two paths Nietzsche and Dostoevsky chose. Thanks, Darryl.

Hamish McKenzie’s Elder Statesman Arc

, one of the co-founders of Substack, is a fantastic person, with whom I’ve had the absolute pleasure to work. At that time, Substack was THE controversial hot topic in media, and Hamish symbolized the scrappy upstart, or the “bad boy” of media, depending on how you look at it.

Yesterday, however, Hamish was cast in a new role. As crossed swords with about whether the Stanford Internet Observatory is an agent of the totalitarian state-industrial complex or a truth-seeking scientific organization, Hamish was the reasonable adult on stage, making practical and unoffending points. I think Hamish will make a great Elder Statesman of Internet Media, and I look forward to more developments in the Elder Hamish arc.

And then things went in an… Interesting direction.

That Apocalyptic Diff

I really like The Diff, a finance/tech publication by . So I went to his talk (cohosted with )with great expectations, and whatever I expected, well, that’s not what I heard.

Their basic argument was that technology will cause the Apocalypse. That’s a lukewarm take at best today, but what really struck me was the shape of the argument. The shape of the argument was “The Apocalypse is obviously going to happen because the Bible says so and technology (specifically AI) fits the bill of how it might happen”.

I was… Surprised. I don’t really ever encounter “thinking starting from religious principles” in my daily life, and I was specifically amazed to hear it coming from one of my favorite tech journalists. This caused me to update some priors - more on that later.

Bait-And-Switch

After that, I went to a talk by Jon Askonas. This talk was a complete bait-and-switch, and I regret that I went. It started off with the premise of “We can look at Christianity through a purely anthropological perspective, it’s a very successful meme and maybe we can learn something about mimetic conflict by studying it” - which, sure, yeah, I buy. But it very quickly turned into a sermon, about how the SINNERS MUST REPENT and whatnot. I dislike that kind of bait-and-switch, I like sermons even less, so that part I did not enjoy at all. The one funny thing that happened at that talk was that I met . I’d never met him before and I thought it was rather amusing to meet a central figure in the rationality community at a sermon. Robin wrote his own piece after the conference, which mostly deals with Girard’s ideas - if you’re interested, you should read it:

Overcoming Bias

Making Sense of Girard

I attended a Girard-themed conference today, and in prep I read his classic Violence and the Sacred. He is a hard read for me, as he is often arrogant, repetitive, opaque, and exaggerates. Even so, t…

Read more

3 years ago · 19 likes · 32 comments · Robin Hanson

Now, at this point, I had heard three talks about how AI is going to cause the Apocalypse, and it really made me think. Some personal background is necessary - I am an atheist, I have always been an atheist, I grew up with no religious tradition whatsoever, and I haven’t even read a single major religious text fully1. So I tend to underestimate how much behavior is caused by religion. But now it makes perfect sense to me - there are a lot of Christians out there who believe that the Apocalypse is going to happen. They start at that point, and any reasoning that does happen, happens way downstream of the fundamental belief in the Apocalypse. Moreover, they’ve now seen 100000 movies about AI causing the Apocalypse. I don’t think it’s possible to have a reasonable argument with them about AI - the belief is too ingrained.

If we look at the problem more broadly - people project their biggest fears on AI, whatever they may be. For Christians, it’s the Apocalypse. For socialists, it’s capitalism, as Ted Chiang recently described:

I tend to think that most fears about A.I. are best understood as fears about capitalism. And I think that this is actually true of most fears of technology, too. Most of our fears or anxieties about technology are best understood as fears or anxiety about how capitalism will use technology against us. And technology and capitalism have been so closely intertwined that it's hard to distinguish the two.

For Peter Thiel, a libertarian, AI is most closely linked to the fear of a totalitarian world government.

Overall, this recent turn of events makes me more pessimistic about trying to talk to people about AI. Given how it touches people’s foundational beliefs that defy logic, logical reasoning is unlikely to change many people’s minds. Moreover, I now think the odds of mind-bogglingly stupid AI research prohibitions are pretty high, so the work might need to completely go underground. More on that in later posts.

P.S. WTF is the Katechon

One problem with religious people is that they love to take a term from a religious text that doesn’t define the term and then argue for 2000 years about what it means. “Katechon” is one such term that was in vogue at the conference. Supposedly, it’s something that “meant the historical power to restrain the appearance of the Antichrist and the end of the present eon” and people argue about whether it was the rebirth of the Roman Empire, or the Totalitarian State, or what. Luckily, we have ChatGPT now, and I managed to cajole it into just running “Katechon” as a DALLE-3 prompt without anything added to it. Here’s what it came up with:

Consider the debate settled. The Katechon is a giant cat with its own Death Star. It’s great, we should build it. Antichrists everywhere beware.

A bad bad no good period

China had a pretty bad time in the 19th and early 20th century. In fact, that’s an understatement - the country was more or less perpetually in a state of violence for over a hundred years. You can argue about when the period started - people point to the White Lotus Rebellion (1794–1804), the Opium Wars (1839-1860, with a break), the Taiping Rebellion (1850-1864) - all of these can be picked as starting points. Then, bad1 things just kept happening, including but not limited to:

• Boxer Rebellion (1899-1901)

• Xinhai Revolution (1911-1912)

• Chinese Civil War (1927-1949)

• World War II (1937-1945)

• Great Leap Forward/Great Chinese Famine (1958-1962)

• Cultural Revolution (1966-1976)

By the end of this period, China had some of the worst economic policies in the world: failed agricultural collectivization, communes, lack of trade, inefficient State Owned Enterprises - the lot.

“Economic Miracle” but really mean reversion

Things finally started moving in a good direction when Deng Xiaoping came to power and dismantled the failed Mao-era economic system. That marks the start of what people LOVE to call the “Chinese Economic Miracle” for reasons that evade me. Really, there was no miracle - China went from “utter failure of an economy with the worst policies in the world” to “globally average economy”. Here’s a long-term view of GDP per capita:

Basically, China did the same thing the rest of East Asia did just 20 years later because they had had an extra 20 years of madness. This chart ends in 2018, just FYI, in 2021 China slightly exceeded the world average GDP per capita but is still well below the East Asia average. Anyway, want to see what a real economic miracle looks like? See South Korea or Taiwan.

But because China’s population is so large, it had outsized impacts on the world, so people noticed and started raving about it. But again, and this is very important to internalize - China did not have a world-class brilliant government that had the best in the world economic policy2 - it just started in a really, really horrible economic place in 1978 and then got to be about average by 2021. It hasn’t jumped over the middle income trap like some of its peers.

Now, conceptually, you can divide the 1978-2021 period into two stages. The first stage focused on growth through exports. That’s when everything got to be Made in China. WTO ascension put that into overdrive and the period lasted until the GFC when the Chinese current account balance peaked as a % of GDP:

The Great Financial Crisis made China look for other sources of growth. In 2012, Xi Jinping came to power, and shortly after, in 2013 he released a 60 point plan for the Chinese economy. It was a visionary document proposing sweeping reforms, and it was applauded by the Global Elites everywhere. It would open up the capital markets, get rid of currency controls, and turn China into a modern service-based Westernized economy. Here’s how a contemporary Brookings report summarized it:

Now, the plan was pretty good, but there was one minor problem with it - Xi pretty much did the opposite of what his own plan said.

What happened instead of a transition to a consumption-based internationally open economy was a transition to a government/quasi-private investment-based economy. The Chinese government decided that the country would build buildings and… Oh boy, did they build.

I’m going to show a few charts from Ken Rogoff’s Peak China Housing paper but if you have any interest in the subject, I recommend reading the whole thing.

China’s real estate investment as a % of GDP has routinely been 2X higher than that of the US at the peak of the housing bubble.

Everybody in China already owns a home.

So now, more homes are sold that are 3rd homes than first homes (!!!).

A lot of the homes are empty.

And, while housing is the big kahuna, it is by no means the only kahuna. China also built too many offices, too many steel mills, too many coal plants, too many airports, too many highways, etc.

How did China pay for all of this? Glad you asked!

China has a WILDLY high savings rate - currently around 45%.

Now, for Chinese citizens, savings options are limited. They cannot freely convert their money to any of the major global currencies, they cannot buy shares of Apple, they can only invest within China. There are two major options to do so:

1. You can buy real estate.

2. You can buy financial assets.

So a lot of people went out and bought too much real estate, we’ve already covered that, but also, people bought a lot of financial assets. Now, the Chinese financial system is enormous and there’s no way I can cover all of it but we can look at a few things.

Firstly, there is A LOT of debt. Total debt in China is now around 280% of GDP.

To be fair, the line between Government and Corporations in China is thin and its position depends on your perspective. Let’s zoom in on two particular parts of this debt most closely tied to real estate. First, there’s the real estate developer debt - that’s fairly straightforward. Companies that build real estate borrow money to do so. A lot of building was accompanied by a lot of borrowing. The only interesting bit here is that the Chinese developers financed their building through pre-sales, so there has been a Ponzi dynamic in place.

The second piece is much more interesting - Local Government Financing Vehicles (LGFVs). Local governments in China aren’t allowed to borrow money directly from the financial markets. So they create these quasi-private companies that borrow money and use the borrowed money to build infrastructure projects - bridges and the like. They have borrowed… a lot of money.

Local governments in China own land. And historically, they’ve financed a lot of their expenditures (including those through LGFVs) through land sales and land-related tax revenue.

To sum up what we have so far: China’s growth for the past 10 years has been fuelled by real estate. It accounted for about a quarter of the economy. Enough housing was built for living in, and then a bunch more was built for savings/speculation depending on how you look at it. A lot of debt has been issued by real estate developers and local governments, and Chinese citizens (directly or indirectly) bought that debt because they are not allowed to invest outside China.

And then the whole thing imploded.

Real estate prices are falling.

Land sales have plummeted.

Real estate developers are collapsing

LGFVs are running out of cash

What will happen next? LGFVs will default/restructure. Local governments will have to drastically cut spending. For the record, local governments’ investment in infrastructure generates an estimated 14%–15% of Chinese GDP, that’s on top of the real estate being 25% of GDP. You can already see some cities shutting down hospitals, withholding wages, stopping bus services, etc.

The domino to fall after that will be the financial system. According to Grant’s Interest Rate Observer:

LGFV borrowings make up 20%–25% of total bank loans, and LGFV bonds account for 51% of all corporate bonds outstanding3.

And that doesn’t count developer loans, suppliers to the industry, etc., etc. The banking system in China cannot survive this coming tsunami of defaults.

Speaking of tsunamis, it’s time we paint a more graphic picture of what’s going to happen.

The real estate industry grinding to a halt and developers going bankrupt were the volcano eruption.

LGFVs going bankrupt will be the earthquake.

The banking system will drown, as already mentioned, in a tsunami.

That’s a lot of disasters. But did you really think this was going to end without a Mechagodzilla? Don’t be ridiculous.

The Mechagodzilla in this case is that China is running dangerously low on foreign reserves. Again, thanks to Grant’s:

The last domino to fall will be the currency. China will have to devalue its currency - we don’t know by how much, but 30% wouldn’t surprise me.

Putting it another way - China is about to experience the equivalents of the US 2008 housing bubble bursting, a Great Depression-tier banking crisis and a 1997 Asian financial crisis-type currency devaluation all at the same time.

Given how central real estate has been to the Chinese growth model, this isn’t an industry crisis. This is a crisis of the entire Chinese economic model. There will be no returning to the status quo.

Here’s a bit from Logan Wright’s testimony to the USCC (I recommend reading the whole thing):

Also:

I completely agree. I think that the idea that China’s economy is or will soon be the strongest in the world is about to be completely discredited, likely for the remainder of our lifetimes.

Now, here’s the thing. I’ve structurally been bearish on China for a long time and on a very recent episode of the Sharp China podcast (paid) said the following (lightly edited for clarity):

…every few years you get sort of the folks who've been really bearish on China for a long time, like, this is the time it's going to collapse. It's finally happening. Oh my God, I'm finally right. And the thing is, is people, I think, intellectually have been right about all the problems in Chinese economy. There are plenty of people who've been right about all the different problems and what they're facing, but so far it hasn't collapsed. And I think we're still not at the point where the economic problems are such that you're gonna see some massive crash or massive financial crisis or some sort of significant political pressure on CCP. Could be wrong, right? You know, you never say never, but it always, I think, is important to try and avoid sort of the binary boom or bust we talked about earlier. Sometimes they really can figure out how to muddle through. And especially considering … when you look at the economic problems in China, one of the things I think a lot of foreign economists, analysts, a mistake they make is they look at it like it's a Western or Japanese economic situation. And they don't talk about all the different levers and structures inside the PRC system that the party has put in place to harden the system, to maintain social stability, to force brokers, banks, many of which are state-owned or have state capital to behave in ways that the government wants to that you wouldn't necessarily see in a quote-unquote free market economy or free-ish market economy. So there are a lot of ways that there's just still a lot of pieces that the party and the government can bring to bear to keep things together. In many ways, you look at what Xi Jinping has done over the last several years, he's been much more focused on hardening the system, specifically to deal with all sorts of crises, including something like a financial crisis.

And having been called out (I’ve been structurally bearish on China since… 2016 maybe?), I decided to go on the record with some predictions, which brings us to what’s going to happen next. Now before we get there, I want to list a few beliefs I have that inform my predictions:

1. China generally lies in its economic data. Whatever numbers we see, the reality is worse.

2. The “brilliant technocrats in the CCP” thing is a myth. China’s growth has been powered mostly by mean reversion. They have… Average technocrats.

3. The stability of the economy is a part of the Chinese social contract. A massive loss of the population’s savings/jobs will lead to discontent.

Okay, so given that, here’s what I think will likely happen:

1. Continuing wave of defaults in the real estate sector and adjacent industries.

2. Mass LGFV debt restructuring.

3. Banking crisis where the equity of pretty much all banks gets wiped out.

4. General financial panic.

5. Currency devaluation.

I expect the bulk of it to happen within 24 months, and the Minsky moment to occur within that period with a 90% probability, likely sooner than later.

During COVID, I listened to the entire Revolutions podcast by Mike Duncan. It’s fantastic (though the first season is the worst, you can safely skip it). One thing that resonated with me is that Mike Duncan thinks that the French Revolution started on the 20th of August, 1786 when Charles Alexandre de Calonne told Louis XVI that the French monarchy was broke. China is about to go through an economic crisis of similar (or potentially greater) magnitude. Their youth unemployment is already over 20% and rising (and, lately, not even published). All of the ingredients for a revolution are here.

Well, guess what? the CCP really, really doesn’t want a revolution. What can they do? Note that I will skip over some technocratic policy choices like - does the federal government guarantee the municipal debts, does it guarantee banking deposits, etc. To me, they are interesting but, ultimately, unimportant. What’s important is - what can the CCP do given that their old economic model is done and there’s no going back?

There are 4 doors the CCP could theoretically take, but two of those are closed to them in practice, as far as I can tell.

The first door is to try to rely on the outside world for growth through a combination of attracting foreign investment and focusing on exports. This door is closed to them because their relationships with the major economies are at a cyclical low. In fact, the US, the biggest market in the world, is actively reducing its dependence on Chinese imports at a very rapid pace.

The second door is economic liberalization and a switch to a consumption-based economy (the 2013 plan). This door is closed to them because it would take a long time and a lot of pain. The reason you can’t do it quickly is - to jump start a consumption-based economy, you need to persuade the Chinese citizens to save less and spend more. And it’s going to be impossible to persuade them to do that when they are losing their existing savings at a rapid, rapid pace. A lot of pain over a long time would cause popular discontent. The CCP would have to increase repression - and you can’t really run economic liberalization with one hand and political repression with the other. Doesn’t work.

The third door is war. In Totalitarian Dictatorship 101 they teach you that if your economy is failing, you need to blame it on something other than poor leadership. You can’t really sell the “you’re losing your savings because we’ve mismanaged the economy” bit to your citizens. But “you’re losing your savings because it’s a necessary sacrifice to reunify China by capturing Taiwan” - that works, not to mention all the war-related jobs that would get created. I assign a 60% probability that this recession will eventually result in a war, though, as always, the timelines are hard to predict. I wouldn’t be surprised if a war was to begin at a weak point in the US political cycle, though, so 2024/early 2025 makes sense but my level of confidence in this prediction is not very high.

The fourth door is the CCP booting Xi out of power and blaming him for the recession. That might work but isn’t guaranteed to work. Given how successfully Xi has consolidated power over his rule, I do not find this too likely but I will be the first one to admit that internal CCP dynamics are completely opaque to me.

The last possible scenario for the CCP isn’t a door they can walk through, but a window to get thrown out of. It’s possible that nothing the CCP does will save the regime. The dynamics of discontent in China is hard to judge from the outside but the CCP has invested a lot in mechanisms of repression, so I think this is unlikely to occur before the CCP tries the war option.

That was a lot of writing, and I’ll stop here. But if there’s any interest, I can write another piece on what the global economic/financial repercussions of a Chinese megacrisis might be. Let me know!

With Machine Learning this changed - somewhat. Now you can use a neural net to throw large and arbitrary amounts of compute at most problems - provided you shape the problem into something a neural net can work with. But in practice, for almost any problem there comes a point where it doesn’t make sense to keep training the neural net and humans need to step in and change something.

Now, we are on the precipice of a new (and almost certainly the last) era in compute. We are going to be able to throw arbitrary amounts of compute at ANY problem. You can already kind of see how it works if you play around with any of the AI Agents, here’s a screenshot of AgentGPT:

It breaks down a task, any task, into subtasks and then uses the available tools to complete the task. It more or less has its own OODA loop. It executes this loop for a while and then just stops mid-execution when it runs out of the compute you allocate for the task.

Now, right now, there is a shortage of GPUs and some other boring market dynamics in play so we can’t really throw truly ARBITRARY amounts of compute at problems as consumers - there are API quotas, GPU quotas, all kinds of stuff. But the beauty of capitalism is that when we want a lot of something, a lot of that shall be built, so the situation won’t stay the same forever. The endgame looks like this:

1. There is a hard physical limit on how efficiently we can convert energy into computation called Landauer’s principle. We are currently racing towards it and in the best case will reach it by 2080 but more likely, we will just slow down at improving on how close we are to it sometime before then.

2. The price of energy is not decreasing. It’s going up in nominal terms but roughly in line with inflation. Now, this might change if we get fusion or something, but the base case is that the price of energy in real terms is roughly flat.

3. That means that barring some unforeseen advances in energy production or our understanding of physics, we are going to run into a pretty straightforward limit of dollars per unit of compute (FLOPS/$ or the like).

4. So, we’re going to live in a world where we can throw arbitrary amounts of money at ANY intellectual problem.

This isn’t a novel idea - that’s approximately how the human world works in Permutation City by Greg Egan (or at the beginning of it, anyway). But we are rapidly approaching that world, the world where we tell AIs:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5.

Now, most household tasks are too trivial to warrant a budget, they’ll be paid for either by ads, a cheap monthly subscription bundle, or even through your own electricity bill if the task is simple enough to complete locally. But for any serious intellectual/creative tasks - the tasks that will remain valuable, there will likely always be a compute budget set.

So in a way, computing history will have had 3 eras:

1. You pay someone to write a program to accomplish a particular task.

2. You pay someone to translate your particular task into something an LLM can understand (this is where we are today).

3. You give an AI agent a task and a budget and that’s it.

Generally stated, the problem we need to solve to get to era #3 is this:

Given any unbounded1 problem X and compute budget Y, what’s the most efficient way to spend Y on compute to achieve the best result on X?

This… Is a hard problem. But my assertion here is that whoever solves this problem the best will build the most valuable company of the century.

The current generation AI Agents just kind of punt on this problem and say “just run this loop until you run out of compute” but this by no means guarantees the best result. In fact, the AI agents of today frequently don’t produce any result - just some intermediate steps of a “thinking” process. What needs to happen for sure is when you ask an AI agent:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5.

It answers with a basic report and when you ask it:

How will the Nike stock do if there’s a communist revolution in Indonesia? Your compute budget for this is $5,000.

It answers with a better report. Now, naturally, if you want to dig into its process, you can do so, but ultimately people want results, not process.

Many other things also need to exist to solve this problem! Estimators of whether a problem is better solved by code or a neural net given a certain compute budget. A database of all the tools and APIs the agent can use or subcontract to in the world. A way to pick the best result out of all the intermediate results, etc.

Is it possible that the solutions for this problem will be domain-specific? That one company will be the best at spend 100$ to make a video game and another will be the best at spend 100$ to write a novel? For sure, that’s possible. But if I were to guess, I’d guess that a large part of the solution will generalize, and besides, if you’re shooting at the most ambitious target - you pick the most ambitious goal, which is the general case.

In September 2000, The Project for the New American Century, a neocon think tank, published a paper arguing that it’s about time for America to formulate a new military strategy, increase defense spending, and make better choices - like killing the Joint Strike Fighter program in favor of developing technologies we’ll actually need. The paper is worth reading, or at least skimming - it gets a lot of things right.1

Rebuilding Americas Defenses

853KB ∙ PDF file

Download

Download

Now, shortly after, 9/11 happened, defense spending did rise, but better strategic choices were not made. The War on Terror dominated conversations about security, and nobody wanted to talk about serious strategic competition with our rivals.

Since then:

1. Russia has transitioned from a nascent democracy to a belligerent totalitarian state with a severe dislike for the US.

2. China has rebranded itself from a country integrating itself into a US-led world order to our main superpower rival.

3. North Korea has graduated from a psycho-cult of a state to a psycho-cult of a state with nukes and ICBMs.

It’s about time for the US to get serious about strategy again.

The worst decision made by America post-Cold War, according to Rebuilding American Defenses was shutting down the Brilliant Pebbles program. Brilliant Pebbles was a global ballistic missile defense program first proposed by Lowell Wood and Edward Teller2. Here’s how it was supposed to work:

The US would launch many autonomous satellites (Pebbles!) into Lower Earth Orbit. They would detect ballistic missile launches and launch themselves at the missiles, attempting to hit them early in the boost stage and destroy them just by the force of impact. Here’s what the satellites would look like:

Now, most of this is a life jacket - designed for protection against laser strikes and such. The core part of each Pebble would look like this:

It’s basically a propellant tank, some sensors to track the missile, and a processor to run the whole thing. Even in the late 80s, this was mostly off-the-shelf technology.

The number of Pebbles you’d need in orbit varies depending on your goals and their exact specifications, but the highest estimate for truly global protection against ballistic missile launches is 100,000 pebbles. There are useful versions of the program that require fewer Pebbles (even 1000 works if your goal is local protection)3.

This is… A great design. It offers many advantages like:

1. Missiles are easiest to hit during the boost stage. They are easily seen (as they burn fuel), and their maneuverability is limited.

2. A swarm of autonomous Pebbles means there is no command center that can be taken out with a first strike.

3. Pebbles can hit their targets from great distances because their relative velocity is so high.

But their main advantage is that they are cheap. They fit the Nitze criteria, from Wikipedia:

Nitze stated that if a BMD system were to be successful in fulfilling this role, it needed to meet three criteria.^[1]

The first two criteria are simple: the proposed system has to actually work, and it has to be able to survive attacks against it. While these may seem obvious, many earlier systems failed one or the other of these criteria. For example, Nike Zeus was highly vulnerable to attacks by the very ICBMs it was supposed to defend against,^[2] while the later Spartan missile left many questions about its ability to actually destroy its targets at reasonable range.^[3]

The third criterion, the one the criteria is really known for, is that the system must be "cost effective at the margin".^[1] This is essentially a common-sense rewording of the earlier concept of the cost-exchange ratio, the amount of money needed to counteract a dollar of offensive capability. Previous BMD systems had always been far more expensive than the missiles they were designed to shoot down; during the Nike-X program in the 1960s it was estimated that every dollar the Soviets spent on new ICBMs would require $20 of spending to counter it, a 20-to-1 ratio.^[4]

The Nitze criteria were peak Cold War theorycrafting, gotta love it4. Anyway, Brilliant Pebbles was the best plan for a global ballistic missile defense ever to exist. If you’d like to read more about the concept and rationale - there’s this deck from Lowell Wood, one of the program’s creators

Brilliant Pebbles

2.72MB ∙ PDF file

Download

Download

The program was in development in the late 80s/early 90s and funding was allocated to build it. Plenty of theoretical studies have concluded the program’s feasibility. The program got to the stage of testing but experienced REALLY bad luck - it suffered three failures in its tests for reasons unrelated to the core program - booster failure, etc.

Then, Brilliant Pebbles was savagely murdered by Sam Nunn5 and others in 1992-1993. Their arguments were roughly as follows:

1. The program is expensive.

2. It might take 10 years.

3. Nukes aren’t really a problem anymore.

However, the key argument was that it would violate the ABM treaty. That’s REALLY what killed the program, and goddamn, does this look dumb in retrospect. The ABM treaty was a treaty between the USA and the USSR to limit the development of their ballistic missile defense systems signed in 1972. Now, in 1972 it made some sense - it would de-escalate tensions and stop the arms race. But by 1992, one of the signatories had ceased to exist. Russia had bigger problems than ballistic missile defense and besides, there were even conversations with Yeltsin about including Russia in the envelope of protection offered by Brilliant Pebbles. But Sam Nunn and his allies didn’t see it that way. They thought we’d magically get to a world without nukes, and so, Brilliant Pebbles was a waste. So they used the ABM treaty as a reason to kill the program. Funnily enough, the ABM treaty was re-signed with post-Soviet countries in 1997, and then the US withdrew from it in 2002. If you want to read the full history of Brilliant Pebbles and its demise, you can do so here6:

The Rise And Fall Of Brilliant Pebbles Baucom

425KB ∙ PDF file

Download

Download

So, Brilliant Pebbles died, many moons have passed, and some things have changed since. Most importantly, everything involved in making Pebbles got cheap and available to private companies.

If you think about the design, your top costs are:

1. Getting stuff into space.

2. Sensors (notably LiDAR).

3. Processors.

Now, let’s look at what happened to the price of these things. The cost of getting stuff into space has fallen ~10X:

LiDAR prices have fallen by ~100X since the early 2000s. I don’t have data going back to the 80s but it’s safe to say that LiDAR was very expensive and got very cheap. LiDAR also got much, much better.

Computation got ridiculously cheap:

I don’t have the exact specs, but my best guess is that a Brilliant Pebble could cost on the order of magnitude of $20,000 today - $10,000 launch cost, $3,000 sensors, $1,000 compute, $1,000 solar panels/batteries, etc. That would mean that the cost of materials for global protection against ballistic missiles is… $2 billion. That is peanuts, really. Defense spending in 2023 is going to be over $800 billion. You could sneak in global protection against getting nuked in there and nobody would notice.

But, as it stands today, the live players are largely outside of the government and in tech businesses. But here’s the thing, building Brilliant Pebbles is A LOT like building a tech startup.

1. You launch cheap satellites into space like Planet Labs or Varda.

2. You make a LiDAR-based guidance system like Waymo or Cruise.

3. You write some distributed autonomous decision-making code, which every top nerd in the country would LOVE to do.

The weird thing about starting a Brilliant Pebbles startup (let’s call it Marbles) is that you’re building something for the government that the government hasn’t allocated the money for yet. But, at the same time, by building the thing, you are increasing the odds that the government will pay for it. If you show a successful interception, and you say how cheap it is, and the theoretical studies were already done in the 1980s - it gets to be a really easy sell. Moreover, you only have to sell the program once. You can pick your time and do it right after North Korea runs a successful test, or Russia/China invades someone, etc. It’s much easier to pass a law when you have a solution on the table, it’s not super expensive, and there’s media cycle urgency around it.

Now, let’s run some numbers to see if the math works:

Let’s say that the all-in program cost is ~$5B (the older estimates put it at ~$50B but we have seen massive price drops). For a project like this, the government will generally allow contractors to make a ~10% margin. So your net profit is expected to be conservatively at ~$500M. What would it cost to run a successful interception test? Hard to say, but Varda has achieved launching a factory into space with $50M in funding. Seems like a similar level of difficulty milestone. So if investors invest $50M, and own half the company, they could make $250M. Not bad, but not quite what VCs are looking for. That being said, the $5B estimate is the lowest-cost possible estimate. If you demonstrate the technology, and the government allocates the money, you can sell the company at that stage to a defense prime contractor, and they will make sure to balloon the project to $10B and earn twice as much. That means, that they’d be willing to pay closer to $1B for the company. That gets into venture-like returns territory. This is all napkin math, of course, you’d want to look at discount rates, maintenance contracts for the long term, etc.

Now, there are certain legal/political (the two are inseparable in this case) risks when it comes to starting a private ballistic missile defense company. The good news is that you don’t really require any controlled materials for testing - you can just build the thing and do it. Even so, anybody who develops such technology in the US is likely to attract attention from the Feds. You’d want to pick investors who can provide you the greatest political cover - like Josh Wolfe who’s testifying before Congress, Katherine Boyle whose sole investment focus is American Dynamism, the Founder’s Fund which has its own Senator, etc.

That said… It’d help if you are politically connected and completely clean from a National Security perspective. That’s why I can’t do it myself - in no world will anyone in the NatSec apparatus think that it’s OK for a Russian/American dual citizen to build ballistic missile defense system prototypes in a warehouse in LA. I was recently told a story about how a Russian immigrant tried building a space company with some limited dual-use applications in the US and was told to sell his stake for peanuts very, very quickly. Ballistic missile defense is a touch more serious than that.

As for the precise laws you’d have to skirt - I’m not a lawyer, IDK, but I can’t see anything super illegal in the early R&D program. You’re a satellite research startup!

Now, there is another advantage to doing ballistic missile defense privately. It helps from a game theoretic perspective. If the US runs a government program to build a global BMD program, it might invite preemptive actions from Russia/China etc. But if as much as possible is done privately, it shortens the time our adversaries have to do anything about it. Ideally, with a program like this, you want the government to announce it and launch the first batch on the same day.

My favorite thing about startups is they can reinvent the rules. Most startups don’t do that, most startups try to follow a playbook, but the great ones change the name of the game. OpenAI did that with its weird investment structure. Anduril did that by starting a defense prime from nothing. Marbles can do that too, and perhaps have more impact than either. After all, what startup has a greater potential to literally save the world?