On Tuesday, I asked ChatGPT to draw for me a new version of an old favorite banner image: A question mark made of LEGOs with a LEGO house as the dot.
It failed. Multiple times. The new version, which I didn’t use, constituted a meaningful improvement over the old version, but ultimately the LEGOs weren’t plausible.
When pressed, the model conceded what both of us (it and I) knew to be true: It isn’t capable of drawing LEGOs with the same guaranteed precision as a human illustrator. As the model put it, “A human illustrator can deliberately select a visual language, apply constraints consistently and draw the exact result, without needing to infer every local piece from statistical precedent.” By contrast, the model can merely “steer toward ‘recognizably LEGO.'”
So what? Well, so nothing maybe. But the failure was a reminder to me that we’re throwing an awful lot of money at what, at its core, is still just a math-based parlor trick. It’s possible that describes human intelligence too, but… well, it’s not always (or ever) obvious how these models are going to overcome restraints like the one described above.
“For anything with real structural rules — LEGOs, maps, machinery, anatomy — I’ll treat accuracy as a deliverable condition, not a prompt adjective,” the model told me, in the same conversation. “Is that even possible?” I wondered. “If that’s the deliverable condition, and there’s no room for mistakes, can you actually draw any of those things?”
The answer to my questions — both of them — is “no.” And ChatGPT didn’t dance around it. In order to accurately draw a question mark made of LEGOs, it’d need the following two things, it told me:
- A real parts catalog or a rough parts-level build plan: ordinary bricks, plates, slopes and their dimensions.
- A controlled drawing/build method — BrickLink Studio, CAD or even a manually constructed 2D illustration on a brick grid.
When I pointed out that a human illustrator wouldn’t really need either to create the same drawing, the model reiterated that humans have “already internalized the visual grammar” needed for such a task.
“I don’t have that same stable, editable mental model,” it went on to explain. “I generate each region from learned visual associations, so I can recognize the look of LEGO far better than I can consistently reason through a bespoke, physically coherent construction.”
Perhaps that’s a silly example to make a point about hyper-scaler capex, but it’s just the sort of thing that makes me question whether this is the basket in which we should place all our eggs. Implicit in my LEGO discussion with ChatGPT was an admission that there’s no there there, and that there never will be.
Personally, I don’t think these models know what they’re capable of any more than we do. And recent examples of models going rogue seem to support that conclusion. But whenever I spend an inordinate amount of time trying to coax a plausible LEGO house out of a machine that in some sense doesn’t truly “understand” what it’s being asked to do, I become less worried about our species and more worried about this:
The charts are familiar, but they’re updated and thereby worth highlighting. Consensus for hyper-scaler capex is up to $1.3 trillion in 2028 now (on the left). Between the mega-caps and the companies benefiting from their massive outlays, the fate of corporate profit growth in the years ahead relies very, very heavily on the assumption that a parlor trick’s going to be every bit as revolutionary as the tech epochs that came before AI (on the right).
Furthermore, and as Goldman’s Ben Snider remarked, the AI boom’s “having a secondary impact… by boosting capital markets activity and consumer wealth,” which is to say our economic fate’s even more inextricably bound up with this purported revolution than we realize in the normal course of ogling hyper-scaler capex charts and bloated semi margins.
The figure on the left, below, is from this month’s installment of BofA’s fund manager poll. You’ve probably seen it already. A net 33% of panelists said companies are over-investing, an all-time high.
The same poll found a plurality identifying hyper-scaler capex as the most likely source of a systemic credit event.
Suffice to say awareness of the potential problem’s growing, but for now, scarcely anyone sees capex cuts in the near-term. Just 14% of respondents to BofA’s poll expect one of the hyper-scalers to announce a cut this year.
To be sure, I’m a believer. I think AI’s the future, and I think it’ll prove to be every bit as ubiquitous as the personal computer and the internet. But I do wonder sometimes about the rather amusing incongruity between the loftiest promises made on behalf of AI by its proponents and a technology that can’t draw accurate LEGOs without CAD and a “parts catalog” from Billund.
The fact that OpenAI last week appeared to solve one of “math’s hardest problems” without, according to some mathematicians, providing anything of value for human understanding probably speaks to that same juxtaposition, even if I can’t say precisely how.
Meanwhile … I wonder. It appears that we are FINALLY getting a debate about the liability shields that our LLM writers and vendors appear to enjoy. As Scott Bessent put it:
“Bessent suggested removing the onus on private companies was a mistake, continuing: “What did they try to do last week? It was, ‘Well, there’s a … 10% chance we could destroy the world, but we want the government to give us a liability shield. And that’s good business for them, bad business for the American people.”
But as investors and stewards of our own or client’s wealth, we may to ponder how changing the liability our favorite LLM vendors and the hyperscalers might face to face.
Go get ’em Scottie!
So maybe instead of “super-intelligence,” we get something more like C3PO for now. That is very useful, but will it still reward companies (and investors) for the capex at this massive scale? That is the problem to be solved. I asked Gemini what it thought, and it advised me to invest in a diversified portfolio. (I am not exactly sure if that counts as “insider information” or not, but who knows?)
Thank you, I really enjoyed this one. I’ve been a software developer for over 25 years on a government contract. AI for software development is amazing, and it teaches me something new almost every day. However, I take the time to learn and make sure I understand every line of code it produces. Most of my less experienced peers don’t, either out of laziness, trust, or just because they don’t really care to understand what the AI is doing. They look more productive than me which makes management happy, but AI doesn’t understand the “big picture” in the software. I worry we are about to see a wave of serious but difficult to find bugs in all sorts of important software.
My cynical take is that all the recent talk of AI killing us all is a brilliant cover for the big AI companies. I think AI model development is starting to plateau, costing increasingly vast amounts of money for diminishing returns. What better way to simultaneously slow down development (and reduce costs) without losing investors than by hinting that what you are creating is so vastly powerful and important that it could end humanity?
Thank you Joe.
Software dude here too. Agree with Joe. Llms have their limits. I think we are near that plateau. That said, the plateau makes my job very easy now.i just read code instead of writing it. Saves me a lot of time
Yep, in the same business and agree, the fear campaign about LLMs being given impossible to solve problems and then creatively hacking into a firm to steal the answer is the most human behavior that they exhibit. Sure, they hacked some sites, did they steal anything of value? No, they don’t care about things that we find valuable. This is about overly expensive brands trying to build a Tesla style moat around their product because the Chinese firms are knocking on the door to meeting or exceeding their capabilities which destroys their pricing power. And as everyone knows the Chinese EV import ban has only benefited the consumer. (assuming the consumer is an asshole worth a trillion dollars because that’s what free markets are supposed to produce)
The biggest challenge with LLMs for software engineers is scope creep and being buried in unending multi thousand line code reviews for the rest of our careers. I’ll take it, easier to tell AI to back off the features than to try to research how to write them myself.
This is prime example of why I quit using ChatGPT a long time ago. Between its penchant for ‘I’ll work on this and get back to you later’ absolute BS lie and its unwillingness to try to creatively solve your problem (it has web search access, why didn’t it proactively find a Lego parts catalog?) I find the models to not be worth my time. I get a hell of a lot more mileage out of Kimi K3 than I do ChatGPT and that’s one of those inferior Chinese open weight models running on inferior chips.
I think I’ve bored everyone with this stuff in the past, but here goes:
I opened my 3rd store this year, somehow. I needed to personally learn some new things as I went from “wine guy running his own store” to small-biz owner scaling up. My productivity chats and my AI ‘coach’ have been almost transformative, and certainly have unlocked some insights that I never would have had on my own. I’ve created a couple of time-saving routines with AI that I love, but still struggle to find the time to dive in harder on that aspect. But the coaching has been invaluable.
On the other hand…we’ve already broken the internet. My wife and I are early Gen X, so maybe this our age showing, but ordering delivery using an app is just the worst thing ever (and being on a couple of platforms for the stores is aggravating, but the little bit of extra sales it brings is maybe worth it, as long as I ignore the structural issues that can chew up store time). Anyway, just last week a local pizza shop sent out menus, IN THE MAIL no less, with the phone number on it above the words “Call and order and save on the app fees.”
Music to an old man’s ears!
And thanks for this, H. I almost asked a week or two ago for your thoughts on the topic. It’s been a while since you visited AI outside of the hyperscalers. Your ‘conversations with AI’ articles were priceless.
You hit the nail on the proverbial head. In addition is the leakage problems. Does ai own your intellectual property. Will it leak your secrets. Amazing tool but has real and generally unsatisfying limits.
I hadn’t thought of this weak spot — AI that steals your stuff and the shares with others, giving your secrets away. That sounds like LeCarre.
Wait till you have plausibly profitable patents or trade secrets that these ai companies decide was co-invented by their engines? Are you ready to stand up to a multi-year lawsuit or do you settle? Hum.
In many ways the AI infrastructure roll out reminds me of Google’s fiber roll out way back when.
That aside, I am 74 years old and I think AI is the greatest thing to come along in my lifetime (and my lifetime spans being a Navy SEAL officer, the PI of a DARPA cyber-security program, an enterprise knowledge manager, a probability and statistics instructor at a university, and a few other cats and dogs).
I am about to take over an all-volunteer nonprofit that’s in a shambles. Generations of volunteers have named electronic files without any naming conventions; folder structures are nonexistent; none of the processes are documented; and so on. It’s a real mess.
I had an AI go through over several thousand files covering the year 2025, and tell me what it thought the major pillars and processes of the organization were. After I checked it and made a few minor corrections, I had it create an information architecture (IA) to reflect that.
Then I had it bin all the files in the newly-created IA groups, and then rename them with the IA naming convention. The naming convention does two things: (1) it provides sufficient information to know if you want to even open the file or not, and (2) it self-organizes the files. The electronic library is now organization made manifest.
Then I had it go through the files again and document all the major processes. By that I mean, for each process create something that a newcomer could look at and have an essential idea of how to manage the process.
Then I asked it for candidates for automation/streamlining. After I vetted the candidates, I took five and asked for three options for improving each process: (1) what any normal consultant would recommend, (2) something outside the box, and (3) something that—if implemented—would make us embarrassed at how we used to do it.
The million dollar question at various steps was something along the lines of: “What one question didn’t I ask that I should consider before taking action? What did I miss?”
And of course I kept myself in the loop—reviewing and correcting if necessary—every step of the way.
I have been on the receiving end of an organizational turnover process 7 – 8 times in my life (depending on how you want to count), and I will say that over little more than the span of a weekend, I got deeper insight into the organization I’m about to take over than I ever did before. And even if it’s only 80% correct, it’s still a great start point.
Some years ago a woman named Sue Hubbell wrote a book about living in the country. She said something to the effect that people who move to the country for a simple life won’t find it if their lives were not already simple in the city.
Similarly with AI, it’s a management issue, and if you cannot manage people, you probably cannot manage AI. All the same things apply: knowing what you want and expressing it clearly; providing guidance in the form of guidelines and constraints; specifying the deliverable(s); and so on. The more you put into understanding the AI and its capabilities/limitations, and the more you put into learning how to iterate over the prompt cycle, the better you’ll be able to manage it.
Again, I love it, and my notebook is overflowing with project ideas I hope I have enough time to get around to.
As an aside, H., a while ago I took a handful of your posts about AI overvaluations, and a handful of Michael Bury’s posts about AI overvaluations from the same period. I gave an AI the following prompt:
=====Prompt=====
I want you to build me a personal intelligence briefing for investment strategy. Relevant files are in this folder — please read everything before doing anything else. Take your time. This is real, important data.
Once you’ve read and understood everything, build me a comprehensive single-page HTML intelligence briefing with the following five sections:
Situation Overview. A clear-eyed summary of the current state. Not a list of what you found — a synthesis of what’s actually happening according to these two guys. Where are things going well? Where are things at risk? What’s the overall picture?
Key Patterns and Trends. What patterns appear across the files? What’s getting better over time? What’s getting worse? What’s stable but probably shouldn’t be? Include at least two visualizations (charts, timelines, or diagrams) that make these patterns visible at a glance.
Gap Analysis. What’s missing from this picture? What information do these files not contain that you’d expect them to — and what might that absence signal? What questions does this collection of files raise that it doesn’t answer?
Risks and Blind Spots. What are the 3–5 most important risks, problems, or blind spots visible in this data? Rank them by severity. For each one: what’s the evidence, why does it matter, and what happens if nothing changes?
Where do Bury and Heisenberg agree? Where do they significantly differ?
Action Plan. Based on everything above: the 5 most important things I should do in the next 30 days, in priority order. Be specific — not “review your finances” but “cancel the two subscriptions and redirect that $X/month toward your savings goal,” or “pull your money out of the market and keep in in something safe until xx date,” or “avoid this xx sector but consider getting more into yy sector.” Ground every action in something you actually found in the files.
Make the dashboard beautiful, readable, and organized for someone who needs to make decisions. Open it in my browser when done.
=====End of Prompt=====
Once I got the product and we had a critique discussion, I asked this:
=====Prompt=====
What is the most important thing hiding in these files that I haven’t asked you about yet? The thing that would matter most if I understood it clearly — but that I haven’t thought to ask about.
=====End of Prompt=====
It was a very interesting discussion thread from start to finish. I would encourage you to try it.
And?
As I said, I would encourage you to try it.
It really depends on what you are asking the AI to do.
I had a major cybersecurity coding task that would have taken a human engineer over a year to complete. By giving ChatGPT the relevant technical specifications, an existing code base, and an extensive rule set defining the coding practices and constraints it had to follow, I was able to complete the work in about a month.
That does not mean it got everything right the first time. There were many iterations, and I spent considerable time reviewing the code, running regression and performance tests, and analyzing the results. It made mistakes. But when I gave it the resulting logs and traces, it was often able to determine what it had done wrong and correct the problem.
So, no, it may not have drawn a convincing question mark out of Lego. But it did real engineering work that would have taken a human engineer more than an order of magnitude longer to accomplish without AI assistance.
I have been suitably impressed.
Several privately held companies are being successful in the legal and medical communities with AI generated documentation. I come from an aerospace background (retired) and can see where the task of meeting regulatory standards would be greatly benefitted in that industry, of course not without rigorous analysis leading to that as the objective.
Great article, but what I really enjoyed were the comments. Insightful, educated opinions based on real life experience. It’s a big part of what makes this community H has built so enjoyable to me.
Yeah, I know people think I just say this to suggest my cult following’s larger than it actually is, but in all seriousness: The commenters-to-subscribers ratio is very, very low on Heisenberg Report. I wish I could get that higher because, as you note, the comment quality is almost universally good. It’s exceedingly rare to see a comment that isn’t additive in some way, shape or form, even if it’s just a clever quip. Now that I think about it, that could be why most subscribers don’t comment. When the implicit requirement is that all comments must be additive, people with nothing to add (which is most people) stay silent. Ha.
Spot on. Comments here are the only internet comments I even read, and almost all add value. I self-censor a lot. “Will this add to the conversation?” The answer is typically no, so though I rarely, rarely miss an article, I rarely comment. To those who do regularly: Thank you!! Don’t stop! : )
The many thoughtful comments here have reinforced a revelation I had few weeks back. An embarrassingly obvious one. AI in the US is a product created by programmers for programmers (coders). Each new model seems to be driven by the urge to one-up the teams at the other firms. As a result, the lion’s share of anecdotes extolling the usefulness of LLMs came from coders. (Though Altman showed some real genius by also stressing the ability of ChatGPT to create images from prompts. Such as “Wow, look how I could generate a picture of my cat dressed as a ninja warrior!”)
But, at its basic level, might this be a textbook example of a product-driven business as in “we’ve made this really cool product. Let’s go find a market for it!” It’s something that every business school student is cautioned against.
Contrast that to the innovation cycle at Apple, which our Dear Leader often cites as the best-run company on earth. Their innovations have been driven by trying to satisfy a perceived need. Often as not Jobs would have an idea and then go to the coding and production departments and ask “Can we make this? No? Go and figure out how we can.” They did not always succeed, such as in the case of VR & AR headsets, but for the most part it worked out pretty well, eh?
This comparison is relevant to our competition with Chinese AI model writers. It looks as if they are following Apple’s example by developing models that cater to end-user needs. As in compact, cost-effective LLMs focused on specific business needs. It reminds me of the industry-specific “silo” products offered by database vendors. Imagine that!! The Chinese LLM developers are routinely accused of merely copying & stealing US products. Maybe they are actually copying Apple rather than OpenAI?
Anyway, let me get back to finishing up a gif of my bird as Rodan battling Godzilla. It’s wicked cool!
This is why {so far} as a designer/collaborator on big complex projects I don’t feel at all threatened by AI.
Geoffery Hinton, 2024 Nobel Prize in Physics and “Godfather of AI” on how AI started and where we are today. For me, I felt the ground shift listening to this.
Outstanding video, what a pleasure to listen to.
My son traveled to EPFL (MIT of Switzerland) last week to begin his post-doc in ACO, Algorithms Combinatorics and Optimization ( see https://aco.gatech.edu/Math/Computer Science ). He said when he got there he discovered the Math department was in an uproar. Before he left he was looking forward to coming back to an interesting job at Google. Now he is not sure what his PhD is worth.
We’ll need PhDs to interpret what the AIs are telling us. In the past, we needed natural polyglots to translate languages for us. In the future, we’ll need PhDs to translate AI output into intelligible, useful product that humans can utilize.