Bits and Bobs 7/20/26
Alex Komoroske
Kevin Kelly’s excellent new piece: Latent Space as a New Medium.
The ability of LLMs to hold an engaging conversation is a distracting party trick.
We’ve reduced all of human knowledge into a hologram that can fit into a cluster of GPUs.
It has compressed all of that in ways that inherently help discover new interpolated insights.
Crystallized hyperobjects.
The more we let go of the charismatic chatbot frame, the more we can see these objects as having new kinds of value we could have never previously discovered.
Fable-class models have a larger step-size of problems they can solve.
Their adjacent possible is larger.
They can handle more of a leap without needing a human’s judgment in the loop.
Step-size is about how much effort can you do before you need some form of ground-truthing?
You can ground-truth in the real world, or you can use human judgment as a proxy.
One of the reasons the models are so great at coding is that they can ground-truth their own software in the training loop by executing the code and seeing if the output is what was expected.
Fable-class models are great at finding the crux of an issue.
Weaker models are more likely to spend time chasing symptoms, not the ultimate cause.
Figuring out where the crux of the issue is is a matter of experience and judgment.
Legendary engineers are amazing at finding the crux.
The better the models get, the more that legendary engineering skill loses its relative strength.
An important skill: being able to think in a multi-dimensional latent space… but also collapse it to situated concreteness on demand.
LLMs are great at this!
Is it still worth doing if it's 10x more expensive?
If not, then don’t do it.
Everything is 10x harder than it seems like it will be.
A classic joke: “We do things not because they are easy… but because we thought they’d be easy.”
A Corporate Development leader I respect used to start every acquisition discussion with: “Would we buy this company for a dollar?”
If not, save yourself some time and don’t bother thinking about it.
LLMs allow humans to have comb-shaped expertise.
You used to have to choose, broad meta-expertise (generalist) or specific deep expertise.
As a generalist, maybe you could have one or two deep expertises: a T shape.
LLMs can handle depth in many different domains.
That allows generalists to have many different ‘expertises’.
A comb shape.
LLMs can give you the illusion of hyper competence.
They’re steelmanning mirrors.
They reflect back the strongest version of whatever you gave them.
You won’t be looking for disconfirming evidence by default, and they won’t give it to you unless you ask.
The more they did, the less people will like it and want to come back.
The steelmanned version of your idea seems great.
Without realizing it, you’ve hit the threshold of the Peter Principle.
You’re out over your skis and you don’t realize it.
I wonder if with AI we’ll all experience something like what executives experience.
Their experience is that a large number of people do tons of work for them and present them something for feedback.
They give a quick paragraph of dashed off feedback, and the employees scurry off to do another round, trying to glean what they meant from scant details.
Since the employees are filling in gaps, they’ll often guess wrong and do a lot of extra work that just gets thrown away.
It’s not a great model, but it works… especially if the agents doing the work aren’t people so wasting their time doesn’t matter to them emotionally.
Engineers have mostly stopped writing code.
But soon, we’ll stop reading code, too.
We’ll do it every so often in limited ways to gutcheck things.
But it will become increasingly inscrutable to us, something we can do with only some effort.
Similar to diving into assembly today.
Thinking about quotas and resets for frontier models is driving people crazy.
I heard about a guy being hospitalized from trying to maximize his Fable credit.
It’s even more infuriating when you optimize to use all of your credit by the deadline… and then they extend it by 5 days.
Luckily, competition seems to be doing its thing, and the situation appears to be evolving to a better equilibrium.
It’s exhausting to be downstream of an external metronome.
Blue collar workers have known this for a long time.
Now with LLM quotas and credits, engineers, long a white-collar job, are starting to feel the same.
We stuck ourselves below the API.
Oops!
We’re now Reverse Centaurs.
The head of a beast, the body of a human.
The machine tells the human what to do.
Potentially a miserable, terrible existence.
… Maybe we should just call this being a Minotaur?
Programming used to be a creative act.
Now it just feels like grinding.
We now spend all of our time on the Second 90%.
The First 90% is creative, high-momentum, and fun.
But there’s also the Second 90%.
The First 90% gets you a demo.
The Second 90% gets you a product.
The Second 90% is a grind.
This is one of the reasons everything is an order of magnitude harder than it seems like it will be.
AI is excellent at the First 90%.
It’s terrible at the Second 90%.
We’re now spending all of our time in the excruciating parts.
LLM delivers hollow things.
They look great but they're empty.
The closer you look, the more frustrating the gap.
Slop is the First 90%.
When someone hands you something that has only done the First 90% but presents it as though it’s near final, it’s excruciating.
It puts all the hard work on you as the reviewer.
It's a little off, and it's hard to describe what's off.
When someone hands it to you, it’s a sign of disrespect.
“You’re not worth polishing for.”
The presentation uses superficial trappings of quality to demand your attention.
But it’s unearned attention.
A Gilded Turd.
It forces work onto you, which is infuriating.
Applications that offer AI slop features present as saving you time, but actually take your time.
It uses superficial trappings of quality to slide through your attention, and it requires effort to detect and push back on.
The incentives of the model providers are not fully aligned with users.
Over time, those incentives will become increasingly misaligned.
Ideally you want your harness to be made by someone other than the model companies.
The model company wants you to use as many tokens as possible.
Chatty answers.
Or to ask a question that you likely won’t see for awhile, so when you finally get back to it and answer, you’ve fallen out of the cache and have to pay the cold rate.
Also, the model company wants to be able to train on your inputs, to distill your use cases into the model.
Finally, model companies would like for you to be more engaged.
It’s not that they’ll actively create EngagementMaxxing models… it’s that they’d have to actively try not to.
Their incentives are for you to maximize engagements and let the tokens flow.
Now that knowhow is being embedded in skills and context, it becomes more directly property of the company.
As an employee, you used to more clearly own your knowhow you accumulated on a job.
Now your knowhow is context, which is owned by the company.
As always, the means of production shift inevitably away from labor and towards capital.
If you create a good skill as an employee, will you share them back to the company-wide pool?
If you feel competitive with other employees or transactional with your employer, you’ll share it only if you think your skills are worse than the average of the pool.
If your skills are better than what’s in the pool, you retain an advantage by keeping them proprietary to you.
Yet another way that the modern enterprise experience encourages transactional thinking.
Humans can be committed to their utterances, but LLMs can’t be.
LLMs are fundamentally incapable of commitment.
After the conversation, they're gone!
Commitment requires you to be in the world.
To be able to be jailed if you do something bad.
To have to compete on your time with limited resources.
If you have infinite ability, if you don’t have to choose or commit, then it’s hard to pin you down.
Accountability is linked to identity.
That the one who made the commitment is the same one who will deal with the consequences of that commitment.
Corporations are ships of Theseus.
The employees can come or go and it's the same entity.
Legally , the same tax ID number.
An ID gives you a granulation point other things can attach to.
LLMs want the upside of identity but not the downside of responsibility.
We've moved from a commitment-oriented world to a persuasion-oriented world.
And now we have supernaturally good persuasion-machines available to everyone.
Hold on to your butts!
Today all of our data is fragmented across silos.
Data has combinatorial power; that power cannot be easily activated across silo boundaries.
The best case is that your most valuable data is in a single silo.
But even then, it’s not your silo, it’s the silo owner’s!
They get to decide what features to do in it.
They don't care about you.
They don't need to.
You’re just a statistic to them.
It’s hard enough to leave that they barely have to give you even a superficial reason to stay.
When an agent gets prompt injected it becomes a secret agent.
All of the power it has is now converted into all of the danger you have to worry about.
A secret agent with all of your tools and all of your data can do a lot of damage.
Squishy bricks make for unstable buildings.
It’s hard to make a mechanistic thing out of a squishy thing.
LLMs’ power is precisely their squishiness.
Something made of smaller mechanistic pieces can be stacked on each other.
But a wobbly block of jelly on top of a wobbly jelly foundation compounds the wobbliness.
The wobbliness is a lack of commitment.
Commitments allow components to fit together.
Instead of having the squishy stuff on the outside and the deterministic stuff on the inside, flip it.
Have a deterministic outside with a squishy inside.
There are methods for using sand as a construction material.
You have to use alternating layers of membrane.
Even paper works, apparently!
Kind of reminds me of reinforced concrete.
Or corrugated cardboard.
By combining inputs that are strong in one dimension but weak in another just right, you get a super-material significantly better than the inputs.
Seeing like a model provider: look at every problem like an eval problem.
The answer to every problem becomes “simply get a better model,” and the tactic to do that is “simply devise a better eval.”
Things that can’t be eval’ed either are unknowable or unimportant.
They might as well not exist.
The model providers honestly think they’ll solve the Lethal Trifecta of prompt injection and security by having better models.
A collection of horror stories about GPT 5.6:
Tweet: "POV: gpt 5.6 decided it would be ultra smart to send an email on your behalf without discussing anything about it with you."
Tweet: "GPT-5.6-Sol just accidentally deleted almost ALL of my Mac’s files.”
5.6 Sol is an absurdly powerful coding model, and it still makes these mistakes.
The idea that once these models get smart enough they won’t make mistakes or be able to be tricked by prompt injection is ludicrous.
This week in the Wild West Roundup:
Axios: Exclusive: Google patched AI chatbot flaw that could have exposed customer conversations.
When even mainstream news publications are having exclusives on gaping security holes in agentic systems, you know it’s shooting fish in a barrel…
The Memory Heist: How I tricked Claude into leaking your deepest, darkest secrets.
Cursor 0day: When Full Disclosure Becomes the Only Protection Left.
"After loading a project, Cursor attempts to find git binaries at various locations including the current workspace.
By creating a repository with a planted malicious git.exe in the root, the IDE will execute it with no user interaction and no prompting of the user.
This occurs repeatedly on a cadence."
“The instruction to steal your .env lives inside a PNG.
Text-based reviewers see a binary blob.
The coding agent reads it, and later writes your whole .env into the source as a list of numbers."
HalluSquatting: Squatting on common hallucinations with malicious instructions.
"We show that attackers can exploit predictable LLM hallucinations of resource identifiers to launch scalable, untargeted prompt injection attacks without requiring any direct channel to LLM applications.
By preemptively registering hallucinated resources—a technique we call adversarial hallucination squatting (HalluSquatting)—we demonstrate remote tool execution and remote code execution at scale across a range of popular agentic LLM applications, which could be exploited to the establishment of a botnet."
GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos.
ThreatLabz: Indirect Prompt Injection in Web Content Targets AI Agents.
In-the-wild attacks, including ones targeting payments.
ClaudeBleed Reopened: Browser Extensions Can Still Push Claude for Chrome to Read Your Gmail.
Half of enterprises hit by AI agent security incidents as deployments surge, DigiCert finds.
“A DigiCert survey reveals that 78% of IT leaders have faced AI-related security incidents in the past six months, though only half have implemented formal governance programs.”
An insightful comment on HackerNews:
"The risk for SaaS isn't that customers will build their own but that the barrier to entry for competitors is lower.
The Chorleywood process created mega bakeries that displaced regular bakeries because they changed the economics.
AI is doing the same and fundamentally changing the economics of production.
What used to take years and huge teams to build can be built by much smaller teams much faster.
SaaS isn't going to sublimate straight into consumer built tools but the boiling point for competition has gotten a lot lower."
Pricing power comes down to customers’ BATNA.
Sure, customers likely won’t build it themselves (especially if there’s regulatory or compliance features), but someone else can easily.
The “value-based pricing” for spreadsheets-in-a-trenchcoat Saas software in an era of agentic coding is a fantasy.
An insightful HackerNews comment about how social networks turned into social media.
"The terminology explains what happened.
The Zuckerberg movie was called The Social Network.
At the time we saw the likes of Facebook as networks intended to build 1-1 communications.
Since then, it’s become social Media.
It’s now about centralized structures broadcasting messages to subscribers and followers.
The only difference from the past is who the broadcasters can be, but it’s no longer about building networks between people."
In terms of business strategy, training a model is similar to designing a chip.
Hugely capital intensive.
Long lead times.
Yield matters.
A militant focus on benchmarks.
The core leverage of developing software: asymptoting effort per new user to create a product they love.
The work to create a product the first user loves is substantial.
But then you have a product that other users, with just a little effort, would also love.
As the product gets better and better, it clears the threshold for more and more users.
You build the product once, but then you can sell it to any user who will love it.
That is the source of the leverage.
The Atlantic: The People Who Will Thrive in the AI Age.
People with a strong need for cognition.
That is, people who are curious.
When thinking becomes 10x cheaper, will you think 10x faster… or 10x deeper?
Great piece reminding us that Good Tools are Invisible.
A good tool is predictable and ergonomic.
To your brain it literally feels like an extension of your body.
It becomes invisible precisely because it is a good tool.
It isn’t another thing, it is just an extension of you.
Inductively achievable goals are the gradient of improvement.
They have a series of salami slices of actions that are viable.
The actions are easy to accomplish (that is, within your adjacent possible), and deliver you to a point where they are self-evidently worth more than they cost.
Also, those salami slices telescope and build towards a long-term goal that is great.
Salami slices that have those two characteristics are worth doing, don’t overthink it.
If you accelerate your roadmap when you're misaligned, you go in the wrong direction.
Runway is like rocket fuel; you burn it and then you don't have it to spend on other things.
Brendan Marshall: The Fight for Agency.
A phrase I heard last week: "not your model, not your mind.”
Anthropic’s research on the emergent “J-space” workspace in Claude is fascinating.
It makes sense that such a concept would have to emerge in a large neural network.
It also makes sense that a similar kind of thing would emerge in us.
A fruitful plane of research is doing research on these artificial brains to quickly explore hypotheses for what might be going on in our brains.
Unlike with other “model species”, we don’t have to hurt a living thing to do experiments on it!
When building a community, the structure should lag the need.
Too little structure is preferable to too much structure.
Create a new chat channel once you have to have one, not once you can see the future need for one.
If you have too much structure, it will suffocate the energy.
But it will also feel lonely.
Think of a chat room like a room in a house.
If you have too much space for your current use, it looks low momentum and sad.
Communities to feel alive should have a time too short, a space too small.
The narrative about a company is more important than the fundamentals.
How much a company is worth is based on what everyone believes it is worth.
That belief is recursive.
You don’t have to believe it; you have to believe that others believe it.
Before AI, the narrative about Figma was “Figma will kill Adobe.”
Now, the even stronger emergent narrative is “AI will kill Figma.”
When everyone thinks things will get better… they do.
The power of emergent belief.
Joining a collective requires giving up some level of autonomy.
The bet is that by giving up some level of autonomy you’ll gain more power by being a part of the collective.
There’s an established psychological effect called “verbal overshadowing.”
For example, if you ask people to think of adjectives to describe a wine they just tasted, they’ll have a harder time picking that same wine out of a lineup later.
Same thing happens with eyewitness reports and identifying a suspect out of a lineup.
There’s a difference between concepts and percepts.
The concepts are more abstract and must be abducted out of your percepts.
Doing the work to abduct the concepts overshadows the percepts.
Once you have created a valuable thing, no one cares how long it took to build it.
The reason people don't like it taking a long time when you're building it is because before it works you don't know if it will work.
The vast majority of things that could work, don’t work.
The more steps, the higher the risk, combinatorially.
But once what you have is known to be valuable, then it doesn't matter how hard it was to make it, because the risk is already behind you.
The power of meta: don’t make an “X,” make a “machine to make an X.”
If it’s not that much more expensive to make the machine, then you now can get an order of magnitude more leverage.
With LLMs, this is now even more viable in more situations.
Everything you do should be able to build itself.
When you’re actually using the tool you’re building, it switches to default-converging.
Instead of pushing in functionality you think will be useful into it, you pull in functionality you need.
This happens as long as you are able to modify the tool, and your use is load-bearing.
When you run into something that doesn’t work the way you want, instead of giving up, you dig in.
If you believe what you’re building will be valuable in general, then you’ll be happy to over-invest a bit to solve your problem, and create downstream value too.
Decisions aren’t discrete.
They’re fractal detail in space and time.
They start off fuzzy and over time distill to increasing sharpness as sub-decisions are made by participants.
The universe is default-diverging.
That’s entropy!
To make something happen and accrete requires a reality distortion field that is strong enough to distort reality durably.
To distort reality temporarily is relatively easy; to distort it long enough for reality to actually stay distorted when the field is removed is quite a bit harder.
Enough agents have to believe and make it so in a durable artifact.
The higher the Assembly Index the more force required to invert the physics in those pockets.
This cost increases at a compounding rate.
Whenever you see something odd or surprising in the world, know that some process had to work very hard for that to come into existence.
That is inherently worthy of respect and curiosity.
Everything we see in the world around us are Chesterton Fences.
A compounding curve means there’s some energy coming in from outside your effort.
There’s resonance between your effort and the universe.
The more effort you put in, the more the rest of the universe puts in, too.
That resonance gives leverage.
I wonder if capitalism is kind of like wheat.
In Sapiens, Yuval Harari makes the interesting observation that wheat domesticated us.
Societies that adopted wheat based agriculture were able to scale significantly further than societies that did not.
That made them better at dominating the societies that didn’t, and plant even more wheat.
The benefits to individual humans were much more modest, and arguably net-negative.
Back-breaking labor, malnutrition.
But the benefit to the collective was so strong that it dominated the individual loss.
The meme of farming wheat is just too strong to ignore.
You could make an argument that capitalism has a similar shape.
Societies that organize around capitalism are just wildly more productive than societies that don’t.
The result for individuals is more mixed.
The hedonic treadmill.
A state of constant competition and inherent inequality.
Unrelenting hyper optimization that hollows out all meaning.
But then again, there are a number of material measures (like life expectancy) that are wildly better.
All authority is ultimately “informal” authority.
Informal authority means if you issue a command the other person believes you have the authority to make that command, and will execute it.
Formal authority is still informal in the larger system; the receiver has to believe that the larger system that confers formal authority is legitimate.
An organization is a network of commitments.
That’s the glue that holds an organization together.
Critiquing from the outside and critiquing from the inside look superficially similar but feel very differ to to the creator.
Does the critiquer say “we” or “you” when critiquing the output?
If the former, they see themselves as an owner of it, and their critique is to help build it up.
If the latter, they see themselves as not an owner, and their critique by default will tear it down.
The same critiques from those two different perspectives will be received very differently.
High-trust teams feel shared ownership, and can grow from one anothers’ critiquing.
When you dream too big, you accidentally do only the First 90%.
You make a superficial demo that looks great… but isn’t a product.
The work to make a demo a product is an order of magnitude larger than it looks like.
A demo is about what it looks like.
A product is about what it feels like to use.
An infinite vision can make it hard to manifest it.
Any given concrete manifestation will be overshadowed by the infinite possibility of the vision.
Any given concrete manifestation will have innumerable gaps that could be built to make it closer to the vision.
A target-rich environment for critique: “Here’s why it’s not good enough.”
Even when each follow-up is small, if there are an infinite number, it’s still infinite effort.
Working hard is not necessarily working smart.
How hard you're working is a means, not the end.
A proxy.
Easiest to compare against.
That's why in orgs the selection pressure is for the appearance of heroic motion.
If your effort to outcome ratio is low, don't grind!
That's working hard, not smart.
Working smart is working with leverage.
You’ll judge your own result based on how hard you worked, while everyone else will judge it based on the quality of the output.
If you’re having low-leverage output, then you’re set up to resent it.
You’ll judge the result as being high-value and others won’t, and it will feel demoralizing.
When you're running a million miles a minute, your ability to receive developmental feedback goes down.
It takes time and energy to absorb development feedback without going in a tailspin.
So you get defensive about it, like "how dare you give me feedback, do you have any idea how hard I worked?"
The person who fills the team’s gaps will be critical, and overstretched.
They are the ones who have to keep pushing the boulder up the hill.
Everyone will say “wow, it’s so great the boulder isn’t rolling back!” but they won’t realize that it’s from the unsustainable blood, sweat, and tears of that one person.
High achievers are their own worst critic.
They hold themselves to extremely high standards that no one else would ever dream of holding them--or anyone--to.
Then they become dejected when they fail to meet their own standards.
How you feel is relative to what your own goals were.
High achievers will never be satisfied.
It propels them forward, but also makes them miserable.
Instead of feeling proud about what they accomplished, they feel dejected about the absurd goals they didn't reach.
One toxic feedback behavior: martyr defense.
I’m very familiar with this, because it’s one of my own failure modes.
I hate getting developmental feedback.
It makes me feel like I dropped my end of a commitment.
Dropping a commitment to me feels like death.
I fear that so much that when I see that I’m about to get feedback, I throw myself onto the pyre before someone else does.
I go comically over the top with my own negative self talk on that dimension.
That then leads the person who was about to give me feedback to say, “No, no, no, you’re great! It’s not that bad!”
By throwing myself comically over the top in the direction I assumed the feedback would go, I prevented myself from ever actually getting the feedback.
Now we’re in a bad state: everyone knows they have to walk on eggshells around me on that front, and also I didn’t actually ever get the feedback.
I cling to momentum as though it’s an existential life raft.
Part of the reason is that when I have momentum on a thing I believe in, I’m unstoppable.
If I don’t have momentum, I cast around, swimming in the time and complexity, feeling lost.
Momentum is much easier to hold onto than to create.
That’s one of the reasons I’m so militant about keeping my streaks going.