Bits and Bobs 7/14/25
Alex Komoroske
In it I talk about the imperative for Intentional Tech in the age of AI, and the power of a coactive fabric to unlock the potential of AI.
I want adaptive apps in the age of AI.
Let’s melt apps into liquid software.
This week someone demoed a notes app that builds itself.
“The app will bend to your will.”
Self-designing UIs might give us software like the Homer-Simpson designed car.
Being a PM is hard!
Thinking through the implications of features in your software.
It takes taste, judgement, experience, savviness.
In a world of infinite software, the data is more valuable than the app.
An observation this week: "I can see a future where it's much faster for all employees to use an IDE with .md files + git instead of Notion, Google Docs or Office."
Imagine a coactive common fabric of meaning.
You can pin things from across the fabric into your own pocket of the fabric.
You can create patterns that describe how to imbue the fabric with emergent capability.
The patterns can be shared intentionally or implicitly.
The fabric comes alive with possibility, powered by collective intention and intelligence.
The web as a medium is loose.
A fabric would be more tightly woven.
Apps are not your system.
They're someone else's system that you're just inside of.
The owner of the origin, not the user, calls the shots.
This essay makes the case that products are jamming AI down our throats.
I think this is actually the tyranny of the same origin model, now being applied to AI features.
It's not AI, it's that the origin gets to decide what features to add, and those are what's good for the origin, not what's good for the user.
Important observation: "AI context is domain-specific and often non-transferable."
Excellent piece from my friend Rohit about Seeing Like an LLM
Contains this definition from Tyler Cowen: “Context is that which is scarce.”
Imagine Borges’ library, but for software.
Infinite software.
Software for every need, just waiting to be discovered.
I love this piece in Cosmos on “social tinkering”.
Gets at the distinction between tinkering and vibe-coding.
It's about how active the stance is.
Are you pair programming with the LLM or outsourcing your thinking to it?
Collaborative curiosity.
This distinction also gets at why Bret Victor doesn’t like LLMs: it moves people to an outsourced thinking position.
The coactive fabric is closer to a spreadsheet that comes alive than an app you can tweak.
Malleable apps are still "your data is stuck in it but at least you can change it."
But spreadsheets are "your data, but you can build on top of it.”
It’s just that spreadsheets are convex to collaboration.
The more a spreadsheet develops, the more it becomes useful to its creator and inscrutable to everyone else.
How can we flip it to be concave to collaboration?
A friend created a kind of software he calls voidware.
Think of what WebSim was at the beginning.
A software experience that hallucinates everything on demand.
It looks like normal software superficially, but underneath it’s 100% LLM.
It seems like that kind of software might be too squishy.
Like jello.
Impossible to pin to the wall and get it to do structured things consistently.
Still, a fascinating experiment in the era of infinite software!
This article on on-the-fly toolgen was interesting.
But I don’t think it goes far enough.
It still has the LLM at the root of the loop, calling the shots, deciding what to rely on.
But any system with an LLM in the driver's seat is prone to prompt injection.
Why not have codegenned code be the root of the loop?
Vibe coding and MCP show the power of infinite software and integration.
But they have low ceilings due to their security model and requirement of very high volition by users.
What if you could remove that ceiling?
The power of AI with MCP and vibe coding, but for everyone.
A number of systems have strong momentum out of the gate, but sprint towards a dead end.
This is the source of the “logarithmic value / exponential cost” curve I’ve talked about in the past.
Strength of momentum and height of ceiling are disjoint.
MCP has strong momentum but a low ceiling.
If your system assumes a high level of trust but there isn't actually that trust, you can't do anything.
The system seizes up, frozen, constrained in a straight jacket.
MCP assumes high levels of trust that are not warranted.
Fine for savvy users tinkering on the command line, a non-starter for everyone else.
It takes humans to distill, research, and vouch for information.
Humans did that for the web (deciding what to link to) and writing (deciding what to bother writing in the first place).
What to link to and what to write was aligned with what the creator found valuable.
Naturally authentic.
That gave a consistent bias amongst the noise for what humans find valuable.
LLMs then absorbed it through osmosis in their training.
LLMs can’t vouch for new things in an authentic way.
When human publishing slows down, how will we synthesize new knowledge?
LLMs need checkpoints of knowledge (context) that is based on human judgment.
Context is a stepping stone that gives you leverage.
Good context is great.
Bad context is terrible.
When LLMs do it themselves they do middling or even bad context and that spirals.
That's why things like Claude.md are not a hack.
They are the human making higher level contextual assertions.
Claude leaving Claude.md notes to itself help improve quality in your codebase is an example of a coactive surface.
The human can then co-create that context, editing, curating, adding.
How can we generalize that interaction?
Your context is not your data in one big payload dump.
It can be derived from it.
But it is, as the name implies, contextual.
Chatbots want to answer your question in one pass.
There are many questions that can’t be answered that way!
An Etsy for vibecoding is hard to imagine.
Vibe-coding makes it 10x easier to code.
But it’s still orders of magnitude too hard to code for most of the population.
You could imagine software as a cottage industry.
Order up some software and have a vibecoder build it.
Artisanal software.
But that doesn’t work due to the security model.
You’d have to trust this anonymous person with your sensitive data.
Whereas it’s much harder to make a tea koozie that looks well made but has a dangerous flaw.
Here’s a walkthrough of a scenario where it’s easy to expose secrets in MCP with Supabase.
I found the HackerNews thread interesting.
The Supabase team was discussing the mitigations they were taking to make MCP infiltrations less likely, including by automatically extending prompts with stronger instructions.
Many of the comments were pointing out that it doesn’t make sense to treat MCP as the security boundary.
Finally people are starting to wake up to the LLM not being able to be used as a security boundary.
An example of a vibe-coded tool that had dangerous vulnerabilities.
Vibecoding only makes sense if you’re doing it for yourself or a handful of people you know personally.
Otherwise you can do some damage if users are operating under the assumption that, like before, the code was written by an expert and accidental leakages are unlikely.
The same origin security paradigm puts a ton of power in the hands of the origin owner.
With great power comes great responsibility.
When software was expensive to do and only experts working carefully could afford to do it, it was a reasonable assumption.
The same origin paradigm is a dangerous liability in the era of vibe-coded infinite software.
The problem wasn’t prompt injection per se, it was just a poorly configured and secured system.
Still, I imagine we’ll see a lot of these kinds of things with companies eager to integrate AI into their publicly-exposed systems.
We're seeing an explosion of AI browsers.
Chromium is easy to fork.
Context is critical.
The cookiejar is the most obvious pot of rich context today.
The tabs you have open, the bookmarks you have as your context.
A fracking approach to context.
However, browser distribution is extremely expensive.
The only player that could plausibly have the juice to do it in the last decade is OpenAI.
OpenAI will try to redefine the browser category.
Just blow it up.
Make the web page rendering be the secondary function, not the primary.
On the path to aggregating everything, OpenAI will at some point allow users to choose competitor’s models in the UI.
They'll say "you can use any model! Look, it's open!"
But no one changes the defaults, of course.
"You could... but you aren't gonna."
Then the model is beholden to the aggregated UX and consumer demand of the aggregator.
It's crazy that Cursor, a VSCode fork that was created even after Github Copilot existed, is now worth more than the IDE companies.
But it turns out that AI is the feature that is so important that everything else is secondary.
An AI UX that is better than what an extension can do is worth it.
It's not an IDE value prop. It's a totally new pacelayer on top.
The ability of an LLM to write good code goes down super-linearly with the number of lines of code.
This is due to assembly theory; by construction the number of possible combinations scales super-linearly..
As you scale up the number of lines, the likelihood those same conceptual lines have been encountered before in that combination goes down at a super-linear rate.
So the question is: to make resilient systems composed of infinite software, how can you make it so each chunk of code is as small as possible?
We’re all vibe-coding micro-apps because those are the default structure from the era of finite software.
But they’re too chonky for LLMs to handle well.
The right substrate for infinite software will allow software to be tiny little drops.
"User" is what dealers call their customers.
"Neighbor" is what communities call their members.
Prosocial tech builds communities.
Antisocial tech builds dependencies.
Prosocial tech engages us with what matters.
Antisocial disengages us from what matters.
Prosocial tech strengthens the social fabric: the invisible bonds between us.
Antisocial tech harvests those bonds for engagement metrics.
Antisocial tech tries to collapse contexts.
Context collapse flattens the world.
Makes it efficient but inhuman.
A zombie.
We are fractal.
We contain multitudes!
We are different people in different contexts.
Prosocial tech must grapple with that fact.
Prosocial is not just about tech.
It's about the whole modern societal context.
We live in the Antisocial Age.
Prosocial is about becoming the version of yourself you want to be, engaged with the community that surrounds you.
A lot of technical systems feel inhuman.
Prosocial systems bloom with emergent collective intelligence.
There is intrinsic value in knowing other people were involved to create the value; and that your actions improve the system for the collective, too.
At some point that is actually more important than a marginal increase in quality that might come from the collective intelligence.
I don’t want utopia, I want protopia.
Utopias are static end-states where all problems have been solved.
They’re impossible.
Protopia is a term coined by Kevin Kelly.
Rather than perfection, protopia describes a state of continuous, incremental progress.
In a protopia, things get a little better each day through small improvements, but problems still exist and new challenges emerge alongside solutions.
Crypto promised us agency,ownership, and a better digital community.
But because of the hyper financialization and legibility baked in, it was like Goodhart’s law on hyperdrive.
The humanity was driven out of the system.
Web3 was, accidentally, even more antisocial than Web2 ended up being.
The modern industry selects for quickness of reasoning, not depth.
The fastest one-ply thinkers.
That's what gives us Goodhart's law at the level of society; everyone thinking a single ply and not thinking at all about the implications of their actions.
One-ply thinkers who are handsomely rewarded might not even realize there’s anything at all they’re missing.
The role that is the closest to Context Engineering is perhaps PMing?
You need to be able to think systematically, but also situated, about what people really want.
Some insights from my friend Stefano on alignment.
“Alignment is defined as reducing the misalignment between the value system of the LLM and the value system of who pays to make it / keep it running.
Either there is a single absolute value system, a sort of platonic eigenvector of goodness, or, there isn't.
If there isn't, then alignment for me is propaganda for you.
ust looking at the history of political parties and newspaper editorial rooms in history it suggests that "eigenvector alignment" is a non-starter.
So we're left with just another "what's good for me is evil for you", as predicted.”
Mike Masnick points out the Grok MechaHitler episode shows another danger of centralized control of models.
As a friend observed, “It turns out we should have been more worried about the alignment of the leaders of the AI labs than about the alignment of the models.“
Socrates critiqued writing because you couldn’t talk with a book.
But now, with LLMs, you can.
It only took us a couple thousand years.
The same origin policy is what dooms us to aggregation, overwhelming permission prompts, and chilling effects on use cases that are too creepy… or all three.
But privacy does not have to lead to all of those poor mitigations.
It's the same origin's fault, not privacy!
Privacy in the same origin paradigm comes from data doing less.
This negates data’s potential energy.
The problem is not privacy, it’s the same origin policy.
A rule of thumb I heard this week: TEEs (Confidential Compute) give a ~10% overhead on computation.
ZKProofs, they told me, have 10,000% overhead.
They both give you the ability to trust remote computation.
As data flows, it loses its integrity.
Where did it come from? Has it been tampered with?
What is the context about it?
If you forget about your units (meters or feet?) in a codebase, you could blow up the rocket.
Code that touches a session token should become radioactive.
In a safe system, code that touches something so sensitive should isolate itself.
In today’s code, that isolation has to be done by construction, by security experts at design time.
To have a system capable of working with infinite software, it will have to be antifragile; auto-isolating code that has been tainted by sensitive data.
Differential privacy techniques inject structured noise into data streams.
That makes any given observation swamped by the noise and thus not identifying, but allows the overall true shape of the distribution to be visible at scale.
This exploits the “consistent bias at enough scale can emerge even from noisy streams” phenomena that powers all emergent phenomena–including evolutionary gradients.
Resonant products are products that people like using and also feel good about using.
Very easy to have one or the other, much harder to have both.
The former is the primary use case.
The latter is the secondary use case, the bonus.
You can't sell a product to someone based on a secondary use case.
But you can make a product that sticks with people because of the secondary use case.
The same origin model means that features that have no primary demand but tons of secondary demand can’t exist.
Consider Ground News, the product that gives a sampling of news by bias.
Everyone says they want such a feature but not many people actually use it.
Everyone would prefer to see bias indicators in their news site, but any news site where that’s the primary use case can’t get enough demand to support the business model.
It’s kind of like a Simpson’s Paradox of app features.
Origins are organized around primary use cases.
Secondary use cases can emerge only if there is enough clear consumer demand for that feature.
But it's always overshadowed by the primary use case.
Every PM says "no one asked for that."
"Want to want" can’t be optimized by one origin because it’s secondary.
Combinatorial phenomena can't be contained within linear walls.
If the thing you think is obvious doesn’t exist, what should you make of it?
If it doesn’t exist, is it because no one else thought of it?
Or because everyone who tried it failed, or found it not worth the cost.
Novelty has this problem.
You need to convince yourself it's something that everyone else missed, otherwise it's likely just not viable.
Great and good are different.
Greatness is about the ceiling.
Goodness is about the floor.
If it's already great, being good as well is useful.
Hallucination driven development is now a thing.
See what the LLM expects to be there and do it.
Infill product development.
Do the obvious thing that's missing.
It's about gap-filling.
Doesn't make your thing great, but it can make it good.
This week I learned from a YouTube video about what limited the biggest land animals’ size.
I would have expected it to be a structural limitation.
Turns out their body plan could have gone even bigger… but the carnivores’ body plan couldn’t go any bigger, so there was no pressure for the herbivores to get even bigger.
This week I learned about the “approach plate” for landing a plane based on instruments.
The precise approach plate is extremely finicky, and it’s very easy to get wrong.
Garmin apparently has a product that will tweak the approach plate for landing your particular plane in these particular conditions.
This gives a sense for the value of context.
If you're flying based on instruments (e.g. auto-curated context), then it's imperative you trust that the curator didn't make a mistake.
If they think you're a 747 but you're a Cessna, you're going to crash.
The distiller could do a bad job distilling the wrong things, or think you have different goals, desires, or context than you actually do.
Order of magnitude thinking is good for physics and strategy.
But it's not good for execution.
People who prefer order of magnitude analysis are terrible estimators for real world processes.
It's possible to be rigidly flexible.
If you only like nudges, flexible, open, option value, you end up confusing people, creating eddy currents.
Some situations require laminar flow for things to cohere.
A trustless society is impossible and undesirable.
Trust is an emergent social phenomena.
It’s a lubricant that makes everything work.
What you want is scaled trust.
Some infrastructure levers trust, and some destroys trust.
Infrastructure that helps your trust go further helps society thrive.
The printing press came, so Martin Luther could emerge.
The LLM has come.
When will our Martin Luther emerge?
The barbell effect, the smiling curve, and the r/k-selection dichotomy show up for the same reason.
There are two stable strategies at the extreme.
1) A small number of very high quality but expensive options.
2) A massive number of low quality but cheap options.
Everything in between slides down the slope towards one of the extremes.
R-selected systems are open-ended.
K-selected systems are close-ended.
The benefits of the two types of systems are powerful, and more powerful when in juxtaposition to the other.
That’s why the world tends to strirate in interleaving layers of r- and k-selected systems.
As one side tends towards r-selected, the neighboring region gets a differentially strong pull towards k-selected.
The world pushes from mush to posterized, fractal, swirling interleaving of open and closed systems.
The asymmetry of bullshit has become significantly more important.
There’s many more ways to be wrong than right.
If you care about being right you have to invest much more effort per statement.
If you don’t care about being right you can overwhelm those who do.
R-selected vs k-selected.
The result is that we are awash in a tsunami of bullshit.
This was a minor annoyance but now that we’re in the Cacophonous Age it’s the driving asymmetry of our time.
The disruptiveness of “work to rule” strikes implies that automation by LLMs could have a ceiling of effectiveness in complex domains.
One way to think about why Goodhart's law happens is because before incentivizing, you had an (implicit) causal model for why the metric drove the outcome.
But by incentivizing it you modify the dynamics and break the causal model.
Since the causal model is implied, you don’t even think about what you’re doing as you smash it to smithereens.
Organizations would often rather have order than truth.
Alignment over competence.
This emerges fundamentally from the emergent imperative to "act like your boss is right".
If you don’t abide by this rule, you have a structurally larger chance of being knocked out of the game.
A consistent bias that emerges clear as day if you average all of the noisy examples together.
Systems choose metrics that make them look good.
The choice of metrics is endogenous to the system.
If every update to the manager the team chooses different metrics to highlight, even if they’re "doing well as reported by the metrics" they’re actually doing poorly.
Consistent progress on consistent metrics shows that things are going well.
Or at least, momentum on consistent metrics are harder to fake.
At a certain point however the metric decoheres from success and becomes an accidental end in and of itself.
At the beginning you want a consistent metric; at the end that's dangerous.
Top down control of a complex open ended product has to be done by things like enumerated Critical User Journeys.
But CUJs are linear; it’s impossible to capture the combinatorial feature set of the system.
It’s like a random subset of spaghetti in the bundle of pasta.
All of the paths that aren’t covered by CUJs rot.
This can easily lead to most of the product rotting while the part the product owners measure seems to be doing better than ever.
An emergent system can easily be constructed where caretakers who intrinsically care and are given space to follow their gut gives great results
The only downside is you can’t prove it will work, or structure its output.
I was intrigued by this old essay that framed ads as cancer.
A provocative but generative frame.
Ads, it posits, are a runaway steamroller that feeds on the host but will eventually kill it.
Goodhart’s law: the swarm will inescapably cheat and kill the collective.
The swarm and the collective must be kept in harmonious balance.
If you allow the swarm to swarm unimpeded it devours the soul of the system.
The spirit of the law is very different from the letter of the law.
The spirit is about an infinite, a prosocial stance, trust, a positive sum, an ends.
A focus on the letter of the law is about means, "if it's legal it's allowed"
For example, consider someone being frustrated that a billionaire pays few taxes due to a loophole, and the billionaire being upset that anyone is mad they are following their incentives within the rules that were given.
From the perspective of the tax payer: “whatever is legal is allowed.”
From the collective perspective “whatever is not aligned with the intention is immoral.”
These two perspectives are distinct!
It’s impossible for them to be fully aligned, for the same reason it’s impossible for the incentives of the collective and the individual to ever be perfectly aligned.
Schelling points help counteract the natural diffusion of open ended systems.
They give a point for high-quality things to congeal and emerge.
Folksonomies work because their UX allows the potential for schelling points to emerge.
Concave systems are auto-cohering
Convex systems are auto-decohering.
An infinite difference.
The difference cannot be seen from a static snapshot.
It can only be seen from the system in motion.
Even then you have to carefully study the flow patterns of micro movements.
The small biases and asymmetries, beneath the distraction of the noise.
It’s easy to slide down slippery slopes and become the baddies.
I was reminded of this by this essay.
On a slippery slope, each step is fine, but suddenly you look up and go, "wait...".
By that time your ego is so tied to it it's hard to admit it, that you followed your incentives to a place you would have never gone intentionally.
No one thinks they’re the baddies.
If they realize they’ve done bad things, they can see how the incentives created a situation where they were channeled into it emergently.
You know you're not the baddie.
You might be acting bad, but you aren't bad.
But when we look at others we can see clearly how they are the baddies; intrinsically bad.
A general trick to make sure you're not accidentally acting like the baddie.
Imagine if the baddie did what you are about to do and see how you’d feel about it if they did it.
If you feel bad about it, don’t do it!
This helps invert the fundamental attribution error and help you understand the emergent effects of your actions–especially the bad ones.
The more aspirational and driven you are, the more likely you don't live near your family.
We move to where the economy wants us, undermining family and community.
Despite all of the noise in the real world of specific examples, there’s a consistent bias.
A structural difference.
What effect might this have at scale in society?
Evolution is a slippery slope to give us what we want, not what we want to want.
Capitalism is an outgrowth of evolution, in the substrate of the Technium.
Immigration is the engine of the United State’s continuing advantage in innovation.
A small consistent bias over noisy data of high gumption people who decide to emigrate.
Emigrating is a hard thing to break out of the status quo, it requires volition.
People know that if they come here and succeed they can be accepted as truly American, not American with an asterisk.
It’s the best place for high potential people to try their hand at striving for the top.
That asymmetry gives a massive advantage to the US.
We’re currently breaking that engine.
It will be years before it’s obvious what we’ve lost.
Harvard Business Review: What Gets Measured AI Will Automate.
Goodhart’s law on steroids.
Paperclip maxing ourselves to death.
Heaven help us.
You can't fight incentives with culture.
Over long enough time scales, culture is downstream of incentives.
Incentives are downstream of business model.
What’s better than being famous is being niche famous.
When in the niche, you get the benefits of being famous.
But you can step outside of the niche and get rid of the downsides of being famous.
For a small enough niche, everyone can be niche famous.
People who are intelligent but poor communicators will attribute failures to others' incompetence that are actually root caused by their own poor communication.
The entity with more power should be held to higher standards.
Power is often invisible but extremely important.
So the more powerful will say “why do I get increased standards? It’s unfair!”
Because the power is invisible to them ("that's just the way it is, people respect me") but the constraint is visible to them, it feels unfair, even thought it’s not.
“You broke it, you buy it”.
If you broke it, you’re responsible for it.
You own it even if you don’t want to.
If you vouch for something you are responsible at least in part for its quality.
Gatekeepers in a security model are structurally required to vouch for things they distribute.
That leads to them being more and more careful at an accelerating rate.
The safe subversive style could also be described as puckish.
Subversive and understated but worldly and clever.
No individual subset of it is dangerous, it’s only between the lines you can see how much it questions the status quo.
Meaning comes from cost.
The things you choose to do despite being costly, that are “irrational.’
They're infinite, they're emergently meaningful.
Infinite things’ infinite characteristic comes from emergent components.
Emergent components can’t be seen concretely; if you keep cutting them down into smaller and smaller pieces, the magic evaporates and you’ll never see it.
If you don't believe in emergence you can't believe in infinite things.
An end is more than the sum of its parts. Infinite.
A means is just the sum of its parts. Finite.