Understanding Isn’t Just Output, It’s Explanation

[Tuncay Başkan]

ChatGPT çıktıktan ve LLM'ler dünyanın en çok konuşulan şeyi haline geldikten sonra çok iyi bildiğimiz bazı kavramları aslında hiç de iyi bilmediğimiz ortaya çıktı. Öte yandan bazı kelimelerin ne anlama geldiği konusunda pek bir fikir birliğimizin olmadığı sonucuna da vardık.

"Anlam", "muhakeme", "yaratıcılık", "zeka"... Aslında bu kelimeler ve kavramlar uzun zamandır felsefenin gündemindeydi ama LLM'lerin ortaya çıkmasıyla herkesin gündemi oldu.

Şuradan görülebilecek bir sohbette Ilya Sutskever "LLM sadece next token prediction engine değildir, next token üretebilmek için bağlamı 'anlamış' olmak gerekir" diyor. Yorumlardan gördüğüm Faruk Güney isimli arkadaş bu konu üzerine beğendiğim bir yazı yazmış;

https://x.com/farguney/status/1925674530350993895

 

Kaybolmasın diyerek buraya da kopyalıyorum;

Understanding Isn’t Just Output, It’s Explanation

I don’t dispute that neural networks work. They do. They’re powerful, efficient, and often remarkably accurate. I’ve seen what they can do in classification, generation, prediction. They scale across domains, adapt to noisy environments, and outperform hand-engineered systems in ways that were unimaginable a decade ago. That’s not in question.

But what is in question and what I care about deeply is what we mean when we say these systems understand. Because from where I stand, that word, “understanding,” has to mean more than just producing consistent outputs. It has to mean something communicable, something we can explain in terms we share, something that connects to our human frameworks of intention, reasoning, and value.

That’s the line I draw: when explanation becomes incommunicable, I stop calling it understanding. I call it behavior.

Behavior Isn’t the Same as Comprehension

Let me be clear. Neural networks are deterministic. Same input, same weights, same output. No randomness, no mystery in the mechanics. In that sense, they’re traceable. Every neuron’s activation can be logged, every weight inspected, every gradient tracked. But tracing a process is not the same as grasping its meaning.

Take a self-driving car. If it crashes, we can reconstruct every signal: sensor readings, activations, final control outputs. We’ll find no ghosts in the machine only math. But if we ask, “Why did it swerve instead of brake?”, we hit a wall. There’s no principled explanation just learned patterns encoded in high-dimensional space, shaped by data, not by human judgment.

We know how it worked, but not why, in any conceptual way that would make sense to a human. That distinction matters. Because when people’s lives are on the line in healthcare, finance, justice, they deserve more than a function trace. They deserve an explanation. One they can interrogate, argue with, understand.

We Designed the Conditions, Not the Knowledge

And that brings me to authorship. When we build something like a coffee maker, we design every part. Heat the water, pump it through the grounds, fill the pot. If something goes wrong, we know where to look. The logic is intentional, transparent, and mechanical. We understand what it does, how it does it, and why it does it that way.

But a neural network isn’t like that. We don’t directly design its knowledge. We design the conditions under which knowledge emerges: the architecture, the loss function, the dataset, the optimization algorithm. From there, something forms something useful, yes, but also something alien.

That’s a different kind of authorship. We are no longer programming outcomes; we’re setting constraints and watching behavior evolve. It’s powerful, but it also distances us from the internal logic of the systems we create. We get results, but we don’t always get reasons.

And that gap between behavior and reason, between traceability and understanding is where I think we need to stay alert.

Reproducibility Isn’t Trust, It’s Caution

If you want to say that a model “understands” because it consistently produces the right answer, fine. I get that. But that’s not the kind of understanding that earns trust. It’s the kind that earns caution. It’s the kind of output that makes me ask, “Do we know what it really learned?” Not in terms of loss curves or test accuracy, but in terms of semantic generalization in terms of value alignment, bias sensitivity, ethical reasoning.

Because these systems don’t reason like we do. They generalize from data distributions. They reflect patterns we may not even know we encoded. And if we don’t understand how those patterns shaped what the model learned, we can’t be sure we’re deploying something safe let alone fair or just.

We Have No Excuse for Opaque Machines

Yes, humans are opaque too. We don’t always know why we do what we do. But we accept that ambiguity in ourselves because it’s natural. We didn’t design ourselves. When it comes to machines, though, we have no such excuse. We chose the architecture. We selected the data. We defined the objective. So yes, we should be held to a different standard not because we’re better than the systems we build, but because we are responsible for them.

When we release systems that behave in ways we can’t explain and still allow them to shape decisions that affect lives, we’re not just building technology. We’re outsourcing agency without accountability.

We Can’t Lower the Bar for Understanding

Physics didn’t spoil us by being intelligible. It gave us a taste of what clarity can look like. We didn’t abandon understanding when things got hard in quantum mechanics or chaos theory. We created new mathematics, new metaphors, new epistemologies to make sense of complexity.

So why should we lower the bar in AI? Just because neural networks are complex, doesn’t mean we should give up on explaining them. If anything, that’s when explanation matters most. When power grows, so must scrutiny.

Conclusion: We Still Deserve the Right to Ask “Why?”

The uncomfortable truth is, we often accept these systems not because we understand them, but because they’re useful. Because they’re fast. Because they work. And that’s not inherently wrong. But if we let utility override comprehension, we’re setting a precedent, one where decisions are made for us, and we stop feeling entitled to ask why.

I’m not anti-AI. I’m not nostalgic for mechanical systems or hand-coded logic. I’m in awe of what learning systems can do. But I refuse to conflate power with understanding. And I won’t accept a world where we trade away interpretability for efficiency, not silently, not uncritically.

If we’re going to build machines that make decisions in our world, they need to be able to explain themselves in our terms. Otherwise, we’re not building tools. We’re building oracles.

And oracles don’t earn trust. They demand it. That’s the difference.

And that’s the danger.

/tb.