Is "obscurity" mostly "indirect dependency"?

[Steven Rose]

Section 2.3 says: "Complexity is caused by two things: dependencies and obscurity."

Rereading APoSD the thought occurred to me that, in my experience working on large software projects, a hugely significant source of obscurity is dependency. Specifically, transitive dependencies.

I haven't done another close reading yet, but I did a quick pass through the book again and realized that a lot of the advice can be viewed through this lens. It's almost as if the quote above could be reworked to say, "Complexity is caused by two things: direct dependency and transitive dependency."

Not completely. I don't claim that transitive dependency is a complete replacement for obscurity, as there are many examples given throughout the book where that wouldn't make any sense (e.g., giving a variable a vague name, just to pick one out of a hat).

But dropping in "transitive dependency" for "obscurity" does work for much of the advice throughout the book.

Strategic vs. tactical programming. The tactical programmer creates uncontrolled dependencies that transit through the codebase. The strategic programmer may compromise code quality in order to speed things up, but perhaps the key difference is the strategic programmer does not allow dependency to transit uncontrolled.

Deep vs. shallow modules. If we take dependency inversion for granted as part of a good design—that is, both the code that uses and the code that implements an interface depend on it, and dependency does not transit the interface from caller to callee—let's compare two systems, one built with a lot of shallow modules and one built with deep, more general interfaces. What do the dependency structures look like in these systems side by side?

There are a lot of dependency arrows going to a lot of places to leverage shallow modules. Such a system ends up with a high number of modules that expose a lot of detail-oriented functionality. If you step back and squint, this is kind of like breaking encapsulation and letting the guts of the module you should have built spill out into your design (addressed by Chapter 5 that covers information hiding and leakage, classitis). Using shallow modules gives the capability of being able to plug all of these little legos together in any way a caller might want … but if the fact is that no caller wants to leverage a lot of that flexibility, you're just creating complexity that's not carrying its weight.

Deep modules force you as an architect to make decisions about what the most important functionality is, and commits you to supporting only those APIs over many versions, but not a lot of nonsense that no one is going to use or worse, APIs that callers are forced to use to get what they want. The relevant example in the book is Java I/O, where you have to layer a bunch of input streams to get buffering, which should just be a boolean passed to the constructor.

Different layer, different abstraction (Chapter 7). I have a great example of this point from a recent project, something I couldn't quite put my finger on, but it's a bit long and involved to explain here. Without going into the gory details, though, I can say that I now regard object-relational mapping (ORM) tools like Hibernate to scatter little landmines throughout your design, and the reason is addressed in this chapter.

Schema dependencies are formed by one table holding data owned by another table, the most frequent example being a foreign key. If I have a Users table (ID, username, etc) and a Phones table (ID, number, type aka mobile/home/etc, user_fk), the Phones table holds a foreign key into Users, i.e., it "depends upon" the Users table. Queries for phones in this schema will certainly join to Users, meaning this dependency will transit up into the code that forms the Data Access Layer in the application.

This isn't necessarily a bad thing. If the schema / code deps reflect a real dependency in the business domain, no problem. But look at what an ORM does: It often reverses these schema dependencies. When I query phones and join the users using an ORM tool, it will construct a User object for me with a collection of Phone objects attached.

This is obviously a trivial example, but the whole point of these objects constructed by an ORM layer is to detach from the database and get passed around in application code, which means that dependencies that were thoughtfully chosen (hopefully) in the schema are now thoughtlessly spammed into the code after undergoing this sort of random transformation that may or may not make sense. It probably does make sense for users and phones, but for the long and involved use case I confronted, the explosion of deps that resulted in code from the ORM tool that no one chose handcuffed us in all sorts of ways no one clocked. It was just furniture in the room.

OTOH, if you just query the DB and deal with the result set, all of that data has to be manually packaged into DTOs, and there are no dependencies mandated by the schema at that point. You are free to package things up however makes sense.

I feel like I could keep going. Pull complexity downwards (Chapter 8) makes for fewer, more general interfaces lower in the stack, which means cleaner dependencies above. Define errors out of existence (Chapter 10) means that a whole separate track alongside the call stack of error handling code poofs out of existence, along with the sprawling multitude of dependencies that would result.

When I first read this book, I found it fills a vacuum that is mostly unaddressed in software. Everyone talks about design patterns and principles like SOLID, but these are all offered up as individual bits of hard-won wisdom with nothing tying them together, which a more general philosophy of software design does. At the same time, I found it difficult to really grasp some of the concepts like shallow vs. deep, and strategic vs. tactical, and I've been turning these over trying to make them more concrete.

One of the questions I had is where to find the balance, how to know when the advice is taken too far. If I view the impact of the advice on dependency, particularly transitive deps, it does seem to make it easier to find the sweet spot.

Interested to hear if anyone finds this helpful at all!