Hello Paul
>> "how we handle these things, in concrete ways, as best we can during design"
Each project is a different story based on many factors like
1- Goal (Prototype/Production Ready/Research/Just Fun/etc.)
2- Scope: Small/Large
3- Contributors: Alone/Team
4- Previous Experience: Is this the first time to do such a project? or not?
5- Similar Projects (Related Work): What I learned from others
6- Known Use-Cases & Tests
7- Programming Paradigm
8- Programming Languages & Development Tools
9- Cost (Time/Money/etc.)
10- Project Spirit (What really matters - Values/Principles)
The most important lesson is to see the big picture for each project and get an answer for each of the previous points, and to know that it will be a partial answer in most cases and will grow along the time.
Say you will start a five-year project, while you have 20 years of experience, and you set a vision for this project and started working
After five years, you will have 25 years of experience including 5 years of related experience on this particular project, believe it or not believe, you will hope that you have a time machine to back in time and change a lot of things, something could be done using refactoring while other things could require a fresh restart.
Trying to know everything in advance is like saying (We will not learn anything new from this project so we can just do the right thing from the first time)
For some projects and based on points like (4) and (5), this could be done if the problem is already solved before (we know the solution)
Summary:
(1) Don't try to have a complete design that includes all of the details, just try to have a complete vision and select/change/adapt the design based on practical needs and according to the vision.
(2) Different factors like (4,5,6) will grow/change during development and you have to manage this
(3) In programming and software development there are many ways to achieve your goals. This includes many right ways and many wrong ways. Don't think about problems like (I have to find the perfect/unique/best solution) - There are always better solutions and selecting/doing one of them is related to many things like (1,9,10).
(4) Successful programmers understand this and provide maintainable solution and know how to manage complexity and choose wisely between different solutions according to our limited knowledge.
I am not trying to provide a complete answer, I just sharing some hints.
Also, these are my opinions (I could be wrong or miss important things)
Greetings,
Mahmoud