This last week, a good friend of mine introduced me to The Drive, a podcast tailored towards health optimization. There was one particular episode where Peter Attia and Layne Norton were talking about different diet strategies and Layne said something that stood out to me. He said, "There are no truths, just trade-offs" which is a slight variation on Thomas Sowell's wording, "There are no solutions. There are only trade-offs."
While the conversation centered around fitness, this same concept can be applied to systems design and project management in software companies.
Gather Requirements
There's an obvious real world example that comes to my mind when I think about this. What if someone asks you, "What's the best pair of glasses?" What would you say?
Very likely you'll need to ask more questions. Is the person near sighted or far sighted? How bad is their eyesight? How often are they out in the sun? Are they active? If so, how intense are their activities?
All of these questions have a major impact on what we consider best. There's no such thing as an objectively best pair of glasses without first considering the requirements. Among the many questions we ask, one of the most obvious that's often overlooked is do you even need glasses? In other words, is this a problem worth solving at all?
As an aside, this may be the very reason that systems design questions have a large weight in the interview process. So often the question is not "Can we build it?" but rather "Should we build it?" and "What's the best way to build it?".
Create Options
Even after gathering our requirements, our design work still isn't complete. Going back to the glasses example, we may have identified that we need near sighted glasses for an active individual that is often out in the sun.
Even with these fixed requirements, we have a lot of options. We could recommend a pair of photochromic lenses that automatically adjust their transparency when exposed to ultraviolet light. We could recommend two pairs of glasses: one standard pair for indoors and one pair of prescription sunglasses that's more secure when doing physical activities. We may even present different options based on price point which wasn't one of our functional requirements.
Another way to think of this is that our initial design is a breadth first design where we don't dive too deeply into any one solution. Instead, we learn just enough about each option to ensure that it meets our functional requirements and to identify obvious strengths and weaknesses.
It's worth noting that at this stage, we're not guaranteeing that a particular solution will work. The glasses example falls short in this case because it's very easy to completely explore the solutions before making any decision. However, in the case of software development we can't possibly know all of the risks for each solution up front.
Our goal at this stage is to spend enough time on each solution so as to establish the obvious risks. From there, we want to choose the option with the best tradeoffs which is the option that's most likely to produce positive results.
Depth First Design
From here, the problem becomes one of depth. But it can still switch back and forth depending on our situation. Let's say for example that we work on a specific problem for one month only to discover that our initial assumptions were off.
At this point, our design shifts back to a breadth first problem where we reconsider the tradeoffs with our additional knowledge. The month of development work invested here is sunk cost and shouldn't be considered as we determine the right path forward.
From here, we may determine to stay the course even with the additional complexity we have found. Or we may determine to switch to an alternate approach using one of the intial options we considered.
Summary
Systems design boils down to gathering requirements and then alternating between breadth and depth until we arrive at a good solution. I'm intentional in saying a good solution as opposed to the best solution because there's always a better solution, likely one that we didn't even consider. Software is always being replaced, upgraded, and molded to fit shifting requirements.
Once again, our goal here is to identify and build the solution that's most likely to produce good results.