December 6, 2022

Beware the Long 'YAGNI'

One of my favorite regularly occurring Twitter arguments is over what should be a simple and uncontroversial question: have you ever had to migrate to a different database in a production application before?

I wrote about our adventures migrating databases for MarkedUp analytics here thus I answer this question accordingly. But the group of developers who fascinate me are the ones who answer along the lines of “no and YAGNI - You Ain’t Gonna Need It.”

The reason for the “controversy” around this question is the “YAGNI” - the attitude that you should never need to migrate databases unless you’re incompetent, I guess.

Merits of YAGNI

YAGNI - “You ain’t gonna need it” - is a challenge issued to developers engaging in compulsive over-engineering. Unnecessary abstractions (i.e. IAbstractRepositoryCacheFactoryBase<T>), frameworkism, and other pitfalls of its ilk. YAGNI is a plea to refocus the development team onto the task at hand and not worry about future problems, technical minutiae, and other irrelevant time-sucks.

YAGNI’s intention is noble and it is a useful tool. However, like all things in software development YAGNI must be balanced against another important software development value: optionality - the ability to keep future options open at feasible cost by preparing for them early-on in the project.

Let’s examine YAGNI in the context of ever needing to migrate databases in any conceivable situation, since that’s how the question was asked.

“YAGNI” in this context is a laughably bad take. First: it’s Dunning-Kruger - if you don’t have experience migrating databases yourself then you’re not really qualified to judge those who have.

However, the biggest problems are more fundamental: short-termism and self-fulfilling prophecies.

Change is Certain

One of the constants of any software system is change over time. Given a sufficiently long enough time horizon it is inevitable that your applications will face new requirements.

Likelihood of change occurring over time eventually reaches 100%

This isn’t woo-woo product manager speak - it’s the laws of thermodynamics: entropy is always increasing and energy must be spent in order to retain order. That entropy could be in the form of your underlying dependencies requiring updates; hardware failing; licenses expiring out of support windows; critical security patches; or much more commonly - your business and its domains evolving.

In the context of how YAGNI was used in our “migrate database” debate it should be qualified with an expiration date: “you ain’t gonna need it within the next two years.”

Long YAGNI: Planning to Fail

“Long YAGNI” - “you ain’t ever gonna need it” - is an inherently stupid and self-fulling prophecy. How certain are you that your software system won’t need to change in significant ways two / five / ten years out? There are certainly cases where this is true and they’re the exceptions.

For business-critical applications, which includes back-office / auxiliary applications depending on their criticality, change is much likely to occur more frequently and significantly as they’re load-bearing to the business the software supports.

Adopting a “you ain’t ever gonna need it” mindset in these spaces leads to painting your software into a corner. If you believe that migrating databases should never happen then you’re not going to make any of the relatively inexpensive, early preparations that would render a database migration economically feasible in the future. Thus if another business problem emerges, i.e. drastic changes in the product or usage are incompatible with your database’s model or constraints, you’re not going to have the option of a database migration to address it.

YAGNI makes sense for defeating truly extraneous and weird over-engineering matters: we don’t need to build our own web framework or our own distributed cache - we just need to get something deployed that can accept orders using basic cloud PaaS. The temptation to YAGNI must be moderated, however, when you start dealing with matters that are more practical - anticipating future business needs for instance.

Over a long time horizon it’s important to consider areas with high impact and high likelihood of change - i.e. if we’re building an eCommerce application it’s very likely that we’re going to support multiple payment networks and fulfillment partners even though we’re only launching with one of each initially. We should make the effort to abstract our internal APIs for working with fulfillment and payment partners so it will be easier to introduce new ones in the future when that requirement is ultimately mandated by the business.

Applying YAGNI to this scenario is moronic - you know this requirement is going to get incorporated sooner or later and building abstractions that make payment / fulfillment relationships variable rather than hard-coded is orders of magnitude cheaper to introduce when you don’t have any orders, customers, or shipments yet. Deferring this design decision now will cost 100x the price later.

Factoring change over time into your decision making early in a project will lead to consistently better engineering and business outcomes. Keep your YAGNI’s short and focused.