Data engineering ยท Decisions

Count the whole technology cost

The cheapest tool on a pricing page can become the most expensive system to own.

Inspired by Fundamentals of Data Engineering by Joe Reis and Matt Housley. This is an original explanation, not a reproduction of the text.

The idea

Infrastructure price is only one line in a technology decision. The real cost also includes integration, operations, training, security, migration, and the time engineers cannot spend on more valuable work. A sophisticated platform may solve impressive problems your organization does not have while creating ordinary problems it must now maintain.

Start with the outcome and constraints: users, latency, scale, reliability, team skills, and time to value. Compare a few viable options over their likely lifetime. Give special weight to interoperability and reversibility, because requirements and tools will change.

Make it concrete

A six-person company considers building a streaming platform to refresh an internal metric every minute. A managed batch service costs more per unit of compute, but ships in days and needs little supervision. The custom option looks cheaper on infrastructure alone; after on-call work, specialist hiring, and delayed product work are included, it is the costly choice. The team chooses hourly updates and preserves an upgrade path.

Keep this: Choose technology for business value over its full lifetime, not novelty or sticker price.

Try it

Take one proposed tool and list its price, people cost, operating burden, migration cost, and opportunity cost. What simpler option now deserves comparison?