Cloud Migration Is Not Free

Chapter 5 — drawn from Optimizing Corporate Efficiencies by Arthur J. Riel

Moving infrastructure to the cloud nearly always saves money — a minimum of 10% total cost of ownership, with the average savings running more than double that. Given the ROI, any workload that can move to the cloud, should. That part is easy to say in a boardroom.

What is harder to say in a boardroom is that cloud migration is full of decisions that do not show up in the pitch deck: which applications actually have to move together, how aggressively you clean things up on the way, where your savings curve bends, whether one cloud vendor is a risk or a strategy, and how you stop a pay-as-you-go bill from becoming a surprise. Get these wrong and the "obvious" cloud savings shrink fast or arrive years later than planned.

The Cohort Problem

The first surprise in most migrations is scope creep by dependency. Applications are best moved together with their underlying databases, to avoid latency issues between the cloud and the on-premise data center. That sounds manageable until you map it out: the 15 applications you have selected move with 5 databases, but each of those databases has dependencies on additional on-premise applications, and each of those has dependencies on other databases. Before long, you have swept up nearly the entire landscape.

There are two ways to handle this. The first is to move the applications and leave the databases on-premise as an interim step. In theory this risks latency problems; in practice, most well-designed applications handle the network latency between on-premise and cloud just fine. Poorly designed, chatty applications may struggle — and those are the ones you decide not to migrate and revisit in a later cohort.

The second option is to replicate the databases in the cloud and keep the on-premise copies running until every dependent application has migrated. This avoids latency entirely, but it creates a bigger problem: two live copies of the same database, some applications writing to one, some to the other. Now you need reconciliation processes to keep them in sync, which adds complexity and risk to your environment. I favor the first approach for exactly that reason — accept a small, usually invisible latency risk instead of running two databases in parallel.

Lift-and-Shift vs. Cleanup-and-Shift

This leads to the bigger strategic fork: do you move applications as-is, or clean them up first? I favor Lift-and-Shift. The cost of migrating now and cleaning up later is the same as cleaning up first and migrating later — I have never encountered a case where that is not true. If the cloud is cheaper than on-premise (and if it isn't, why are you moving anything at all?), then the fastest path to savings is to migrate as much as you can, as quickly as you can. Refactoring applications before migration is expensive in time, risk, and cost, and it often does not have the ROI to justify the refactoring.

I once had this debate with a CIO at a large financial services firm who did not want to replicate a messy on-premise environment inside his new, clean cloud environment. My response: if you want a utopic city on the hill, where anything that lives there has to walk, talk, and behave a certain way, that's fine — but I need a slum outside that city for the things that will never be worth cleaning up. The cloud is not one monolithic environment. You can define separate zones with different standards for what is allowed to live there.

If you choose Cleanup-and-Shift instead, go in with your eyes open: you'll start with a much smaller footprint, and that footprint will grow slowly as refactoring drags on. Your savings will grow just as slowly. All large organizations, and many smaller ones, have a number of solutions that will never make sense to refactor or upgrade. Why not run them in a lower cost environment?

Linear Savings vs. Step-Function Savings

When you calculate the payback on a migration, not all savings behave the same way. Some costs — compute, electricity — scale down linearly as you migrate. Others are step functions. If your 5,000-server data center runs on a 30-person support team, migrating 500 servers doesn't mean you can eliminate 3 support positions. Migrate only 10 servers and you will almost certainly still need all 30 people. And even a data center with just 50 servers left will still require a minimum 3–5 person team — that cost persists right up until you unplug the last server and close the facility.

Before you build your migration business case, list every cost component and mark which ones are linear and which are step functions, along with where each step actually falls. Otherwise, your projected savings curve will look nothing like the one you actually experience.

The Multi-Cloud Debate

Every CIO I have worked with has wanted a multi-cloud strategy at some point, and the logic sounds reasonable: if one vendor increases prices or takes some other adverse action, you shift your footprint to a different vendor and solve the problem.

In practice, "shift your footprint" is an expensive, slow transaction — and not just because of the cohort migration issue above. Over time, your teams start using cloud-specific services. AWS in particular, with Azure not far behind, offers a deep library of proprietary services and utilities. Unless you rigorously prohibit their use, moving from one cloud to another gets progressively harder the longer you are there. And why would you want to prohibit the use of features that provide you with faster time to market, lower development and maintenance costs, and ease of use?

My preference is to select a single cloud vendor, maximize usage to capture volume discounts, and lock in price protection at contract renewal. A typical 3–5 year term gives you plenty of runway for an exit strategy if you ever need one. So far, the major vendors have not given clients much reason to think that exit strategy will be necessary, but it is worth continuing to monitor new-client pricing over time regardless.

Governing the Bill

Whatever your cloud strategy, real-time cost monitoring is non-negotiable. With a pay-as-you-go model, a mistake can generate material costs in hours, not months. Build in alerts with multiple thresholds so problems escalate quickly — the innovation team's "interesting experiment" can trigger an avalanche of spend from a single unwitting developer.

Work with your cloud vendor's architects to make the right infrastructure choices from the start: reserved instances for workloads with consistent compute needs, on-demand resources only for genuinely variable ones, and spare cycle auctions for overnight processes. Storage needs the same discipline — standard storage classes for typical needs, and low-cost archival tiers (like AWS Glacier) for data you are required to retain but rarely retrieve, such as compliance archives.

Cloud vendor architects have a wealth of knowledge in these areas and it has been my experience that they are eager to share their expertise with clients.

The Pattern Underneath All of This

None of these decisions — whether they be cohort sequencing, lift-and-shift vs. cleanup, single-cloud vs. multi-cloud, or cost governance — is really about technology. They are about resisting the temptation to either over-plan (waiting for the perfect scenario) or under-plan (moving fast with no cost controls). Cloud migration is not free, but it is rarely as complicated as the hesitation around it suggests. The organizations that get the most value are the ones that map their dependencies honestly, pick a philosophy and stick to it, and put governance in place before the first workload moves, not after the first surprising bill arrives.