Migrations: What, Why, When, Where, How?

What is a migration?

In short, a migration is a movement from one place to another. That might be a movement of data from one data platform to another, or a movement of a web application from one server to another, or a service contract from one provider to another. There might be underlying changes, or it might be a “simple lift ’n’ shift operation”, but at its core is movement.

Common migration use cases include:

• Moving your DNS provider
• Moving your application to a different server
• Moving to a whole new hosting provider
• Moving your data from an old system to a new one of the same type (i.e. a major upgrade)
• Moving data from one system and into another, on an ongoing basis
• Moving from an old legacy system to a completely new one, of a different type.
• Moving from an old agency to a new one.

Migrations can mean a lot of things to different people, depending on what is required.

Why bother migrating?

Often the question arises: do we need to migrate? The answer will vary depending on circumstances, and will certainly depends on 

1. The motivations for the move
2. The risk or cost of maintaining the status quo
3. The value of keeping legacy data

For example, if developing a new website to replace an old one, you may wish to rewrite the content along with the new site, or you may wish to transfer (i.e. migrate) the old content to the new site.

You may wish to consider moving DNS provider if your existing provider offers a poor set of features; or you may wish to move hosting provider if your current platform is unstable. It is a question of balancing the risk vs the hazard vs the effort vs the reward.

DNS

Moving DNS provider should be as simple as pointing your domain to the new service and waiting for the updated setting to propagate around the world. The key to smooth DNS changes is the TTL (Time to live). The longer the setting, the less strain is put on DNS infrastructure, and the slower changes appear. The shorter the setting, the faster and smoother the transition. Whilst a TTL may often be 24 hours during normal operations, setting it to 5 minutes before any move will save hours of heartache.

Unfortunately, not all DNS providers, nor domain name providers are equal, so your experience may vary.

Server Migration

A server migration is where one machine (or platform) is, say, being decommissioned, and applications on it must be moved to another machine or platform.

The trick here lies with advance preparation and planning. If all the server level software remains the same, it can be relatively straightforward, however that is commonly not the case. Often versions of database servers, web servers, PHP engines etc all change, and with that come knock on effects. The route to a smooth transition is in testing before the move. 

If your new machine is set up, and a copy of your application and its data is put in place, then thorough end-to-end testing should be carried out before the final migration. Once all testing is complete and all stakeholders are satisfied that all is present and correct, once more the application code and data can be copied across from old server to new. At that point, traffic can be directed to the new machine, often through a DNS change.

Hosting migration

Migrating to a new hosting platform is very similar to a server move, with the caveat that different hosting platforms can and do manage configurations and setup quite differently. Here we expect to have to unpick legacy settings and ensure the set up of the new platform adequately services any requirements covered by the old system.

As with a server move, the key lies in 

1. adequate testing in advance of the migration
2. a TTL of 5 minutes in advance of DNS changes
3. A final copy of code and data at time of move

Major Upgrades

Major upgrades involving a migration used to be a common cycle. Moving from Drupal 6 to 7 usually required a migration for example. The same is true of Drupal 7 to 8 where there is no real upgrade path. With Drupal 8, that cycle has all but ended, as upgrades from 8 onwards will be relatively smooth and migration free. But with the oncoming end of life of Drupal 7 this year, there is one big batch of migrations left: the jump from Drupal 7 to Drupal 9.

In this case, the database structure has changed between versions, and the migration must map values from the legacy version to the newer version. Fortunately, Drupal comes armed with modules, plugins and extras to help developers along the way.

Complete System Change

A common use case for migrations is the ‘complete system change’. This, for example, might be where an organisation is moving from a legacy CMS (content management system), such as WordPress, or Typo3, or SharePoint, or Terminal4 to a modern system such as Drupal 9.

In this case, the legacy system is the 3rd party external data source, and the task of the migration is to access the existing data, parse it, process it, and inject it into the structures being built for the new system.

This can be relatively straightforward if, for example, dealing with the known quantity of a structured WordPress database, however it can be greatly complicated if, say, one must parse generated flat HTML files, or if the data exists in a more esoteric CMS. Often source content may be in a single chunk, whereas a new site will be built with modular building blocks or layout controls. Any variation from the old system to the new must be accommodated or deliberately deemed unimportant for an effective migration.

Migrations such as these can be complicated, and therefore can come with a hefty price tag. It is worthwhile considering the following questions before engaging in the development of a full-on custom migration:

1. Is the old content required?
2. Is it still fit for purpose?
3. If usable, does it need to be edited after migration?
4. How much usable content is there?
5. Does the structure match the new site?

Unless the volume of content is prohibitive, it can often be more economical and practical to forego the tender ministrations of an automated migration, and to allow human fingers to copy and paste the legacy content into its new home, editing as you go. 

Ongoing Migrations

Sometimes there is a need to import data from an external source on an ongoing basis. Like the automatic import of a news feed, or an import of tabular data from a spreadsheet. Migrations can be created to take data from a third-party external source and import it into Drupal’s data structures, creating pages, blocks, users etc.

The difficulty with ongoing migrations is often not the creation or running of the migration itself, but usually lies in inconsistent source data. The real challenge in these cases is in validating that the data is good, without seeing all possible permutations, and that the end result matches what is desired. To take a term from the world of accounting, a ‘reconciliation’ of post-migration data can lead to a far more robust migration. However, whilst easy to say, such reconciliation and cross-checking routines are not easy to implement.

Moving Agency

Moving from a familiar agency to a new one can be a terrifying prospect. And the idea of migrating your web applications and everything associated with them to a new agency is even scarier. Fortunately, it does not have to be. Relationships, if not cared for, can fall apart and people need change. Should you find yourself needing to move, it helps to plan the migration. You’ll want to have the following lined up:

1. Do you have access to code?
2. Have you access to the data, both database and file assets?
3. Have you credentials to log into control panels for the domain, DNS, and any ancillary services?
4. Have you credentials for any 3rd party, e.g. social media, accounts?
5. Do you hold any documentation for your project?
6. Do you hold ownership of each of these items, or do you need to take over ownership from your old agency?

Plainly, there is much to consider, but it is often done successfully, and when done migrating to a professional and competent agency, frequently with a minimum of fuss.

The Many Faces of Migration

As we can see, migration means a myriad of things, and each one has different pitfalls, different challenges, and different strategies to ensure success.

Annertech has been successfully managing all the above kinds of migrations for over 10 years, migrating hundreds of thousands of content items and millions of files and helping many, many clients make the journey from the old to the new.