If you’re considering migrating data and have found the perfect storage platform to move to, there’s one question that might be on your mind: how do I actually get the data there? When you use Movebot, the migration itself will be the easy part. So how do you handle everything else around it?
Here we’ll show you how to create a data migration framework to plan out your migration so that the data-moving process will be one bottleneck you won’t have to worry about.
Data migration framework overview
A basic data migration will follow the same basic steps: Data assessment and inventory, deciding what to migrate, backing up data, doing a proof of concept test, running the migration, and final checks/go live. The type of data you’re moving and for what reason–like changing platforms compared to a merger or acquisition–will change how long and how much effort goes into each section, but the basic migration framework will generally be the same when migrating files.
Documenting the process
If you’re new to migrating data or have a complicated project, recording and documenting the migration process can help keep the job on track. And in some industries, thorough documentation could be required for auditing purposes and to ensure sensitive data is handled properly.
Typically, the bigger and more complicated the migration, the more important it is to plan and document everything. For smaller, straightforward moves like moving a Shared Drive as is from one platform to another, simply doing the move without an intricate plan or documentation could be fine if the migration tool has detailed reporting like Movebot does, as you’ll quickly be able to find out if any files failed to transfer or if any issues came up. But when moving dozens of terabytes or migrating for a merger, planning and documenting thoroughly can stop potential issues before they become problems.
One thing to keep in mind when planning the migration is to always give extra time if you can to finish the project so nothing gets rushed. Even with tools like Movebot that can transfer data quickly, it’s better to have extra time and be finished early than to miss an ambitious deadline. Plan and document what data there it is, where it’s going, and how it will get there. It can be helpful to plan what data will be used for test migrations and how to validate everything made it across as well.
Taking Inventory - What data is there, and what do you need to move?
Part of a solid data migration framework is taking inventory of what’s in the source platform. Once you’ve decided on which cloud platform is best, someone should take inventory to determine what’s there and what needs to be moved. Files that are outdated or duplicates could be deleted, and ones that are older or just for record-keeping and likely won’t need to be accessed could be archived to save space and to help make the destination cleaner and easier to use.
This stage is also a good opportunity to check the existing folder structure, permissions, and naming conventions. Is everything still valid? Does naming make sense? Do the right people have access to the right data? If any of those are a no, a migration is a great time to make changes so users can start out working in the new platform the right way.
With some data migration tools (like Movebot), you don’t have to move everything in the source to the destination, so some of this stage can be done as part of the migration. Movebot’s Discovery Scan checks for duplicate files, and you can also break down migrations to the folder level. You can also choose to exclude/include items to be migrated/ Since Movebot supports over 30 platforms, you could easily separate important files that need regular access and files that should be archived and move them to different destinations.
Estimating time and costs
When you know what data you want to migrate, you can get an idea of how much there is to migrate, and as a result, start figuring out estimates for how much the project will cost. Depending on which migration tool you go with, there are a few things you’ll need to factor into determining the cost.
First is the cost of the tool itself. Usually, this will be either based on the amount of data you move or the number of licenses you need. With Movebot, figuring out the price is easy. Use the pricing calculator to get an estimate, with costs at no more than $0.75/GB with lower rates the more data you move. There aren’t any other costs like managing licenses, so the other main cost factor will be time, which will be on the lower end since Movebot is completely SaaS and scales automatically.
Most other tools require you to download their software and run it. Usually, they’ll recommend to run it on one or multiple virtual machines, with an additional license for each required instance. So you’ll not only need to estimate how many licenses you need, but how many VMs to run and manage as well. And since many migration tools don’t scale automatically, you'll need to manually monitor and make adjustments during the migration to make sure everything is going as expected.
Data backups
Regular backups are always a good idea, and especially important before a migration. While there’s practically no chance of data being corrupted or destroyed in a migration, it’s still a good idea to run a full backup before the migration.
This is also good opportunity to create a backup and recovery plan for the new storage platform. When you’ve tested that your backups are working right, the next stage is to run test migrations.
Proof of concept test
Before starting the bulk migration, it’s a good idea to do a small, POC to make sure that the migration tool and the process meets your expectations. The most straightforward way to do this for migrations is to run some test transfers.
For these tests, try to simulate the main move as closely as you can. Are you going to migrate permissions, change filenames, and convert doctypes? Make sure the test migrations cover those things. After the tests, check that the files arrived as expected. Once you’re satisfied with the results, it’s time to run the bulk migration.
Running the data migration -- The old way
Depending on the data migration tool you use, the migration itself can either be one of the easiest or one of the most challenging parts of the process. With the wrong tool, there’s a lot of things you have to plan before you move a single file.
First is determining how much compute power you need, how many VMs you need to spin up and manage, and how many licenses you’ll need. You’ll probably have to download the tool for each machine (with each one using up a license) so you have to balance license cost and migration performance.
Then you’d run the migration itself. This might involve scheduling downtime to make sure the process doesn’t interfere with anyone’s work, since changes could mess up the process or the updated versions might not end up in the destination. Someone would need to keep a close eye on the process and VMs and make sure they don’t act up so that the migration doesn’t stop or fail. With some tools, a stall or failure might mean having to restart the whole process over again.
When that (slow) process is done, someone would then need to thoroughly audit the destination and make sure all the files moved over. Tools that don’t have adequate reporting might not let you know if something didn’t make it as expected, so it could be complicated to determine if something went wrong. The whole process is a lot of hands-on work and monitoring, with extra time and resources that are hard to estimate ahead of time.
Fortunately, there’s a better way with Movebot.
Running the data migration -- The Movebot way
With Movebot, migrations are easier than ever thanks to our simple 3-step PAC migration plan of Plan, Advance, Cutover.
Plan is the first step of the migration plan, and if you’ve already decided on what data to move and how it will be structured, this part is practically done. All you’ll need to do in this case is connect your platforms and map the transfers. With Movebot, you have a lot of options to help reduce the amount of planning you’ll need to do. Automatic filename sanitization and deduplication help prepare your data for the destination without all of the manual work, which would otherwise have the potential to be a massive undertaking depending on file, folder, and naming limitations. Mapping can all be done within Movebot as well.
Advance is the next stage, where you start running the transfers that you mapped. When a transfer is running, you’ll have access to the Performance Advisor that gives live stats about the ongoing transfer. With Movebot, you can run multiple transfers at the same time, with full control over which ones run at what times. Run these transfers until the bulk of the data is moved over.
Cutover is the final step, where you migrate the last bit of files and run delta migrations. With Movebot, delta migrations scan for changes in the source and only migrate new and newly-modified files. Because of this, users don’t ever have to stop working in the source. Simply run deltas until you have all of the data moved across and use Movebot’s built-in detailed reporting to double-check.
Final checks and go live
Before cutting off access to the old platform, do final checks to make sure nothing is missing or copied over improperly. If the migration was to a different platform, users will need to adjust to the differences.
Now there’s a good chance there will be some growing pains for a bit. The new platform will probably do things slightly different, and it’s common for some urgent tasks to fall into old ways because they’re familiar. The larger the user base, the longer you should set aside for training, as the more users have time to work in the new platform before making the final switch, the easier it will be.
One option is to have a built-in transition period, where the new platform is used for specific work and slowly building on what’s used until everything is done in the new platform. Once there’s enough confidence in the new platform and users get used to the differences, you can safely remove access to the old one, although it’s recommended to do this at least 30 days after the migration
Try Movebot for your next migration project
The migration tool you use for your project has a massive impact on the data migration framework. Depending on the tool, there will be different steps, costs, resources, and timeframes to think about, so it’s important to pick the right tool for the job.
And a lot of times that tool is Movebot. With the widest platform support, no infrastructure to worry about, and detailed reporting, Movebot streamlines migrations and makes them easier than ever before. Find out for yourself with a free trial. Each signup gets 50GB free immediately, without any required sales calls and no credit card needed to try. Register for an account and start your free trial now to get started.
