One of our most requested features is adding the option to sync data between two different storage platforms. The idea is that if you make a change in one platform, that change will immediately be reflected in the other storage platform. Add a file in Google Workspace, and have it appear in SharePoint. Make a quick update to Campaign Final FINAL and overwrite the older version of the file that's already there.
While it might seem simple at first glance, this is actually an incredibly complicated function that can have a lot of unintended consequences. And because these consequences can be catastrophic, the Movebot team has decided not to pursue file syncing and instead focus on what we already do extremely well: migrating files and mailboxes between platforms.
So what exactly are some of these issues with syncing files, and why do we caution against it?
Types of file syncing
There are a few different types of file syncing methods such as one-way sync, adding files only, and two-way synchronization.
One-way sync
A one-way sync only syncs changes in one direction. If you make a change in one platform, that change will be reflected in the other, but not the other way around.
For instance, suppose you have both a Dropbox and a SharePoint account, and you need files from SharePoint to go into Dropbox. With a one-way sync system, anytime you make a change in SharePoint, that same change will sync up to Dropbox. But if you make a change in Dropbox, the SharePoint Site will be unaffected.
This is one method of doing file backups. If you want to make a copy of a specific folder in one platform to another, you could set up a one-way sync on that folder to an identical folder in the backup platform. Doing this would function as a near-live backup, with peace of mind that changes in the backup platform won't affect the main store.
Syncing new files only
Syncing new files only is a way to make sure both platforms have new files, but it only activates when a brand new file is added. This means if you make a change to the file, the change won't sync, but anything new will.
This method has its pros and cons. It's safer than a full two-way sync, because you won't lose anything by accidental deletions. There also shouldn't be prioritization conflicts because there's only one operation happening, so even if there is an error, you're still likely to end up with the new file in both places.
However, since it only syncs new files, you won't get any updates and will need to sync updates another way. This is great when you use it for a folder that will have files that don't need to be modified, but can be a strong downside for collaborative work.
Syncing new files one-way is one of the most common ways to sync files between platforms because it's safe and simple to set up. The downside is that changes won't sync, but this kind of syncing operation tends to be effective for information files that don't need to be updated, such as sending daily customer order files to a folder in a different storage. Often these are combined with steps like delays, like the daily orders being sent on the first day of the month to an archival platform. Since the archival platform is for long-term storage and the files should rarely if ever be modified, syncing new files one-way is an excellent solution.
Two-way synchronization
A two-way synchronization system syncs everything you do on either platform to the other one. You can set this up to happen at a drive or directory level, or for the entire organization. A two-way sync might seem great for keeping employees who work on disparate systems up to date with each other, but there are some major issues that can cause you to lose updates or even entire files or drives if you aren't careful--or if an error happens outside of your control.
The problems with a two-way sync system
Two-way sync issues can cause major issues if set up improperly or if an error occurs, so it's important to set up your syncing properly and be aware of potential risks, particularly when doing an automatic sync.
Who's the winner?
Deciding which system takes priority is one of the biggest issues with a two-way sync. Sure, maybe you already have a preference in mind, like the latest update always wins no matter what. But when changes happen close together, deciding which one is earlier or later isn't always straightforward.
Sync operations don't happen in real time. The two platforms need to communicate with each other, then determine which operation takes priority. And since it's not live, any hiccups in communication can get those update times altered, enough that it might be a challenge to know which update happened last and as a result, which one takes effect.
This problem only gets worse when an update fails. A user can make several changes to a file, expecting those changes to sync to the other platform. But if that update fails, another user could make changes to the same file (not knowing the update failed), creating a more recent update. Now you're back to deciding who's the winner, and there's a good chance the whole thing will be a mess and somebody will have to redo their work.
Multiple users
If one person wants to sync files across multiple platforms, there's unlikely to be major issues. This is because that person has to log in and perform an action on a file, already expecting that change to happen across their other platforms. There's a much lower chance of the same user working across multiple platforms simultaneously and not knowing which version of the file is up-to-date.
But with multiple users, there's a much higher chance this will happen. If multiple people are making changes in one or both platforms, there's the chance that they'll make a change that will be in conflict, and then the syncing operation has to try and determine which one to use as the latest update. When changes happen close together (or there's an error), this can create a conflict, and the wrong version might be considered the newest and become the one that syncs across platforms.
File deletions
With a two-way sync system, deletions are a risk for a few reasons. One is accidental deletions. If you delete the wrong file by accident in one platform, it could be gone from the other one too, and if you were using the second platform as a backup, it's a quick way to lose that as well.
Of course, most platforms have a recovery system in place for accidental deletions, but it's not always as straightforward as clicking to recover, and they don't have an unlimited timeframe, so you'll have to notice the error before recovery time runs out. Data retention times vary widely depending on the platform and the plan chosen, and typically business and enterprise tiers have much longer retention times.
Even still, deleting data will always carry some risk when syncing is involved, so it's crucial to check that everything happened as expected--or don't use included deletion events at all as part of the synchronization.
Movebot focuses on data migrations
The Movebot team decided that having no access to the data was more important than having a data syncing feature. This way you have full control of your data and a better level of security. Instead, we've focused on doing one thing and doing that one thing better than anyone: data migrations.
You won't find a faster or easier data migration tool than Movebot. As a platform-agnostic migration tool, you can migrate files and mailboxes to and from over 30 different storage platforms, moving terabytes per day with a scalable SaaS tool that just works. And you have full control of your data. Movebot doesn't store data at rest or in transit and doesn't delete files or even alter them without you making the decision.
Because of this, in the extremely rare likelihood that there’s a glitch in the system, the software can’t affect your data since it doesn’t store or have access to it. You won’t get an “unknown error” and have your files deleted because there is no way for Movebot to delete them without access to change them. Instead, the transfer will simply be unsuccessful.
Even when there's minimal risk of data being lost, the risk is still there. This is why Movebot works as a copy-and-paste operation; when you migrate from one storage platform to another, we'll make a copy of the data in the source, alter it if required with features like filename sanitization to fit the requirements of the destination, and put that data into the destination. Files are never moved, so nothing is deleted.
Run migrations with you in the driver's seat with Movebot, with no complex licenses or other complications to worry about.
What if syncing files is essential for my team?
Movebot is designed to move data, not to sync two systems or create backups. You’re breaking up with your cloud service and not coming back… Unless your ex brings some really good gifts. And if you do decide to go back, Movebot will be there, unjudgingly, to help you move your data when you get back together–and won’t delete your keepsakes box.
Syncing files is an important feature and there are plenty of solutions for that. We recommend using a one-way sync with purpose-built automations to sync your files across platforms. This should avoid any of the major issues of two-way syncing. For automating frequent file transfers and backing up on multiple systems, a solution like Couchdrop might be what you’re looking for.
But if what you're looking for is large-scale data migrations from on-prem to cloud, cloud to cloud, or between cloud tenants, Movebot is the perfect solution. To find out more about how Movebot can help you with fast, secure, and simple data migrations, sign up for Movebot free and see how it works with a free 50GB free with no credit card required.
