Data Migration: Strategy and Best Practices
Migrating data to a new storage platform isn’t something most organizations do very often. But when faced with a data migration project, it’s important to get it right. The process is complex, requires a lot of planning, and has the potential to be very disruptive to the users.
Data migration is not an overnight process. Depending on the volume of data and the resources available, it can take weeks or months to complete. But if you do it right, migrating your company’s data will be relatively painless.
So how do you ensure success? How do you keep disruption down while still making sure everything goes smoothly? And what do you need to consider before even starting?
We’ve put this article together to help guide you through the process of migrating data and the key factors that will affect your success. We’ll also share some tips on how to get started, and – most importantly – how we can help.
What Is Data Migration?
It is estimated that over 80% of the data in today’s enterprises was created in the last two years. In addition, newer regulations require that certain types of data be kept for extended periods. As a result, many businesses are running into the limitations of their current infrastructure. This explosion of digital information has resulted in an increased need for IT professionals to more effectively manage and maintain enterprise data — and to store it in the most economical way.
Data migration is simply the process of moving your data (file, object, or block) from one location to another – typically between different storage systems or to the cloud. The need to move your data could be created by a number of factors including: existing storage infrastructure has reached the end of its useful life, to move select data to a higher (or lower) performing storage system, or to take advantage of the cost savings and efficiencies of the cloud.
Whatever your reason for migrating data, to effectively manage the process, organizations need a comprehensive approach to the problem. This includes proper planning, tools, and techniques.
Planning: Know your goals and understand your options before you start!
Our team has worked with many customers on data migration projects, and we’ve learned that planning is everything. Make sure you’re aware of as many of the potential issues and challenges as possible ahead of time and have a plan for how you will deal with them. We’d like to share some of our best practices in this area to help ensure that your next migration is a success!
Key considerations for a successful data migration
Understand your data
Start with identifying exactly how much data you have. It sounds simple, but it’s surprising how many organizations we’ve worked with don’t really know. If you’re unsure, there are analysis tools that can scan your existing environment and provide this information.
Next, identify how fast the data is growing and what the change rate is. These are important metrics for understanding how many iterations of data transfers might be required to complete the job.
Understand your network
It’s worth pointing out that the laws of physics are immutable — if the amount of data and change rate are high enough and the network bandwidth is not fast enough, it will be physically impossible to keep up, and the project will never be complete!
Understand your source environment and user needs
Data migrations can be taxing on production storage and may have a noticeable impact on user experience. For this reason, migration jobs tend to run during low-impact hours, overnight or on weekends. It’s important to understand what resources will be available on the source system and what the user performance requirements are.
Security Considerations
If you’re moving data to a remote site or to the cloud, you’ll want to make sure whatever tool you’re using will encrypt the data in transit. You’ll also want to make sure user permissions, ACLs, directories, and metadata are preserved? Are there any regulatory requirements, such as GDPR, that must be met when migrating sensitive or personally identifiable information (PII) such as health records, financials, etc.?
Create the Schedule
Based on available resources and user needs, should the migration job be run only at night? On weekends? Are there any exceptions? Should throttles be implemented that limit the amount of resources a migration job can consume? Does the tool you’re planning on using give you this flexibility?
Prepare for possible unforeseen issues or delays
What happens if something goes wrong or the job takes longer to complete than originally planned? Will you be able to continue using your current system, and at what cost?
Executing your Data Migration Plan
Once you’ve done all the prep work and are ready to go live with your migration project, it’s time to execute the plan.
Provided you have covered all the bases, the work required at this stage is limited:
Kick off the migration job
As the heading suggests, hitting the “go button” is probably the simplest part of the process.
Monitor the process
Once the job is underway, it’s vital you monitor the progress. The most important metric is the data transfer rate. Are you moving data at the speeds you expected? And are you on target for meeting the project deadline? There are plenty of reasons why a migration might stall or even fail – from human error to networks not performing as advertised to hardware issues such as network switches or storage controllers failing during data transfer.
Trust but Verify
Verifying success post-migration is just as important as planning for success beforehand! To ensure everything went according to plan, it’s important to look at and measure results such as:
Data integrity/availability – was all data migrated successfully without any errors? Can you run a checksum report to verify? Data accessibility – Are there any issues with access permissions on files once they were moved across? Did all users have full read/write access after migrating?
Did anyone notice any issues with their day-to-day workflows after migrating data?
That’s it! Once your migration is complete, it’s time to move your users to the new platform and start using it.
Work with a Migration Engineer Today
If you have any questions about Parsec Labs or the specific migration solutions you need, please don’t hesitate to reach out to us. We have a dedicated team of experienced engineers ready to discuss your needs and help you develop the right plan.
