How to ensure a successful data migration

Moving data from one system to another can sometimes feel like getting a very wiggly, recalcitrant child ready for bed. You may need to figure out where they’re hiding, chase them around the house, brush their teeth, get them out of their dirty clothes, bathe them, pick out acceptable pajamas, hold them down while you squeeze them into said pajamas, read a few books, and sing a song or two before turning off the lights (or is that just my house?). Migrations can be one of an implementation’s most complex and challenging parts. You find and extract the data, clean it up, determine the best setup, put it into the system, and then possibly soothe it further to get it “to bed”. On top of that, legacy and target systems rarely align seamlessly—you can’t usually drop an active, excited child straight into bed.

Migration approach

At Fíonta, we take an in-depth and thoughtful migration approach to ensure that the resulting database aligns with any new business processes and software differences. We develop complex custom logic, translation sheets, and sometimes even data, which results in actionable historic and future thinking systems. Organizations often attempt to reduce costs and save money in the migration step of an implementation project. Simply migrating data “as-is” can land organizations in situations where their legacy data does not align well with new business processes or software functionality. Hopping back to the wiggly kid analogy, if you skip any of these critical steps, your child (and you!) may not have a restful night. You may end up with historical data that does not blend well with future data and a laundry list of data cleanup tasks.

I often hear, “I don’t know how you do this” (regarding data migrations, of course, not parenting, though sometimes I wonder). It takes a unique person with an analytical mind, a love of puzzles, bulldog determination, big-picture thinking, and a laser focus on details to do this job well. Our migration team members have between 8 to 20+ years of experience with data migrations across different platforms and 1000s of organizations. I have 24 years! So, we have “been there, done that” and have reverse-engineered countless databases.

Ten tips for migration success

And, because I’m a parent, I’ve included parallel bedtime tips!

1. Having context and insight (Love and understanding lead to the best outcome at bedtime). Our team is very specialized in their craft. We have worked with hundreds of nonprofits and associations. Most programmers can read a map that details how a particular field should migrate and throw the data into the system, but our team can understand how that mapping and logic affects the big picture. We know how changes to data setups will interplay with reporting outcomes and overall business models, and we use that understanding to “sing” your data into the right place.
2. Collaboration is a MUST (It is a huge help to have a partner or older siblings backing you up at bedtime). Migration folks tend to focus on the nitty-gritty details, often unearthing data characteristics that may have yet to surface in Discovery or Design. We work with our internal and customer teams to produce solutions that best meet the needs of our customers. We empower our teams by offering options and the information to decide what data cleanup tasks customers can complete minimizing the budget. It is an exercise in prioritization as we work to ensure the efficient use of budgeted time and to deliver the highest value for our work.
3. Check the data (Make sure you’ve got the right child). We recommend that customers provide full backups of data from the systems we are migrating when possible. We can also work with exports to Excel or .csv files, but the export process may not be easily repeatable. It can sometimes introduce new data structures or result in a loss of data integrity if elements are accidentally omitted. We often receive new data backups at various stages, allowing the team to test complex logic or see how a change in the build may impact data. We must ensure we receive and set up the correct data by comparing the most recent records and record counts with what the customer sees in their database. Also, equally important is to compare the initial data submissions with the subsequent data submissions.
4. Clean-ish data (Throwing a muddy child into bed will result in more laundry…and more work for you). Data is rarely (if ever) pristine, meaning that even given the best effort of the project team, perfection is not possible in a migration. However, an iterative process enables our customers to request changes and get to a more refined database. We migrate data into test environments in sprints and/or provide 1 to 2 full test migrations before the final migration. Doing so allows customers to review the data in the new system and provide feedback. You know your data, so this step is critical as we move toward the final migration. With each iteration, changes are reviewed, prioritized as needed based on the scope of work, and implemented, getting you closer to the database of your dreams.
5. Change is inevitable (Children grow and develop new attitudes constantly). Between the start and end of a migration project, customers will continue working in the system(s) they are replacing. Expect changes and new data to be added between the initial submission and subsequent submissions. It is vital to accommodate the updated data. Some good examples include new values in a drop-down, data cleanup occurring in parallel to the implementation, and additional fundraising campaigns or reporting groups. In some of these cases, we may need to modify our code, update a translation table, or even add a new translation table, depending on the change.
6. Organization is paramount to success (Set up a routine at bedtime and follow it step by step). We can expect notable change requests between different data loads on larger projects. Methodically focusing on one change at a time and getting clarity around the change helps to keep things more digestible. You must first focus on the details and then zoom back out to focus on the big picture, which occurs throughout our process. A migration team member must be meticulous and organized. We are tracking hundreds of details. The approach we use to track information is determined at the project level based on the size and budget of the project. The key here is that regardless of the tool used for tracking, we must track all decisions and changes along the way in an organized fashion. We keep our mapping document in lockstep with our change decisions and migration code. On many projects, we use JIRA to track changes. This system allows us to collaborate more effectively as a team, with conversations displayed in a more organized fashion and statuses clearly defined.
7. Use and share our past experiences (If you find something that works at bedtime, re-use it on subsequent nights). We standardize our code and tools where possible and always apply our experiences to each project. We develop shared reusable mapping and code, e.g., extensive documentation on our tools and best practices, robust map and code base for Raiser’s Edge migrations, tools to analyze IMIS customizations, Fonteva conversion documentation, and functions to help manage data manipulations, etc. We are constantly updating our resources to increase our projects’ efficiency.
8. Transparency regarding change impact on the project (Set expectations with your child about when bedtime is happening). It is crucial to keep the team informed of any changes to the schedule. Implementations have several dependencies to the migration, so everyone must know how the migration is going.
9. Not everyone loves technical details (Every child is different and responds to different approaches). While some might enjoy watching us work our programming magic, others want data issues broken down into functional terms. We work with our project teams to determine the best approach based on your team.
10. Data security (We protect our children in every way we can). Data security has never been so important. Our team is hyperaware of and trained to manage extremely sensitive information. Our teams participate in several trainings throughout the year to stay updated on data security and are internal experts on data security.

Hopefully, this blog gives you a peek into the world of data migrations at Fionta (and some of my parenting experiences). I enjoy what I do daily as it is incredibly satisfying to work out data puzzles and solve complex problems. Our clients are amazing people who are changing the world in extraordinary ways. When I find a clever way to minimize cost, meet a schedule, organize complex data, increase efficiency, and generally do my best, I find fulfillment as I contribute to these inspiring causes in my own small way. At the end of the day, when I put my boys to bed, I know that I’ve helped to improve the world for them.

With this motivation at Fionta, we strive to provide superior data services to the organizations we serve. We know this process can be daunting. Sometimes, the data is covered in mud, naked, and screaming, refusing to get into their pajamas and go to bed. We get them there with patience, love, hard work, and experience. If we help you wrangle your wild data child one day, know that you and your organization will be well cared for by our collaborative and experienced team.