MSP Strategies For Reducing Data Migration Costs To And From The Cloud
By Greg Holick, IMT
For years, solutions providers have searched for the most efficient and cost-effective way to manage and automate data movement from on-premises IT environments to the cloud and back. In particular, it has been in the interest of managed service providers to become more involved in the revenue stream generated by these events. Additionally, there has been the need to lower the operating expenses of ground-to-cloud data migration as many organizations make the move from on-site IT for more agile cloud services.
The business disruptions brought about by the Coronavirus are moving this front and center as many businesses are moving toward utility- or cloud-based IT services to reduce the high cost of traditional IT investments and services -- scaling back IT expenditures as business demand requires. The ability to track overall IT spend more closely so that users better understand the strategically important systems and what areas are unnecessarily driving costs is expected to increase in priority over the next several years as businesses streamline their operations.
According to recent research by Technavio, the cloud migration services market is expected to grow by $7.1 billion during 2020-2024. The growing adoption of cloud-based migration is encouraging the development of a wide range of innovative solutions specifically targeted to businesses and other organizations as they move to reduce expenditures. This has been one of the primary factors driving global cloud migration services market growth.
The acceleration of digitized communications and the resulting content is creating vast amounts of data which has to be processed, managed, and migrated to enhance revenue-generating potential. After this growing volume of data has been reviewed and analyzed, it must be stored for long term accessibility. Service providers today are looking toward cloud infrastructure for the storage and administration of such information to complement and inevitably supplant portions of traditional in-house applications and systems.
Unfamiliar with the pricing schedule and caveats found with MSP-based cloud environments or public cloud provider storage and processing services, many customers find that the removal or transition of data to and from the cloud to be not only complex but very expensive in comparison to local production operations. Being able to weigh the pros and cons of using these services compared to on-premise operations is an important consideration for many customers. Until now, there have been too few tools available to help IT solutions providers and their customers find the right mix of solutions.
According to Tom Coughlin, President of Coughlin Associates, "Data is the life-blood of modern organizations and moving that data around is critical to the organization’s operations. For that reason, data mover software that controls the migration of data, as well as its movement between clouds and on-premises operations, is a vital operational tool. This is especially important during the COVID-19 pandemic when most work is being done remotely. These tools must be able to help organizations balance the capital costs for on-premises storage and processing against the operating expenses (OpEx) of storage and processing in the public cloud." *
The requirements of organizations today call for solutions that predict the cost and speed of data transfer to and from the cloud and that saves time and money. A new breed of data management software is putting users in control of their budgets and providing insights for making smarter storage decisions. Developers of these solutions have designed products that work with all types of storage – on-premise, hybrid, and cloud. Users can define detailed policies or manually transfer data. Add to this the ability to automatically monitor and track every job, generate reports on demand for departmental chargebacks and client billing, and flexible integration with 3rd party applications to bring alignment with a variety of workflows - and organizations are compelled to act. Next-generation solutions even go so far as to offer unlimited data movement and subscription-based pricing to support a broader range of customers.
"When using remote cloud-based operations, movement of data is required. Much of this locally generated data is unstructured and needs to be moved to a private data center (private cloud) or a public data center for analysis and perhaps for longer-term storage (including disaster recovery, data protection and archiving)," added Coughlin. "Data movers may also be used to move data out of the cloud or between various remote data centers (multi-cloud operations). The costs for moving data often depend upon the required performance levels but are ultimately gated by the organization’s network performance."
Select systems allow for test-runs that deliver predictive analytics so that customers can model the cost and duration of proposed data migrations to and from the cloud. In these cases, the producer's content can be moved automatically and efficiently, according to a policy-based migration. The more effective data movers oversee cost and data migration resource visibility. AV production may use this information to help determine which of the available cloud environments are most appropriate, not only in terms of cost but also services and performance levels that meet customer service level agreements (SLAs).
MSPs can benefit immensely from new data mover services in support of their customers. As many business operations have only been possible during the COVID-19 pandemic because projects have been orchestrated remotely, often using cloud-enabled tools. These environments are using video proxies and collaborative work solutions to facilitate workforces nationally, and often internationally. When shifting workloads to and from the cloud, the industry's newest data movement tools allow MSPs to gain a percentage of the cloud transfer budget to monitor how individual file transfers affect costs with a real-time dashboard. A budget limit can be set to prevent excess spend. Data transfer jobs are tracked and itemized so that chargeback routines can be implemented.
Because of various performance allocated storage environments and data transfer performance rates involved in data migration and increased use of multiple clouds for various operations, the process of calculating costs for different options can be complex and expensive. At the same time, a larger number of organizations are sensitive about the costs of operations and, for a larger organization with multiple cost centers, being able to allocate fees to the appropriate cost center is critical for sustainable operations.
Cloud data services charge differently for various types of storage, depending upon the speed
available to access the data and for large movements of data, they also charge different fees for bulk data movement, standard data movement, and expedited data movement. Realizing the options available among different services and organizations is critical to vetting the preferred vendor. Data movement solutions must integrate with cloud-native APIs to move past the need for constant integrations and management of the process.
In one example of a sports organization that relies heavily on video content captured from games, practices, and interviews, their legacy storage infrastructure could not support the move to 4k and 8k video nor the performance and storage requirements needed. The team needed a simple and cost-effective way to migrate off their legacy storage system. Using an advanced data mover solution, they were able to tackle their Tier 1 cloud migration projects leveraging the elasticity of the cloud and allowing them to bring their data archive online. The data movement software was installed and configured in a VM environment. It enabled content migration from legacy storage to a new and better-performing storage environment.
The data mover solution was integrated with a media asset manager (MAM) and provided with automated policy-based data movement from on-premise to the public cloud. Furthermore, a dry run feature provided insights into cloud storage and retrieval costs. As a consequence, the organization’s active content is now searchable and available to users and the automated migration reduced required IT resources. This not only saved time and money but minimized on-premises storage costs by using elastic cloud storage coupled with real-time analytics to control the cloud budget and spending.
Electronic sports, or esports using video games, is a rapidly expanding entertainment industry. An active electronic sports company has multiple publicly viewed games and leagues that are inspired by card games, board games, and role-playing fantasy (RPF) games. The media workflows include videos of competitions with players, in-game footage, and commentary. Media is also needed to create in-game video content. This company had disparate islands of data with cloud data stored in proprietary formats and workflows. Content could not be searched quickly and easily, and the company needed more insight into cloud costs.
A management and automation tool was implemented within its data migration platform in a hybrid cloud deployment with archive on ingest into a cloud-based archive. The data mover transitioned data to and from the cloud within the integrated tool. Processing is now completed using lightweight proxies which are streamed into its video editor. As a consequence, its gaming videos, legacy game material, artwork, and promos are now searchable and available to multiple groups. Editors can now work from home (WFH) and edit via proxy and conform on-premise, increasing employee productivity. The real-time cost analytics help control the cloud budget and spending.
Coughlin noted that "Today’s data growth and the availability of fast networks are enabling organizations to combine on-premises storage and processing with cloud and multi-cloud elastic storage and processing resources. The right data migration platform should provide easy to use data movement that allows clients to understand and control the costs of data movement to and from the cloud."
About The Author
Greg Holick Is vice president of product management for IMT Software. Holick oversees all aspects of shaping product direction and roadmap for IMT’s software division and its inaugural product, SoDA. Holick works closely with channel and workflow application partners to bring innovative solutions software tools and applications to customers that automate and simplify business operations, workflows, and hybrid cloud data management.