Moving IT to the cloud is one of the main objectives for many companies. But once achieved, they find managing data in a hybrid environment a challenge. This is when metadata management becomes more important than ever.
Cloud vs. on-premises? It’s one of those topics that you hear broached at tech conferences, business meetings, even cocktail parties (but very boring ones). There are positives and negatives to either – cost vs. control, perceived higher security vs. easier deployment, etc. The arguments are well-known and well-worn but more companies of all sizes are making the move all the time.
Indeed, for many firms a move to a full cloud-based IT environment seems all but inevitable. According to a survey by cloud company Denodo of enterprises large and small, moving to the cloud is not just about saving money – even though companies can significantly reduce costs, eschewing huge investments in computers and servers when they adopt cloud solutions.
According to Forrester, for example, it’s the need for advanced technologies and access to services that drives many to move from on-premises to the cloud. In its report on digital transformation in 2018, Forrester says that for many firms, it’s not a question of “if” to move to the cloud – it’s when. According to the report, more than half of global enterprises in 2018 were set to adopt “at least one public cloud platform to drive digital transformation and delight customers. This is a ‘magic threshold’ signifying the imminent ubiquity of cloud computing and the future of doing business in today’s digital economy.”
Digital Transformation as a Primary Motivator for Moving to the Cloud
For many, it’s about digital transformation – the need to keep up with the latest IT advances and not miss out, in order to ensure that they are as agile as possible in a hyper-competitive environment. “Cloud computing is just a part of digital transformation—the landscape is immense and there’s no one-size-fits-all solution—but it’s a rapidly growing part that decision-makers can’t afford to ignore,” the Forrester report says, adding that it expects the total global public cloud market to continue to grow in the coming years at an annual 22% compound rate.
And cloud service providers – naturally – have been doing all they can to encourage that move, providing all the services, software, ultimate security measures, and systems companies might need. Tried and true tools that have been in use for years in the office – in their on-premises forms – are now available on-line. For example, Microsoft’s BI stack, including SSIS, MS-SQL, Tabular, OLAP, SSRS and PowerBI, are all available for Azure customers. Google, IBM, Oracle, and all the others have been doing the same thing. The idea is to make companies as comfortable as possible with a move to the cloud.
But for many companies, some pieces of the cloud puzzle are still missing – such as the management part of things. Tools are just that – and to get their full value, you have to use them effectively. Legacy, on-premises systems are connected by IT personnel who manage the workflow and ensure that everything melds together for smooth operation. Firms will want the same experience when they move their IT to the cloud.
The BI Team’s Dilemma
One area that concerns companies thinking of a cloud move is data management – which is difficult enough to do in-house, much less in the cloud. Business intelligence teams in charge of data management need to ferret out data that could be stored in many different places – databases for different departments, databases for the entire organization, data storage units for social media information, reporting tools, ETL tools, analysis tools, corporate documents, customer information, etc. – and provide answers based on that data to develop reports, forecasts, statements, comply with regulations, etc.
Finding that data quickly may be crucial – such as when the people who enforce GDPR, for example, come calling, and demand that a company show it has the ability to locate its data and thus comply with “right to be forgotten” rules for example – as the GDPR regulations require. It’s not enough to comply with those rules when a request comes in; according to the regulations, GDPR regulators can require a company to show that it has the ability to document data’s “provenance” – where it was located, what database it was stored in, which tables or cells are involved, etc. Failure to do that could result in EU sanctions, even if no actual request to remove the data came in.
It’s crucial that companies be able to account for inaccuracies that could skew data integrity and accuracy is preserved. Organizations need to be able to find, understand and trust their data – as well as figure out and trace the root of errors when they occur. With so many different systems to go through and inconsistent associated metadata, BI teams often have to search manually for the data they require which can be extremely time consuming.
Customer information, for example, could appear in numerous locations, but labels for the relevant data – type of business, location, sales, etc. – could be different, making a simple search across databases impossible, and even on-premises data tools from Microsoft et al don’t include the tools to easily automate and perform that work. Neither do the online versions.
Metadata Tools to the Rescue
Offering tools to do that – providing cloud services that allow for easy resolution of data legacy questions by resolving metadata issues, with automated searches and easy construction of searches – would help make things easier for firms seeking to move operations online, and closing the gap between the need for better data management and the benefits of cloud-based IT.
Those tools available would give BI teams more power to get things done more quickly; instead of spending days or even weeks searching out relevant data, they could use the cloud tools to easily resolve the metadata issues. Any “butterflies in the stomach” of IT, BI teams, or C-Suite executives, for that matter, would be resolved by this. Fortunately, there are firms that offer these services, even if the tools offered by the big cloud providers don’t.
That the cloud offers a new world of efficiency, agility – and greater profits – is pretty clear at this point. Resolving data legacy issues will encourage even more companies to “reach for the clouds.”
