CLOUD ADOPTION : AN OPPORTUNITY TO TRANSFORM DATA MANAGEMENT

Cloud provides agility, scalability and power. This is enabling organizations to adapt to changing business models and markets whilst connecting to the ever-expanding external data ecosystem. It is expected that nine out of ten companies will have elements of their applications or infrastructure in the cloud by the end of 2019. We also know that financial services are particularly feeling the pressure. 

As the charge towards the cloud continues, it’s easy to forget the lessons learned from multiple failed big data and data lake projects of the past. Migrating to the cloud is no different – and the challenges remain the same. Some of the key benefits of the cloud can also become its biggest risks. With a low cost of storage and relative ease of data collection and ingestion, comes the temptation to blindly load data without knowing the end goals of the business, in doing so losing context, semantics and value. These problems are addressed through solid data management and governance. It comes as no surprise that effectively implementing cloud governance was cited as the key challenge for cloud-adopting enterprises in 2019.  

Banks should seek consistency in data governance across on-premises, cloud and hybrid architectures, through adjusting pre-established policies and governance to include cloud requirements. They should also use this opportunity to re-think enterprise data management and realise the full value of organizational data, through leveraging the toolsets and compute power of the cloud. 

In this article, we’ll examine three of the key challenges facing Chief Data Officers and enterprise architects – and how they can be addressed through data management and the cloud. 

Keeping data relevant and navigable

Data producers and users should be brought along the cloud journey from the start, to ensure that the correct metadata and context is attached to the data. Cataloguing and metadata management should be enforced in organizational policies, but also promoted as a key enabler for the democratization of data in the institution. Existing data cataloguing infrastructure may be extensible to the cloud. If this is not yet in place, a cloud-ready solution should be considered as part of the cloud migration planning.

Upon ingestion to the cloud, physical data should resolve to an ontology, mapping to business semantics captured under governance by data owners. Banks will also need to establish a model for consumers to further curate and enrich these semantics. This will create real business benefits - increasing the ease with which the correct data can be found and analyzed by the userbase, and feeding back its uses across the bank to the data producers. This feedback mechanism allows the data producers to ensure that the data is being used in the right way, by the intended audience

Ensuring data reaches the intended audience with the right context

Smart adoption of cloud makes the consolidation and analysis of business data more accessible than ever. Consequently, security and privacy considerations must be re-evaluated and accounted for. With the proliferation of regulations like GDPR, information security and compliance with ISO 27k standards are now at the forefront of any cloud migration as a shared responsibility between organizations and cloud vendors.

Banks must ensure that business users receive data with the context relevant to them, or else run the risk of users making misinformed decisions by analyzing data with insufficient context. As cloud expands across the organization, and business users increase, technology departments must be ready to deliver access to datasets that support a diverse number of use cases. Different users and applications may require entirely different views of the same dataset, depending on the context in which it is used. The only way to truly meet this demand is through the curation of the data ontology, enriching the data catalogue and evolving in time to support multiple unique user views.

Data owners should establish data lifespans in the cloud. Once data has reached the end of its useful life and any data retention timescales are met, records should be archived or disposed so that they are no longer used to drive decision-making. Cloud infrastructures allow for smart ramp-down of resources - reducing risk and optimizing cost.

Maintaining and optimizing data quality in the cloud

The business decisions made from data are only as good as its quality permits. Therefore, monitoring and maintaining data quality is imperative for successful cloud adoption. Prior to any migration activities, banks must define consistent data quality standards and taxonomies for both on-premises and cloud data, before implementing tooling to support the enforcement of these standards. Data quality platforms must be able to handle a huge volume of data, and enable quality monitoring across both streaming (event-driven) and batch jobs.

Modern data quality platforms provide smart machine learning algorithms, made readily achievable through the vast compute power of the cloud. These algorithms provide the ability to measure and score data quality whilst autonomously learning new data quality rules. This supersedes the time-consuming exercise of manually defining bespoke rules, which can quickly become obsolete. Instead, there is a move towards a self-sustaining and ever-improving trusted source of data. 

Good data management is a start. However, ensuring employees know how to harness and use data without introducing bias, is a major hurdle which organizations will need to overcome in order to prevent poor quality insights. Banks will need to train and enable employees to use data to support decision-making and reap the benefits of their investment, through a focused cultural change initiative driven from the top. Look out for our upcoming insights on how banks can develop an effective and sustainable culture of data and analytics.

Capco has deep expertise in data management, cloud architecture and machine learning. Let us help you leverage your cloud journey to transform your data management capability and optimize cost, mitigate risk and improve organizational decision-making.