The Modern Data Platform – THE ANSWER

Modern data platform

In the first part of this Modern Data Platform blog series I discussed the traditional data warehouse and some of the issues that can occur in the period following the initial project.

The blog series continued with an exploration of the 5 common challenges that I hear from customers.

  1. INFLEXIBILITY
  2. COMPLEX ARCHITECTURE
  3. SLOW PERFORMANCE
  4. OLD TECHNOLOGY
  5. LACK OF GOVERNANCE

Each blog concluded with a summary of the challenge and a broad outline of what a solution would need to deliver. If you have read the previous blogs, then I hope you can see why I believe that we need to take a different approach.

But it isn’t just us at itelligence who think this way, take a look at the following quotes from Gartner and IDC.

  • Gartner 1 “Emerging data sources, trends and technologies challenge the effectiveness of data warehouses in supporting analysis and decision making.”
  • IDC 2: “The data warehousing market based on relational databases will continue to be disrupted by several non-relational and/or non-schematic information management software categories. Data warehouses will not disappear as they have a key place in an organization’s data architecture.”

*1 ”2016 Strategic Roadmap for Modernizing Your Data Warehouse Initiatives” Mark Beyer and Lakshmi Randall, Gartner, October 2016

*2 Worldwide Business Analytics Software Forecast, 2016–2019 by Dan Vesset et al, IDC, July 2016. Doc # 257402

At the end of each blog post I teased a webinar session where we would “unveil” the technical components that underpin the Modern Data Platform. On the webinar I was joined by my colleague David Milburn who described in detail the construction of the Modern Data Platform and more importantly explained how it addressed the 5 challenges.

This post pulls back the curtain on the MODERN DATA PLATFORM by summarising the content from the webinar. We explain what it is, what makes it different, and most importantly, how it addresses the 5 common challenges.

Introducing the Modern Data Platform

It is worth reminding ourselves of the broad capabilities that we have envisaged for the Modern Data Platform.

We need a Data Pipeline that is capable of ingesting and storing data from an increasingly large variety of sources ranging from structured databases to unstructured files or web feeds.

Next we need to be able to process this data to deliver instantaneous query responses that are independent of the data types.

Finally, we need to ensure that we can support advanced analytics scenarios without needlessly copying and transporting data to other systems.

MDP - Capabilities

(Please click on image to enlarge)

The Modern Data Platform – Conceptual Architecture

If we look beyond the core capabilities towards a conceptual architecture, then we can begin to see how the various parts of the platform come together to address to overall requirements.

MDP - Conceptual Architecture

(Please click on image to enlarge)

At the core is a unified in-memory data management and processing capability that has been built from the ground up to leverage the advances brought by modern hardware platforms. We also have a dynamic data tiering capability to address scenarios where we do not need instant access to all of our data. In this case we can integrate with and leverage lower cost data storage options such as Hadoop.

We surround this core with a unified metadata and modelling capability that accesses the desired data sources through a consistent suite of access services.

Finally, we wrap this all in an end-to-end operations framework that allows us to seamlessly orchestrate, manage and optimise the platform.

The Modern Data Platform – The Core, SAP HANA

It should come as no surprise that the unified in memory data management and processing core of our Modern Data Platform is realised through SAP HANA.

Over 5 years after its original launch and with countless successful implementations behind us, we have no hesitation in proclaiming SAP HANA as the best choice for the Modern Data Platform.

The modern data platform needs to provide much more than just core database and query services. SAP HANA not only extends database services way beyond most competitor platforms, but also includes advanced Application, Processing and Integration Services that are consistent with a full platform rather than a rag tag collection of disparate technologies.

MDP - The Core SAP HANA

(Please click on image to enlarge)

Once again though, why take our word for it. The Analysts agree that SAP HANA is a step above and beyond comparable platforms that you may consider when building a Modern Data Platform.

MDP - Why SAP HANA

(Please click on image to enlarge)

And if you are laboring under the misapprehension this technology will cost the earth to purchase, then think again. SAP HANA Edge Edition starts from as little as £15k.

Leveraging Big Data

One of the key factors that results in complex architectures is the way in which solutions have typically evolved to address new requirements. A common example of this is the increasing desire to leverage “Big Data” technologies.

MDP - Vora

(Please click on image to enlarge)

A modern data platform should not only integrate with Hadoop; it should be able to interoperate with it. SAP Vora has been developed to serve as a bridge between the HANA core and a Hadoop cluster to facilitate high performance advanced analytics that can seamlessly access data wherever it is stored.

Modern Data Platform – Reference Architecture

The third blog in the series introduced the concept of the “complex” or “extended” architecture whereby newer technologies are added to an existing architecture in order to address new requirements.

l

(Please click on image to enlarge)

The reference architecture below shows how the SAP Modern Data Platform simplifies this architecture without compromising the ability to interoperate with existing investments.

MDP - Reference Architecture

(Please click on image to enlarge)

Addressing the 5 Challenges

So how does an SAP HANA based Modern Data Platform help you avoid or minimise the impact of the 5 core challenges we find in traditional data warehouses?

The following serves as a brief summary, for more detail I suggest that you access the recording of the Modern Data Platform Webinar.

INFLEXIBILITY

In the first challenge we discussed inflexibility and three areas which typically create challenges. How does the modern data platform address these types of challenge?

Business are looking for speed of thought, agility and the ability to make decisions as quickly as possible. HANA & Hadoop are enabling flexibility through:

  • Virtual / Federated data access reducing the need to aggregate data for certain use cases
  • Create a mixture of both physical and virtual models to enable flexibility for change
  • An ability to store and analyse data irrespective of structure and form
  • A shift to the “ELT” paradigm which allows data to be stored irrespective of structure and then accessed when we apply the “schema on read”

COMPLEX ARCHITECTURE

Complex architecture is becoming an increasing challenge for organisations resulting in ever increasing costs of ownership and governance challenges. How does the modern data platform help reduce architectural complexity?

The complexity is reduced in the following ways:

  • Co-location of data and application services which process and analyse data in-situ. This reduces a lot of data movement which arises from best of breed solutions and enables a more agile data preparation platform
  • Elimination of the need to model and index data for performance reasons. Because of SAP HANA’s in-memory capabilities and Hadoop’s processing power, building aggregates and cubes for performance reasons are eliminated
  • Provision of a virtual model which enables unstructured data to be accessed as if it is structured data using SAP HANA smart data access and SAP Vora for example

SLOW PERFORMANCE

How does the modern data platform support improved performance for the end user experience?

IT & users don’t like slow performance but if a user cannot do their analysis or consume their report as part of their job the solution will be moth-balled. The modern data platform addresses back-end data preparation and information consumption in the following ways:

  • For SAP ERP customers, the application of S/4 HANA or BW on HANA is a must which enables in-memory analysis of data. From an application perspective this enables operational reporting and analysis on the data without moving it from the ERP solution. For BW, it supports natural extensions to BW for integrating non-ERP application, Hadoop and IOT data to gain richer insight
  • Our ETL processing is improved through a variety of technical innovations such as processing the data where it resides e.g. MAP-REDUCE, virtual models to avoid movement and in-memory computing for real-time analytics. Elimination of redundant tasks such as unnecessary aggregate and index creation also improves processing
  • Provision of a fully scalable platform requiring commodity hardware rather than expensive appliances
  • Improvements in architectural performance from extraction to insight also improves project delivery performance

OLD TECHNOLOGY

What technology advancements are within the Modern Data Platform to support innovation in my business?

  • SAP HANA and Hadoop are written ground up to solve these problems and they are not relational database bolt-ons
  • Starting a Modern Data Platform journey can be done with Cloud/Commodity server storage to operate a “pay as you grow” or Trial enabling an organisation to become familiar with techniques to address a specific business issue
  • Capabilities within the modern data platform enable analytical models to be run where the data resides
  • Columnar storage enables very fast analytics to be performed on data, eliminating traditional ROW/COLUMN issues
  • The modern data platform supports a tiered data architecture from hot in-memory data through to cold mass storage, eliminating the need for archiving potentially useful data
  • The modern data platform provides capabilities to ingest streaming data e.g. IOT data such as Telematics or POS transactions and process this in near real-time to adapt to business models

LACK OF GOVERNANCE

So all this data and technology innovation provides me with an architecture of two components to support my modern data warehousing and business analytics needs. However, what capabilities exist to support my governance initiatives?

  • The modern data platform provides comprehensive security and auditing facilities and enables multi-tenancy solutions, controlled access to tables of data and audit data about events within the overall platform
  • The reduced data movement and transformation that happens with data, enables a simplified data landscape and reduces potential governance issues arising from mistakes, calculation issues and job failures
  • Capabilities for looking at lineage (i.e. what processing has been done on the data between source and target) can be seen in a GUI form and navigated by someone to look at the data processing rules
  • Data stewardship is built into the modern data platform and enables data quality issues to be dealt with by the business stewards

What Next?

If you are still with me, then we can agree that an SAP HANA Modern Data Platform can free your organisation from the constraints of your existing data warehouse.

So what can you do next?

Whether it is a technical presentation exploring the capabilities of the HANA platform, or some help in formulating a business case. If the message of these blogs and the webinar has chimed with you, then my colleagues and I at itelligence are available to help you take the next steps along your journey to adopting a Modern Data Platform.

MDP - What Next

(Please click on image to enlarge)

I hope you agree that the SAP HANA Modern Data Platform delivers on all the requirements for a next generation data warehouse. Enabling organisations just like yours to radically simplify their existing legacy or overly complex solutions in order to lower running costs, improve agility and gain breakthrough performance that delivers real business value, without compromising on governance requirements.

The Modern Data Warehouse webinar recording can tell you even more.

Capture

To discover how the Modern Data Platform can work for your organisation contact us.

 

Similar posts

Blog-Post-SAP-BPC-Right-on-Track-FeatureImage
Read more
Read more
Architecture
Read more
Read more
2-1
Read more
SAP Lumira High Quality Data Visualisation
Read more

Leave a Reply

Your email address will not be published. Required fields are marked *

Follow us: