During my consulting experience, I met many customers who were experiencing a significant data growth in their systems. One of the most interesting findings was that not all data generated by people, system, sensors or other external sources is actually used.
In order to understand which data is HOT – accessed, used and changed the most, WARM – aged data accessed rarely, COLD – aged data that do not change any more, a good visualization is very important. This is especially necessary when convincing the data owners and business to take action or to decide what to do with WARM and COLD data that generates lot of costs but low value. Usually cold data or even cold applications are retired from the system, or alternatively cold and warm data is archived in a cheaper storage that will consequently improve the performance of analysis over the HOT data.
We were looking for ways to visualize the HOT, WARM and COLD data in SAP systems. As a conclusion Datavrd has developed SAP Fiori visualization using a Scatter plot chart. In our example we are visualizing HOT, WARM and COLD data in SAP FI module.
What we can see from the Scatter Plot chart:
- Most data in the system is accessed rarely (most dots on the scatterplot chart are in the cold area -> left corridor)
- LEFT-TOP corner -> Financial data accessed rarely with large amount of data
- LEFT-BOTTOM corner -> Financial data accessed rarely with small data volume
- RIGHT-TOP corner -> Financial data accessed often with large data volume
- RIGHT-BOTTOM corner -> Financial data accessed often with small data volume
USE CASE: WHAT IS THIS VISUALIZATION GOOD FOR?
A typical use case to use this visualization is data management. In our example all FI object in the LEFT-TOP corner would be the best candidates for archiving (biggest objects accessed rarely). Another example is archiving of COLD and WARM data in business warehouse systems. For example, in SAP BW data the visualization of HOT WARM and COLD data is used to identify data partitions of SAP BW InfoProviders that could be offloaded from SAP BW system to a cheaper storage such as Hadoop using the SAP Nearline Storage interface that is available in all SAP BW NetWeaver systems.
ALTERNATIVE VISUALIZATION TECHNIQUES
The above example shows a scatterplot chart to visualize HOT WARM and COLD data in SAP Fiori web based application. Showing data on this kind of chart is a fast way of understanding your current position or amount of COLD data compared to the data that is often accessed and used. When it comes to showing the independent data objects and identification hot, warm and cold partitions within these objects we can use a Sparkline charts in Microsoft Excel which is the easiest way of showing unused, aged and infrequently queried data. Please see below is the example from an SAP BW system, which shows a list of InfoProviders divided into a yearly partitions to visualize how users are querying old partitions of the same InfoProviders.
WEBINAR: What is going on in your SAP landscape
Date: Monday 3rd April, 2pm SGT | 8 am CET
Speaker: Tim Oehme, Solution Architect at Datavard