Food For Thought | Architectural Guidelines 

Author: Benito van Breugel

In our previous Food For Thought blog we have dived into the business question where to put your data platform, in the cloud or on-premise. However, no matter what you choose, in each situation the architectural guidelines are essential for a consistent solution that will add significant business value. How this technically can be achieved, will be addressed in the next Food For Tech blog, but first, let’s put focus on the business side.

A data solution that serves the organization comes in many ways, defining a good architecture and respective guidelines are key in getting the highest value out of your data & analytics solution. But why is this important? If the end users just get wat they ask for, is that good enough? If the financial controller can report the correct numbers, or, if the internal buyer can find the right pricing overview out of the ERP system, will that be enough to do their job? From a simple point of view, they maybe have enough information to do their job. However, how do you guarantee that the information domains and values have the same definition? How do you get a common understanding of the data across domains? Above all, how do you handle change, security and new requests? If there is no consensus on the architectural guidelines, there will be no sustainable solution that can cope with these matters.

First, your organization should be ready to implement and make effective use of a data & analytics solution. The organizational architecture describes the teams, roles and responsibilities in order to have a clear structure that will enable a data & analytics solution that suits your requirements. Do you want to focus on financial reporting, based on your ERP system(s), or do you want to steer on (near-) real-time data and predict/determine your next decisions almost instantly? These choices will require a different organizational architecture to produce the best outcome in the end.

Data management will enable the effective use of data in the business by describing definitions, naming conventions, business processes and complexity. This will help to understand how data entities are being leveraged by business domains, how they are managed and where they are located. Better data management can improve data quality in some cases. However, the “garbage in is garbage out” principle, remains. Be explicit in the conventions set for the data fields. For example, the rule that every numeric field should be noted as “accountnr”, should be consistent overall. Do not accept other fields like “tax_nr”, “customernum” or “client_number”. Modelling a consistent solution leads to better understanding and more effective use of your data solution.

Data migration & transformation focusses on describing the ability to transform the data from sources, data definitions and business requirements, while keeping data quality at a high standard. It defines where transformations should take place and how changes in data sources are accepted. Also, this defines migration and recovery processes in case of data disruption. Data governance is about the security components of the data. Next to this, it describes; who is responsible for accessing data, who is able to receive data and who is owning the data definitions across the company. Functionally, this is tackled by assigning specific data owners and data stewards in the organization to own these topics.

Third, A clear platform architecture that is enterprise ready, will prevent changing priorities and eventually will save you money in the long term. An organization should know what they want and actually require. Do not spend time on demos (of tools) or concepts that just partly fit the strategy. In the end, a platform architecture should be scalable, flexible, and robust. In other words, it should be designed to easily cope with changes whilst remaining stable, available and cost-effective. Above all, in case of any new requirements, the platform can be easily and effectively adjusted or extended with new features. As an example, if your manufacturing execution systems (MES) are key in producing your product, it will be interesting to gain real-time insights from that system and combine it with other data-domains on the fly, rather than getting the data with a delay of minimal 1 day through batch-data-processing. This will allow you to act quickly on any issues in the day-to-day-operation and prevent large callbacks or corrections afterwards. Your platform should be designed such that it has the ability to easily be extended, in order to support real-time-data-processing in addition to batch-data-processing. Moreover, accept that change is the only constant, and data requirements will change from descriptive to predictive and prescriptive analytics. Your platform should be ready for it.

Defining an overall architectural strategy for your data & analytics platform is not the first topic on your list, if you want to gain insights from your data. However, having an overall architectural strategy is crucial. This will help define structure, be more cost-effective, and preventing miscommunication in your data journey. Moreover, an overall architectural strategy will enable a clear vision for the whole organization and applies focus on working towards the “one source of truth concept”.

If your organization is ready for the next step in this journey, it is time to focus on the technical “how”. In our next Food For Tech-blog we will focus on the technical implementation Food For Analytics offers that enables a scalable, flexible, and robust platform architecture for your organization.

Stay tuned!