Food For Thought | What is my role?
Author: Benito van Breugel
As consultant I am working with data on a daily basis for various clients. Data is anywhere, everywhere, and available in all kinds of formats. In this day and age, it is essential to get valuable insights out of that data. How data is turned into insights, depends on a number of things; The strategic roadmap, goals, and chosen technology-stack.
In essence, I am there to help our clients turn their data into valuable information, which leads to new and actionable insights. Each client defines my role slightly different. However, tasks and responsibilities often stay the same. So, why does the name of my role change from time to time? Switching from the role of Business Intelligence (BI) consultant to roles such Data Analyst or Data Engineer doesn’t seem the same..., right?
Secondly, the introduction of Big Data and Artificial Intelligence acted as an additional catalyst for accelerated cloud-adoption. The volume of data is growing exponentially and when you combine that with globalization and the ever-increasing storage capabilities and compute power, you end up with a huge untapped potential from which you can derive great value!
And finally, more programming languages have matured and are applicable to perform, amongts others, data transformations. However, this requires one to develop specific skills in order to keep up and even stay ahead of the data-game.
Personally, I feel that over the past few years these 3 developments are the reason that roles are changing. Nevertheless, there is one linking pin between all of these, which is software engineering. Core application (web) development has been there for ages, but with the shift to the cloud, new opportunities pop up. As a BI consultant, I am ready to use software engineering best practices and leverage CI/CD-pipelines whilst increasingly adapting test driven development. That’s why you see BI Developers, ETL developers or DWH developers nowadays working as a Data Engineer,.
Anyways, back to the data and getting value of out it. Are we doing things differently? The answer is simple: No, we are not doing things differently. While we develop data-platforms and information products in a different way, we still need to get insights out of that same data. Looking at the layers of the ‘data hierarchy’, as depicted below, we still do the same thing, but just a bit different.
While a BI consultant is (has been) working on almost all layers and thereby is required to have an excellent understanding of both the business- and technical-domain, a Data Engineer focusses solely on collecting and organizing data by leveraging the latest and greatest tech. Therefore, you also see a growing number of Business Analyst-(BA)-roles in the teams that serve as the linking pin between business-stakeholders and their respective team. However, in general, the BA lacks the excellent understanding of the technical-domain and the engineer that of the business-domain. Hence, the functional requirements of a business stakeholder sometimes get lost in translation. Which could potentially leads to a delay in the delivery of an information product and a missed opportunity to improve the business. Platform engineers enable the teams to develop on the right platform and platform-components. Simply put, they make sure the engineers and analyst can do what they do best and deliver value to the business. Eventually they all work together to get insights out of the data. Moreover, predictive and prescriptive analyses are performed by the Data Scientists that use that data.
Another thing, that is not in the picture, but does fit in nicely at the Organize- and Analyze-layer is the introduction of Microsoft Power BI. Microsoft Power BI is a great tool that enables a BA to perform analytics on large datasets and provide great insights. It brings the ‘back-end’ BI work to the front of the report and introduces a lot of self-service analytics and possibilities. Microsoft Power BI consultants are a new part of the family that provide data & insights to an organization. However, without a proper data-architecture, data-governance and master data-management in place you’ll end up with a data swamp rather than valuable insights. So be mindful when going all-in on self-service analytics.
Reflecting on the above, my role or title largely depends on the core work I do at the client and chosen technology-stack. I can be a Data Engineer, Data Analyst, Data Scientist or Business Analyst, depending on what is asked at that point in time. But in the end, my goal will always be to help the client gain the best and most valuable insights out of its data. No matter what the data platform or my role is. As longs as I can use my skills to unlock an organization’s untapped data potential and connect with the business I am happy with whatever you name it.
And remember, with lots of data comes great responsibility!