Food For Thought | What is my role?

As consultant I am working with data on a daily basis for various clients. Data is anywhere, everywhere, and available in all kinds of formats. In this day and age, it is essential to get valuable insights out of that data. How data is turned into insights, depends on a number of things; The strategic roadmap, goals, and chosen technology-stack.

In essence, I am there to help our clients turn their data into valuable information, which leads to new and actionable insights. Each client defines my role slightly different. However, tasks and responsibilities often stay the same. So, why does the name of my role change from time to time? Switching from the role of Business Intelligence (BI) consultant to roles such Data Analyst or Data Engineer doesn’t seem the same..., right?

Secondly, the introduction of Big Data and Artificial Intelligence acted as an additional catalyst for accelerated cloud-adoption. The volume of data is growing exponentially and when you combine that with globalization and the ever-increasing storage capabilities and compute power, you end up with a huge untapped potential from which you can derive great value!

And finally, more programming languages have matured and are applicable to perform, amongts others, data transformations. However, this requires one to develop specific skills in order to keep up and even stay ahead of the data-game.

Personally, I feel that over the past few years these 3 developments are the reason that roles are changing. Nevertheless, there is one linking pin between all of these, which is software engineering. Core application (web) development has been there for ages, but with the shift to the cloud, new opportunities pop up. As a BI consultant, I am ready to use software engineering best practices and leverage CI/CD-pipelines whilst increasingly adapting test driven development. That’s why you see BI Developers, ETL developers or DWH developers nowadays working as a Data Engineer,.

Anyways, back to the data and getting value of out it. Are we doing things differently? The answer is simple: No, we are not doing things differently.

Despite varying approaches in developing data platforms and information products, the shared objective remains extracting valuable insights from the data. Looking at the layers of the ‘data hierarchy’, as depicted below, we still do the same thing, but just a bit different.

BI consultants, skilled in both business and technical domains, traverse layers, integrating and comprehending diverse aspects in data-driven roles. Conversely, the Data Engineer focuses on efficiently collecting and organizing data using cutting-edge technology.

Specialization led to Business Analyst (BA) roles bridging business and technical teams, though lacking in-depth expertise in either domain.

Despite collaboration, miscommunications between business and technical domains may occur, delaying information product delivery and hindering business improvement opportunities. Platform engineers ensure teams operate on the right platform, empowering engineers and analysts to contribute expertise and deliver value to the business.

Ultimately, the synergy of BI consultants, Data Engineers, Business Analysts, and Platform Engineers collaboratively strives to extract insights from data, while Data Scientists engage in predictive and prescriptive analyses to unlock further potential.

Another thing, that is not in the picture, but does fit in nicely at the Organize- and Analyze-layer is the introduction of Microsoft Power BI. Microsoft Power BI empowers Business Analysts for analytics on large datasets, facilitating the delivery of profound insights efficiently. It brings the ‘back-end’ BI work to the front of the report and introduces a lot of self-service analytics and possibilities. Microsoft Power BI consultants are a new part of the family that provide data & insights to an organization. Without a solid data architecture, data governance, and master data management, you risk a data swamp instead of valuable insights. So be mindful when going all-in on self-service analytics.

Reflecting on the above, my role or title largely depends on the core work  I do at the client and chosen technology-stack. I can be a Data Engineer, Data Analyst, Data Scientist, or Business Analyst, depending on the requirements at that time. Ultimately, my goal is to assist clients in gaining the best and most valuable insights from their data. No matter what the data platform or my role is. I'm content as long as I can use my skills to unlock untapped data potential and connect with the business.

And remember, with lots of data comes great responsibility!