What does a data engineer do

Data science vs data engineering

The job description ofData Scientsts has only just arrived in Germany, we are facing new job titles. “Is that really necessary?” Many people will ask. But the answer is very clear: yes!

Which data scientist does not know this: a recruiter calls, speaks of a great new challenge for a data scientist as you obviously claim it on your LinkedIn profile, but when discussing the vacancy it quickly becomes clear that you are has almost none of the required skills. This mismatch is mainly due to the fact that the job of the data scientist summarizes all possible job profiles, method and tool knowledge that an individual can hardly learn in his or her life.

Many open jobs that are to be filled under the heading of Data Science rather describe the job description ofData engineers.

Read this article in English:
"Data Scientist vs Data Engineer - What is the Difference?"

What does a data engineer do?

In data engineering, the main aim is to collect or generate, save, historicize, process, enrich and make data available to subsequent instances. A data engineer, often referred to as a big data engineer or big data architect depending on their rank, models scalable database and data flow architectures, develops and improves the IT infrastructure on the hardware and software side, and also deals with topics such as IT security , Data security and data protection. Depending on requirements, a data engineer is partly an administrator of the IT systems and also a software developer, because he or she can add his own components to the software landscape if necessary. In addition to the tasks in the field of ETL / data warehousing, he also carries out analyzes, for example those to examine the data quality or user access.

A data engineer mainly works with databases and data warehousing tools.

A data engineer tends to be a trained engineer / computer scientist and is rather far removed from the actual core business of the company. The career levels of the data engineer are usually:

  1. (Big) data architect
  2. BI Architect
  3. Senior Data Engineer
  4. Data engineer

What does a data scientist do?

Even if there may be many points of overlap with the field of activity of the data engineer, the data scientist can be delimited by using his working hours as much as possible to analyze the available data in an explorative and targeted manner, to visualize the analysis results and to convert them into a red Tighten the thread (storytelling). In contrast to the data engineer, a data scientist rarely sees a data center because he taps into data via interfaces provided by the data engineer.

A data scientist deals with mathematical models, works primarily with statistical processes and applies them to the data in order to generate knowledge. Current methods of data mining, machine learning and predictive modeling should be known to a data scientist, although each of them has their own individual priorities. Data scientists generally work close to the specialist area and need appropriate specialist knowledge. Data scientists work with proprietary tools (e.g. from IBM, SAS or QlikTech) and also program analyzes themselves, e.g. in Scala, Java, Python, Julia or R.

Data scientists can come from a wide variety of academic backgrounds; some are computer scientists or electrical engineers, others are physicists or mathematicians, and quite a few are economists.

  1. Chief Data Scientist
  2. Senior Data Scientist
  3. Data scientist
  4. Data Analyst or Junior Data Scientist

Data scientist vs data analyst

I am often asked what the difference is between a data scientist and a data analyst or whether there is a differentiating criterion at all:

In my experience, the term data scientist stands for the new challenges for the classic term of the data analyst. A data analyst carries out data analyzes like a data scientist, but more complex topics such as predictive analytics and machine learning or artificial intelligence are more for the data scientist. A data scientist is, so to speak, a data analyst ++.

And a business analyst?

Business analysts can (but do not have to) also be data analysts. In any case, they have a very strong connection to the specialist area or the company's core business. Business analytics is about the analysis of business models and business successes. The analysis of business successes in particular is usually IT-supported and this is where many business analysts come in. Dashboards, KPIs and SQL are the tools of the trade of a good business analyst.


Benjamin Aunkofer

Benjamin Aunkofer is lead data scientist at DATANOMIQ and university lecturer for data science and data strategy. In addition, he works as Interim Head of Business Intelligence and gives seminars / workshops on BI, data science and machine learning for companies.

Tags:Chief Data Scientist, Data Engineer, Data Science, Data Scientist, Career