What is a Data Pipeline?

I’ve been seeing some questions about data pipelines lately. I realized I haven’t written a post that gives the level of detail necessary for a good definition of a data pipeline in the context of data engineering. Instead of just giving my opinion, I’ve brought...

The Two Types of Data Engineering

There are two different types of data engineering. There are two different types of job types with the title data engineer. This is especially confusing to organizations and individuals who are starting out learning about data engineering. This confusion leads to the...

Why Real-time is the Future

One of the benefits of teaching and consulting is the sheer number of organizations, teams, and people I get to work with. Since I deal with so many different groups, I can see patterns emerge much faster than others. One pattern I saw early on was real-time Big Data....