Data Engineering Technology Tree

“What we know is a drop, what we don’t know is an ocean.” ― Isaac Newton

Data engineering is one of the disciplines where you just know a drop. Some companies are saying it’s easy, a…

Data Teams Survey Results

Between August 19, 2020, and October 17, 2020, I ran a survey to get more data for my latest book Data Teams. Overall, we had 86 respondents.
This survey was designed to get information abo…

What It Looks Like When a Team Is Missing

Data teams require all of their parts to be complete and succeed. When one of the teams of a data team is missing, the other teams will suffer.
Often, organizations or team members don’t und…

Announcement: Data Teams Is Out!

I’m thrilled to announce that Data Teams: A unified management model for successful data-focused teams is available for purchase!  My goal is to drive a real increase in the percentage of s…

Kafka’s Got a Brand-New Poll

Kafka 2.0 added a new poll() method that takes a Duration as an argument. The previous poll() took a long as an argument. The differences between the two polls don’t stop there. You should know about the differences before porting your poll from a long to a Duration. In general, an overloaded method should have […]

Saving Money on Data Engineering in the Cloud

In my last post, I gave some general suggestions on how analytics and data engineering teams should be dealing with COVID-19. Now, I want to give specific advice on how data engineering teams…

Big Data and Analytics in the COVID-19 Era

Big Data and analytics are going to change in this COVID-19 era. I want to share with everyone the same messages that I’m giving my own clients. I’m hoping that this post will help those data…

I Come Not To Bury Cloudera But To Praise It

It’s been a tumultuous past few weeks for big data vendors. First MapR is having problems (their update). Now, Cloudera is having problems.
As of today, Cloudera closed at $5.21 (June …