The Data You're Not Looking At

The WWII lesson that will change how you analyze data

During World War II, the military had a big problem. They were losing too many planes in battle.

So they brought in a team of math experts to help. One of them was Abraham Wald.

The military wanted to add armor to their planes. But armor is heavy. Too much armor makes the plane slow and burns more fuel.

They needed to be smart. Where should they put the armor to protect the plane the most?

They studied the planes that came back from battle. They marked where the bullet holes were. Most of the holes were on the wings and the tail.

The answer seemed obvious. Put the armor where the bullet holes are.

But Wald said no.

Put the armor where the bullet holes aren’t, like the engine.

Everyone was confused. The engines didn’t have many bullet holes. Why armor them?

Wald explained it in a simple way. You are only looking at the planes that made it back. What about the planes that didn’t?

Planes hit in the wings came home. Planes hit in the tail came home.

But the planes that got hit in the engine? They crashed. They never made it back to base.

The bullet holes you can see show where a plane can get hit and still survive. The missing bullet holes show you where a plane gets hit and dies.

We Do This Too

We do the same thing with data. We look at the survey responses we got. But we forget about the people who never answered.

We study the customers who bought our product. But we ignore the ones who looked and walked away.

We analyze the employees who stayed at the company. But we don’t ask the ones who left why they quit.

We’re looking at the planes that came back. We’re missing the ones that crashed.

Listen to What's Not There

Let me show you what I mean.

Your team sends out a survey. You get 1,000 responses. You analyze them and find out that 90% of people are happy with your product. Amazing, right?

Wait.

How many people got the survey? 10,000.

That means 9,000 people didn’t even bother to reply.

Those 9,000 silent people might be so frustrated they did not want to reply. They might already be looking for a different product. They might be your real problem.

But if you only look at the 1,000 responses, you’ll never know.

The empty rows are trying to tell you something. You just have to listen.

How To See What Everyone Else Misses

You don’t need fancy tools to fix this. You just need to change how you look at the screen.

Every time you analyze data, ask yourself one simple question: Who or what is NOT in this dataset?

Here are some ways to find the answer.

1. Compare your data to the whole group. If you have 500 survey responses, check how many people you sent the survey to. The gap between those two numbers is important.

2. Look at your funnel. Don’t just celebrate the people who converted. Study the people who dropped off at each step. Where did they leave? Why did they leave?

3. Check for patterns in who’s missing. Are certain types of people not showing up in your data? Maybe younger users aren’t responding. Maybe people from certain regions aren’t buying. That’s a clue.

4. Ask why people opted out. If someone unsubscribed from your emails or left your platform, find out why. The people who leave often know more than the people who stay.

Beyond The Chart

Anyone can make a chart based on the numbers in front of them. That is the easy part.

But the best analysts are the ones who notice what is missing. They look at the empty rows and realize something is wrong.

Don’t just count the hands that are raised. Ask why the other hands are down.