The Numbers Are Skewed. And?

A caveat is useful when it changes the decision. Otherwise it may just be a well-dressed reason to stop thinking.

By Lisa Ramirez · The Sensible Plan

“The numbers are skewed.”

Sometimes they are.

Fine.

Then I want three answers: What is causing it? How much does it matter? Does it change the decision?

Because “skewed” is not a conclusion. It is the beginning of the useful bit.

Correct is not the same as useful

Data is rarely perfect. One campaign started earlier. One market is larger. One audience had more time to respond. A small group behaves very differently from the rest. A tracking change has made last month and this month less tidy to compare.

All of those things can be true.

The question is whether they materially alter what we should do next.

There is a peculiar comfort in identifying a flaw. Once somebody has announced that the numbers are skewed, the room can nod gravely and avoid the inconvenience of making a decision.

It sounds analytical. It may even be technically correct.

It is still not enough.

Find the source of the skew

Start by naming the problem precisely.

Not “the data is messy.” Not “the audiences are different.” What, exactly, is different?

Is one channel carrying more branded traffic? Did one location receive more budget? Are returning users mixed with first-time visitors? Did a promotion change the offer halfway through the period? Is a very small sample producing a very dramatic percentage?

The more specific the cause, the easier it is to test its effect.

Vague caveats grow. Specific ones can be measured.

Separate distortion from inconvenience

Some differences genuinely invalidate a comparison. Others merely make it untidy.

If one group received a fundamentally different offer, combining the results may conceal the thing we are trying to learn. Split the groups and look again.

If one segment is overrepresented but the same pattern appears across every meaningful segment, the overall number may be skewed while the decision remains exactly the same.

That distinction matters. We do not need perfect data to make every decision. We need data that is fit for the decision in front of us.

Run the decision test

Before allowing a caveat to stop the work, I use a simple test:

Remove it. Exclude the questionable segment, channel or time period. Does the direction of the result change?

Separate it. Look at the relevant groups on their own. Do they tell materially different stories?

Stress it. Assume the skew is larger than we think. Would we make a different choice?

Set a threshold. What would the result need to be before we changed course?

If the decision survives all four, the caveat belongs in the footnote—not in the driver’s seat.

If the decision changes, excellent. The caveat has done some work. Now we know what needs to be corrected, retested or treated separately.

Imperfect data still requires judgment

“We need more data” is sometimes the right answer.

It is also one of the easiest answers to give when nobody wants to own the next move.

More data has a cost. It takes time, delays action and can create the comforting illusion that certainty is just one dashboard away.

Usually it is not.

The job is to understand the limitation, decide how much uncertainty matters and make the best available choice without pretending the number knows more than it does.

That is not lowering the analytical standard. It is using analysis for its actual purpose.

To decide.

The numbers may be skewed. The useful question is whether the decision is.