Where the data comes from

Every number on a report began life somewhere else. Knowing where, and how it travelled, explains most of the surprises in BI, including the most common one: two reports that show different figures for the same thing.

Source systems

A business runs on systems that record its daily work. Each one is built to do a job, not to be analysed.

The till records each sale as it happens.
The accounting system records invoices, payments and the official books.
Payroll records hours and wages.
A booking or ordering system records what customers have asked for.
Spreadsheets hold everything the systems do not: budgets, targets, lists someone maintains by hand.

These are called source systems. BI does not replace them. It reads from them.

How data travels

Reports rarely read a source system directly. Doing so would slow the till down and each system stores data in its own awkward way. Instead, data is copied out on a schedule.

Extract. A copy of the new records is taken, often overnight.
Store. The copies land in one place, such as a database or a data lake.
Shape. They are cleaned and combined into tidy tables.
Refresh. The report reads the tidy tables and updates.

Each step takes time and each step can fail. When a number looks wrong, the cause is usually found by walking back through these four steps.

What “live” really means

People ask for live data. Usually what they need is recent data, and it helps to know the difference.

Live means you see a sale seconds after it happens. This is expensive and rarely needed outside operations screens.
Refreshed hourly or daily is what most reports are, and it is enough for almost every decision.

The important habit is to show the time. A report that says “data up to 6:00 this morning” is trustworthy. One that shows no time leaves people guessing.

Why two reports disagree

When two reports show different numbers for “sales”, one of four things has happened.

Timing. One was refreshed this morning and the other last night. Or one counts a sale on the day of the order and the other on the day of payment.

Definition. One includes tax, the other does not. One removes refunds, the other shows them separately.

Scope. One covers all three cafés, the other leaves out the one that opened last month.

Manual changes. Someone adjusted a figure in a spreadsheet after it was exported.

Notice that none of these is a calculation error. This is why “the numbers are wrong” is rarely true and “the numbers mean different things” usually is.

The questions to ask

For any number you rely on, be able to answer three things.

Where does it come from? Which system, which table.
As of when? The time of the last refresh.
Who owns the definition? The person who can say what is included.

A team that can answer these for its ten most important numbers has solved most of its reporting arguments in advance.

Try it

Pick one number you see every week. Find out its source, its refresh time and its owner. If you cannot find one of the three, you have found a gap worth raising.