Loading module…
Loading module…
Module 1 of 6
We live in an age that worships data — "the data shows," "data-driven," "let the numbers speak." But data does not speak; it is spoken for, and the people who understand what data actually is can read it clearly while those who do not are misled by it, sometimes deliberately. This course builds data literacy: the ability to read data and statistics critically, spot misleading presentations and hidden bias, and use data soundly in your own thinking and argument. It requires no mathematical background — the "without math trauma" is a real promise, because data literacy is far more about understanding what data is and how it can mislead than about calculation. This first module establishes the foundation that the whole course builds on: what data actually is, which turns out to be less objective and more constructed than the "let the numbers speak" worship suggests, and understanding which is the first and most important step toward reading data clearly.
The first principle, in the epigraph, is that data is not the world but a set of measurements shaped by human choices — what to measure, of what, how, and why — and every one of those choices shapes what the data can and cannot tell you. This principle is the foundation of data literacy, because it means data is never simply "the facts speaking" but always a constructed representation shaped by choices, and reading data well means understanding those choices and what they did to the data. The data worshipper treats data as objective truth; the data-literate person understands data as constructed measurement and reads it accordingly — seeing the choices behind the numbers, and thus what the numbers can honestly support and where they mislead. Building that understanding — what data actually is, as constructed measurement rather than objective truth — is this module's work and the foundation of the course.
The foundational insight of data literacy, from which everything follows, is that data is constructed — the product of human choices about what and how to measure — rather than objective truth simply read off the world. Grasping this reframes data from "the facts" to "a representation shaped by choices," which is the essential shift that makes critical reading possible.
Every dataset begins with choices about what to measure. The world is infinitely complex, and data captures only some aspects of it — the aspects someone chose to measure. A dataset about a school captures test scores, perhaps, or attendance, or graduation rates — but not the countless other things about the school that were not measured (the quality of relationships, the students' wellbeing, the things hard to quantify). What gets measured is a choice, and that choice shapes what the data can tell you: the data can speak to what was measured and is silent about what was not, so the choice of what to measure determines the data's scope and its blind spots. This first choice — what to measure — is already a human judgment that shapes the data, selecting some aspects of reality to capture and leaving others invisible, and reading data critically starts with asking what was measured and, crucially, what was not.
Every dataset also involves choices about how to measure — the definitions, methods, and instruments that turn aspects of the world into numbers. To measure "poverty," you must define it (income below some threshold? which threshold? accounting for what?) — and the definition shapes the data (different poverty definitions produce different poverty numbers for the same reality). To measure "unemployment," you must define who counts as unemployed (looking for work? for how long? including whom?) — and the definition determines the numbers. These definitional and methodological choices are not neutral: they shape what the data says, so the same underlying reality, measured with different definitions or methods, produces different data. Reading data critically means understanding how things were measured — the definitions and methods — because the "how" shapes the numbers as much as the reality they measure. The how-choices mean that data is not the reality but the reality-as-measured-by-these-choices, which can differ from reality-as-measured-differently.
And every dataset is collected for some reason, by someone — a purpose and a source that shape it. Data is collected by people or organizations for purposes (to answer a question, to support a decision, to make a case), and the purpose and the collector shape the data (what they chose to measure and how reflects their purpose and perspective). This does not make data worthless or mere opinion — data can be a valuable, if constructed, representation of reality — but it means data always comes from somewhere, shaped by someone's choices for some purpose, and reading it critically means understanding that provenance. Together these — the choices of what to measure, how to measure, and for what purpose by whom — make data constructed measurement: a representation of reality shaped by human choices at every step, not objective truth simply read off the world. This is the foundational reframe: data is not "the facts speaking" but "a constructed representation shaped by choices," and the data-literate reader holds this always, reading any data by understanding the choices behind it — what was measured and not, how, and by whom for what purpose — because those choices shape what the data can honestly tell you. This reframe, from data-as-objective-truth to data-as-constructed-measurement, is the foundation of all critical data reading, and it is the first and most important thing data literacy teaches.
Key Insight: Data is constructed measurement, not objective truth — the product of human choices at every step: what to measure (selecting some aspects of infinitely complex reality, leaving others invisible — so ask what was measured and what wasn't), how to measure (definitions and methods that shape the numbers — the same reality measured differently produces different data, as with "poverty" or "unemployment" under different definitions), and for what purpose by whom (provenance that shapes the data). This doesn't make data worthless, but it means data is never "the facts speaking" — it's a representation shaped by choices. The foundational reframe, from data-as-objective-truth to data-as-constructed-measurement, is what makes critical reading possible.
A crucial consequence of data being constructed measurement is that there is always a gap between the measure and the reality it is meant to represent — the number is not the thing, but a proxy for it — and understanding this gap is central to reading data critically, because much data misuse exploits the confusion of the measure with the reality. This section examines the measure-reality gap.
The gap arises because measures are proxies for the realities we care about, and proxies are imperfect. We often care about something hard to measure directly (the quality of education, the health of an economy, the wellbeing of people, the performance of an employee) and measure it via a proxy that can be quantified (test scores for education quality, GDP for economic health, income for wellbeing, sales for employee performance). The proxy stands in for the reality, but it is not the reality — test scores are not education quality (they capture some of it and miss much), GDP is not economic health or wellbeing (it captures production but misses distribution, sustainability, and much else), sales are not employee performance (they capture one dimension and miss others). This gap between the proxy-measure and the underlying reality is fundamental, because the realities we care about usually cannot be measured directly and completely, so we measure proxies that approximate them imperfectly — and the gap between proxy and reality is where much data misreading and misuse lives.
The gap creates several critical reading issues. The proxy captures only part of the reality. Since the measure captures only some of the reality (test scores some of education quality, not all), reading the measure as if it were the whole reality misses what the proxy does not capture — concluding a school is good because test scores are high ignores everything test scores miss. Critical reading holds the gap: the measure tells you about the proxy, which approximates but is not the reality, so conclusions about the reality must account for what the proxy misses. The proxy can diverge from the reality. The proxy and the reality can come apart — the measure can improve while the reality does not, or vice versa — because the proxy is not the reality. This divergence is important because it means a measure moving does not guarantee the reality moving the same way (test scores rising might reflect teaching-to-the-test rather than better education, GDP rising might not reflect improved wellbeing). Critical reading questions whether the measure and reality are actually moving together or diverging.
The gap is also exploited, which makes understanding it defensively important. Goodhart's Law captures a key dynamic: "when a measure becomes a target, it ceases to be a good measure." When a proxy is used as a target (optimize test scores, maximize GDP, hit sales numbers), people optimize the proxy — often in ways that diverge from the underlying reality the proxy was meant to represent — so the proxy improves while the reality may not, or may even worsen. Teaching to the test raises scores (the proxy) without raising education (the reality); gaming sales numbers hits the target without serving customers; optimizing any proxy-target tends to corrupt its connection to the reality. Goodhart's Law is a crucial data-literacy concept because it explains why measures used as targets become misleading (the proxy is optimized, diverging from the reality) and why "the numbers went up" often does not mean "the reality improved" when the numbers are targets. More broadly, the measure-reality gap is exploited whenever someone presents a favorable proxy as if it were the reality (touting the measure that looks good while the reality it imperfectly represents is worse) — a common data-misuse move that understanding the gap defends against. Reading data critically thus centrally involves holding the measure-reality gap: the number is a proxy, imperfect and partial, that can diverge from and be gamed relative to the reality it represents — so conclusions about the reality must account for the gap, and claims that treat the measure as the reality (especially favorable measures, especially targets) must be questioned. This gap-awareness is one of the most important critical-reading skills, because so much data misuse exploits the confusion of the measure with the reality.
Key Insight: There's always a gap between the measure and the reality it represents — measures are imperfect proxies for the realities we care about (test scores for education, GDP for economic health, income for wellbeing), capturing only part and able to diverge from the reality. Critical reading holds the gap: the measure tells you about the proxy, not the whole reality, so account for what it misses and question whether measure and reality actually move together. Goodhart's Law is key: "when a measure becomes a target, it ceases to be a good measure" — optimizing a proxy-target (teaching to the test, gaming sales) improves the proxy while the reality may not. Much data misuse exploits confusing the favorable measure with the reality.
Since data is collected by someone, in some way, the collection process itself profoundly shapes the data — and understanding how data is collected, and how collection introduces limitations and distortions, is essential to reading it critically. This section examines data collection and its effects, because how data was gathered determines much of what it can and cannot tell you.
The foundational collection issue is the sample — who or what is actually in the data. Most data is collected from a sample (some subset of the reality) rather than the whole, and the sample determines what the data represents. If the sample is representative of the whole (a well-drawn sample), the data can speak to the whole; if the sample is unrepresentative (skewed toward some part of the reality), the data speaks only to that skewed part, and generalizing it to the whole misleads. Sampling bias — when the sample is not representative — is one of the most important and common data problems: a survey of website visitors represents website visitors, not everyone (they are a self-selected, unrepresentative sample); a study of volunteers represents volunteers, not the general population; data from those who responded represents responders, not everyone (non-response bias — those who did not respond may differ systematically). Reading data critically centrally involves asking who or what is in the sample and whether they represent the whole the data is being used to speak about — because an unrepresentative sample means the data does not represent what it is claimed to, however large it is (a large unrepresentative sample is still unrepresentative — size does not fix bias).
A classic and instructive example of sampling bias is survivorship bias — drawing conclusions from the survivors while ignoring the non-survivors who are not in the data. Studying successful companies to find success factors ignores the failed companies that may have had the same factors (so the "success factors" may not distinguish success from failure); studying the planes that returned from missions to find where to add armor (the famous example) misleads if you armor where the returning planes were hit, because the planes hit in the fatal spots did not return to be in the data. Survivorship bias is a pervasive, subtle sampling problem: the data contains the survivors, the non-survivors are absent, and conclusions drawn from the present survivors while ignoring the absent non-survivors are systematically wrong. It illustrates the general critical-reading move: ask not just what is in the data but what is absent from it (the non-responders, the non-survivors, the excluded), because what is absent from the data can be as important as what is present, and much misreading comes from ignoring the absent.
Collection also introduces measurement effects and errors that shape the data. How data is collected can affect what is collected: survey questions' wording affects the answers (leading questions produce leading answers, the framing shapes responses), the collection method affects who is reached and how they respond, and measurement instruments have errors and limitations. These collection effects mean the data reflects not just the reality but the collection process — the questions asked, the method used, the instrument's limitations — which can distort what the data says. Reading data critically involves understanding the collection: how was this data gathered, what sample, with what method and questions, subject to what errors — because the collection process shapes the data, and data from a flawed collection (biased sample, leading questions, error-prone method) is flawed data regardless of how it is presented. Together these collection issues — the sample and its representativeness (sampling bias, survivorship bias, non-response bias), what is absent from the data, and the measurement effects and errors of collection — mean that how data was collected profoundly shapes what it can tell you, and critical reading requires understanding the collection to judge what the data actually represents. Data from a good collection (representative sample, sound method) can be trustworthy; data from a flawed collection (biased sample, distorting method) is untrustworthy no matter how it is presented, and the data-literate reader judges the collection, not just the presentation. This collection-awareness — asking where the data came from and how, including what is absent — is a core critical-reading skill, because the collection determines the data's foundation, and a flawed foundation undermines everything built on it.
Key Insight: How data is collected profoundly shapes what it can tell you. The sample is foundational: data usually comes from a subset, and if the sample is unrepresentative (sampling bias), the data speaks only to that skewed part — a website-visitor survey represents visitors, not everyone; a large unrepresentative sample is still unrepresentative (size doesn't fix bias). Ask what's absent, not just present: survivorship bias (conclusions from survivors ignoring absent non-survivors — the returning planes, the successful companies), non-response bias. Collection also introduces measurement effects (question wording, method, instrument error) that distort. Judge the collection, not just the presentation — a flawed collection means flawed data however it's shown, and collection-awareness (where did this come from, how, what's absent?) is a core critical-reading skill.
Different kinds of data can support different kinds of conclusions, and understanding the types of data — and the crucial distinction between what each type can and cannot establish — is essential to reading data critically and using it soundly. This section covers the types of data and their appropriate uses, which prevents the common error of drawing conclusions data cannot support.
A foundational distinction is between quantitative and qualitative data. Quantitative data is numerical (measurements, counts, scores) — it supports quantitative analysis (how much, how many, comparisons, trends) and is what "data" usually connotes. Qualitative data is non-numerical (descriptions, observations, text, testimony) — it supports understanding of meaning, context, and the nature of things in ways numbers cannot. Both are valuable and they answer different questions: quantitative data tells you how much and how many (the scale, the trend, the comparison), qualitative data tells you what and why and how (the meaning, the context, the mechanism). A common error is privileging quantitative data as "real data" while dismissing qualitative data as "anecdote" — but both are data, answering different questions, and the sound approach uses each for what it can tell you (quantitative for scale and comparison, qualitative for meaning and context), often combining them. Understanding this distinction prevents both the over-privileging of numbers (missing what qualitative data reveals) and the misuse of qualitative data for quantitative claims (an anecdote is not a measure of scale).
A crucial distinction for sound conclusions is between correlation and causation — data showing that two things vary together (correlation) versus data establishing that one causes the other (causation). This is perhaps the most important data-literacy distinction, because the confusion of correlation with causation is among the most common and consequential data errors. Correlational data (two things vary together) does not establish causation (one causing the other), because the correlation could be coincidence, could be driven by a third factor causing both, or could run in the opposite causal direction — so concluding causation from correlation is a fundamental error (the ice-cream-and-drowning correlation does not mean ice cream causes drowning; both are driven by summer). Establishing causation requires more than correlation — typically controlled experiments or careful causal reasoning that rules out the alternatives — so correlational data, however strong the correlation, cannot by itself establish that one thing causes another. Critical reading centrally involves distinguishing what is correlational (things vary together) from what is causal (one causes another), and refusing to accept causal claims from merely correlational data — because so much data misuse and misunderstanding comes from reading causation into correlation.
Different data types and structures support different conclusions, and matching the conclusion to what the data can support is the core skill. Observational data (data from observing the world as it is) can show correlations and patterns but generally cannot establish causation (because it does not control the alternatives) — so observational data supports "these things are associated" but not "this causes that." Experimental data (data from controlled experiments that manipulate one thing and observe the effect, controlling alternatives) can establish causation, because the control rules out the alternatives — so experimental data can support causal claims that observational data cannot. Cross-sectional data (a snapshot at one time) shows a state but not change over time; longitudinal data (following over time) can show change and trends. Matching the conclusion to the data type — causal claims need experimental (or carefully-reasoned causal) data not just observational, trend claims need longitudinal not just cross-sectional data — is essential to sound reading and use, because drawing a conclusion the data type cannot support (causation from observational, trends from a snapshot) is a fundamental error. Understanding the types of data and what each can establish — quantitative vs. qualitative (different questions), correlation vs. causation (the crucial distinction), and the data structures (observational vs. experimental, cross-sectional vs. longitudinal) and their supportable conclusions — is what lets you read data for what it actually supports and refuse the conclusions it cannot support, which is central to critical data literacy. The most important of these, correlation vs. causation, deserves special vigilance, because reading causation into correlation is the most common and consequential data error, and refusing it is one of the most protective data-literacy skills.
Key Insight: Different data types support different conclusions. Quantitative (numerical — scale, comparison, trend) versus qualitative (meaning, context, mechanism) — both are data answering different questions; don't over-privilege numbers or misuse anecdote for scale. Most crucially, correlation (things vary together) versus causation (one causes the other) — correlational data can't establish causation (the correlation could be coincidence, a third factor, or reverse-direction), so reading causation into correlation is the most common and consequential data error. Match conclusions to the data structure: observational data shows correlation not causation; experimental data (controlling alternatives) can establish causation; cross-sectional shows a snapshot, longitudinal shows change. Reading data for exactly what it can support — and refusing what it can't — is central to data literacy.
Data does not interpret itself — humans interpret it, for purposes, within framings — and understanding this human element is essential to reading data critically, because the interpretation and framing of data shape its meaning as much as the data itself, and much data misuse operates through interpretation and framing rather than false numbers. This section examines the human element in data.
The foundational point is that data requires interpretation, and interpretation involves judgment that can be sound or unsound, honest or manipulative. A number by itself means little; its meaning comes from interpretation — what it represents, what it implies, what conclusion it supports — and this interpretation is a human act involving judgment. The same data can be interpreted differently (soundly or unsoundly), so the interpretation, not just the data, determines the meaning presented — and reading data critically means evaluating the interpretation, not just accepting the data's presented meaning. Crucially, data can be accurate while its interpretation is misleading: the numbers can be true, but the conclusion drawn from them wrong (reading causation into correlation, generalizing beyond the sample, drawing conclusions the data does not support). So critical reading assesses the interpretation — does the conclusion actually follow from the data? — separately from the data's accuracy, because true data can be misleadingly interpreted, and much data misuse is exactly this: accurate numbers, misleading interpretation.
Framing — how data is presented and contextualized — profoundly shapes its perceived meaning, often without changing the data at all. The same data framed differently conveys different meanings: "a 50% increase" versus "an increase from 2 to 3" (the same change, framed to seem large or small); "90% fat-free" versus "10% fat" (the same fact, framed positively or negatively); presenting a number without context (is this large or small? compared to what?) versus with context that shapes its interpretation. Framing effects are powerful and often manipulative: by choosing how to present accurate data (which comparison, which units, which context, which emphasis), one can shape its perceived meaning dramatically without falsifying anything. Critical reading sees through framing to the underlying data: what is the actual number, in what actual context, and what does it actually mean apart from the framing? — recognizing that the framing is a presentation choice that can mislead even with accurate data. Recognizing and seeing past framing is a core critical-reading skill, because framing is one of the most common ways accurate data is used to mislead.
The purpose behind the data's presentation shapes its interpretation and framing, and understanding the purpose is part of critical reading. Data is usually presented for a purpose (to inform, but often to persuade, to support a position, to sell, to advocate), and the purpose shapes how it is interpreted and framed — someone presenting data to make a case will interpret and frame it to support that case (choosing the interpretation and framing that favor their position, from accurate data). This does not mean all data presentation is manipulation (much is honest), but it means the purpose shapes the presentation, and critical reading involves understanding the purpose — who is presenting this data, for what purpose, and how does that shape their interpretation and framing? — to judge whether the presented meaning is sound or is a purpose-driven spin on the data. The human element — that data requires interpretation (which can be sound or misleading, honest or manipulative, even from accurate numbers), that framing shapes perceived meaning (powerfully, often manipulatively, without changing the data), and that purpose shapes interpretation and framing — means that reading data critically is not just about the numbers but about the human interpretation, framing, and purpose around them. Much data misuse operates here — not through false numbers but through misleading interpretation, manipulative framing, and purpose-driven spin of accurate data — so critical reading must engage the human element: evaluating the interpretation (does the conclusion follow?), seeing past the framing (what does the data actually say?), and understanding the purpose (how does it shape the presentation?). This human-element awareness is essential, because the most common and sophisticated data misuse is not fake numbers but the misleading human treatment of real numbers, which only critical engagement with the interpretation, framing, and purpose can catch.
Key Insight: Data doesn't interpret itself — humans interpret it, and interpretation involves judgment that can be sound or misleading even from accurate numbers (true data, wrong conclusion — reading causation into correlation, generalizing beyond the sample). So evaluate the interpretation (does the conclusion actually follow?) separately from the data's accuracy. Framing — how data is presented ("50% increase" vs. "2 to 3," "90% fat-free" vs. "10% fat," with or without context) — powerfully shapes perceived meaning without changing the data, often manipulatively; see past it to what the data actually says. And purpose (who presents this, to persuade of what?) shapes interpretation and framing. Much data misuse isn't fake numbers but misleading interpretation, manipulative framing, and purpose-driven spin of real numbers — caught only by critical engagement with the human element.
This closing section synthesizes the module into the data-literate mindset — the stance and the questions that, applied to any data, produce critical reading — which is the foundation the rest of the course builds on and the practical core of data literacy.
The data-literate mindset rests on the module's central reframe: data is constructed measurement, not objective truth — shaped by human choices (what and how to measure, collected by whom for what purpose), an imperfect proxy for reality (with a gap between measure and reality), from a particular collection (with its sampling and measurement effects), of a particular type (supporting particular conclusions), interpreted and framed by humans (for purposes). Holding this stance — data as constructed, choice-shaped, proxy, collected, typed, and interpreted representation, not as objective truth speaking for itself — is the foundation of critical reading, because it makes you see the choices and the human element behind the numbers rather than taking the numbers as simply "the facts."
From the stance flow the questions to ask of any data:
The data-literate mindset is critical but not cynical — it questions data without dismissing it, because the goal is to read data well, not to reject it. Data, though constructed, is valuable: it can genuinely inform, and well-collected, soundly-interpreted data is among the best tools for understanding reality. The data-literate person does not conclude "data is all manipulation, ignore it" (cynicism, which forgoes data's real value) but "data is constructed and must be read critically to use well" (literacy, which extracts data's value while avoiding its misuse). This balance — critical enough to catch the choices, gaps, biases, and misleading interpretations, but not so cynical as to dismiss data's genuine value — is the mature data-literate stance. It reads data as what it is: a valuable but constructed representation, to be understood through its choices and used for what it can honestly support, neither worshipped as objective truth nor dismissed as mere manipulation. This balanced, critical-but-not-cynical mindset, applying the reframe and the questions to any data, is the foundation of data literacy that the rest of the course builds on — the statistical understanding (Module 2), the critical reading of charts and tables (Module 3), the evaluation of quality and bias (Module 4), the sound use of data in argument (Module 5), and the honest communication of data (Module 6) all rest on this foundation: data as constructed measurement, read critically through the questions, valued but not worshipped, used well because understood clearly.
Key Insight: The data-literate mindset rests on the reframe (data is constructed measurement, not objective truth) and applies critical-reading questions to any data: what was measured and what wasn't; is this the reality or a proxy (and what does it miss or how is it gamed); where did the data come from (sample, representativeness, what's absent, method); what type is it and what can it support (correlation vs. causation especially); and who is interpreting and framing it for what purpose. These require no math — only understanding what data is and the habit of asking. The stance is critical but not cynical: question data to read it well, not to dismiss it — data is constructed but genuinely valuable, to be understood through its choices and used for what it can honestly support, neither worshipped nor dismissed.
You now have the foundation of data literacy: data as constructed measurement rather than objective truth (shaped by choices of what and how to measure, for what purpose), the measure-reality gap (the proxy is not the reality, and can diverge and be gamed — Goodhart's Law), the effects of collection (sampling bias, survivorship bias, what is absent, measurement effects), the types of data and what each can support (quantitative vs. qualitative, and crucially correlation vs. causation), the human element (interpretation, framing, and purpose that shape meaning even from accurate numbers), and the data-literate mindset (the reframe, the critical-reading questions, and the critical-but-not-cynical balance). You can read any data by asking what it actually is and what it can honestly tell you — the foundation of all critical data reading.
The next module, Statistical Fundamentals Without Math Trauma, builds the statistical understanding that data literacy requires — not calculation but the conceptual understanding of the key statistical ideas (averages, variation, probability, significance, and the ways statistics mislead) that let you read statistical claims critically. As promised, it requires no mathematical background: it builds the conceptual understanding of what statistics mean and how they mislead, which is what reading statistical data critically actually requires. Everything builds on this module's foundation: statistics are constructed measurements interpreted by humans, and understanding them conceptually — what an average hides, what variation means, how probability is misread, what significance does and does not establish — extends the critical reading of this module to the statistical claims that data literacy must engage.