Data Moves

Meet the data

Here are two small tables that we will use to learn the different ways you can work with a dataset. We will focus on one data move at a time. Each one is represented as:

The move in English (what you would ask for);
The line of Python (what an AI would run);
The table before and after the operation.

`schools`
school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

NaN is how Python shows a missing value ("not a number"). Here, Grant's absences and Hale's district are missing.

In schools, what does one row represent?

`funding`
district	year	per_pupil
North	2025	12,000
South	2024	9,500
South	2025	9,800
East	2025	10,200

In funding, what does one row represent?

Part 1 of 3 · Five moves on a tableMost of what an analysis does to a table is one of these five moves.

1 · Sort rows One table

“List the schools from largest to smallest.”

schools.sort_values("students", ascending=False)

How to read this line

`schools.sort_values(...)`	A dot means do this to the thing on the left: sort `schools`.
`"students"`	Sort by this column.
`ascending=False`	A named option: biggest first. Leave it out and the default (smallest first) applies.

Before

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: which school will be in the first row?

After 8 → 8 rows

school	district	students	absent
Diaz	South	600	60
Adams	North	400	60
Hale	NaN	350	70
Ford	East	300	75
Baker	North	250	50
Grant	East	200	NaN
Cole	North	150	45
Evans	South	100	30

What changed: only the order. But the order of rows is what people read first, so it shapes the story.

2 · Keep rows One table

“Keep only schools with at least 250 students.”

schools[schools["students"] >= 250]

How to read this line

`schools[ ... ]`	Square brackets with a condition inside: keep the rows where it is true.
`schools["students"]`	The `students` column. Quoted name in brackets = a column.
`>= 250`	The condition: at least 250.

Before

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: how many rows will come out?

After 8 → 5 rows

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Diaz	South	600	60
Ford	East	300	75
Hale	NaN	350	70

What can go wrong: the question quietly narrows. Any average you compute next describes schools with at least 250 students, not schools. Nothing in the output table says so.

3 · Keep columns One table

“Show just each school's name and number of students.”

schools[["school", "students"]]

How to read this line

[["school", "students"]] A list of column names inside the brackets (note the double brackets): keep those columns.

Before

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: how many rows will you see?

After 8 → 8 rows

school	students
Adams	400
Baker	250
Cole	150
Diaz	600
Evans	100
Ford	300
Grant	200
Hale	350

What changed: only what you see. The data behind the table is untouched.

4 · Add a column One table

“Calculate each school's absence rate.”

schools["absent_rate"] = schools["absent"] / schools["students"]

How to read this line

`schools["absent_rate"] = ...`	A single `=` stores a result. Here it stores a new column named `absent_rate`.
`schools["absent"] / schools["students"]`	Divide one column by another, row by row.

Before

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: what will Cole's absent_rate be?

After 8 → 8 rows, one new column

school	district	students	absent	absent_rate
Adams	North	400	60	0.15
Baker	North	250	50	0.20
Cole	North	150	45	0.30
Diaz	South	600	60	0.10
Evans	South	100	30	0.30
Ford	East	300	75	0.25
Grant	East	200	NaN	NaN
Hale	NaN	350	70	0.20

What to look out for: when you add a new column you're creating a new variable, so have a few checks in mind first. In this example, every school should now have a different absence rate (if instead absence rates are the same within a district, you may have divided by the full district's students instead!). Each absence rate should be between zero and one (if not, you may have divided the wrong things together).

5 · Summarize One table

“Total students across all the schools.”

schools["students"].sum()

How to read this line

`schools["students"]`	The `students` column.
`.sum()`	Add up every value in it. Other summaries work the same way: `.mean()`, `.max()`, `.count()`.

Before

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: what will come out?

After 8 rows → 1 number

2,350

What changed: the table is gone. One number now stands for all eight schools. A whole-table total, average or count is the simplest summary, and often the first number a report leads with.

Part 2 of 3 · Group byYou can also split the rows into groups, apply a move to each group, then combine the results.

6 · Group by Group by

“Total students in each district.”

schools.groupby("district")["students"].sum()

How to read this line

`.groupby("district")`	Split the rows into one pile per district. On its own this produces nothing you can see.
`["students"].sum()`	Apply a sum to the students column in each pile, then combine the answers into a new table.

Before (coloured by district)

school	district	students	absent
Adams	North	400	60
Baker	North	250	50
Cole	North	150	45
Diaz	South	600	60
Evans	South	100	30
Ford	East	300	75
Grant	East	200	NaN
Hale	NaN	350	70

Predict: how many rows will come out?

Split → apply → combine

East

Ford · 300

Grant · 200

sum = 500

North

Adams · 400

Baker · 250

Cole · 150

sum = 800

South

Diaz · 600

Evans · 100

sum = 700

NaN

Hale · 350

sum = dropped

After 8 → 3 rows

district	students
East	500
North	800
South	700

What changed: one row now represents a district, not a school. The three districts add up to 2,000 students, but the whole table adds up to 2,350. The missing 350 is Hale, and nothing in the output says so.

Group by is not a move on its own: it changes what the next move does, one group at a time. Here it turned one total (the last screen) into one total per district. In class you will see it paired with other moves.

Part 3 of 3 · Combining two tablesSo far every move used one table. Often what you need is spread across two.

7 · Match Two tables

“Add each district's funding per pupil to the schools table.”

schools.merge(funding, on="district", how="left")

How to read this line

`schools.merge(funding, ...)`	Match rows of `schools` to rows of `funding`.
`on="district"`	Rows match when their district is the same.
`how="left"`	Keep every school, even one with no match in `funding`.

schools

school	district	students
Adams	North	400
Baker	North	250
Cole	North	150
Diaz	South	600
Evans	South	100
Ford	East	300
Grant	East	200
Hale	NaN	350

funding

district	year	per_pupil
North	2025	12,000
South	2024	9,500
South	2025	9,800
East	2025	10,200

Predict: how many rows will come out?

After 8 → 10 rows

school	district	students	absent	year	per_pupil
Adams	North	400	60	2025	12,000
Baker	North	250	50	2025	12,000
Cole	North	150	45	2025	12,000
Diaz	South	600	60	2024	9,500
Diaz	South	600	60	2025	9,800
Evans	South	100	30	2024	9,500
Evans	South	100	30	2025	9,800
Ford	East	300	75	2025	10,200
Grant	East	200	NaN	2025	10,200
Hale	NaN	350	70	NaN	NaN

What to look out for: if there are more entries in one table than the other, a match can multiply rows. Each observation went from being at the school level to being at the school-year level. Since only one district had two years of data, schools from the South now have two rows while schools from the North and East have only one.

If this happens, be careful about making inferences! Is the data on each school comparable when there are two years for some and not others? The question to ask of any match: did the row count change?

Recap: data moves

	Move	Python you saw
Five moves on a table	Sort rows	`schools.sort_values("students")`
	Keep rows	`schools[schools["students"] >= 250]`
	Keep columns	`schools[["school", "students"]]`
	Add a column	`schools["absent_rate"] = ...`
	Summarize	`schools["students"].sum()`
Group by	Group by + Summarize	`schools.groupby("district")["students"].sum()`
Combining two tables	Match	`schools.merge(funding, on="district")`

Note: When AI writes code, the table usually has a generic name like df (short for “data frame”, Python's word for a table) or data. It's just a name: df[df["students"] >= 250] is the same Keep-rows move on a table called df.

You may have noticed that along the way, the table changed: Hale vanished in the Group by, and Diaz and Evans doubled in the Match.

After each step of an analysis, ask yourself and check:

What does one row represent now?
How many rows went in, and how many came out?