Project 3: How sure are you?¶
One week No models Groups of 3 or 4 Presented 16 November
Nobody hands you a CSV this time. You go and get your data from two places, make it into one table, and then answer a question in a way that says how sure you are of the answer.
What you are doing¶
Three things, and the third is the one that makes this project different from the last one.
You collect it yourself. One API and one Wikipedia table. You have done both of these separately. Now you do them in the same week and join the results, which is where it gets interesting.
You write down what you built. A data contract: every column, its type, its unit, where it came from, and what is missing from it. Somebody else should be able to read that table and know what they are holding.
Every number comes with a range. Not "62 percent". Something closer to "62 percent, give or take 4, and here is why that is not just luck". A number with no range is an opinion with a decimal point.
No models. No training, no predictions, no accuracy. If you find yourself importing scikit-learn, you have misread the brief.
The week¶
Six days of work and a presentation on Monday. That is short, and the shape of the week matters more than it did last time.
Monday is a gate, not a warm-up
Before anything else, make one successful request to each of your two sources and save what comes back to a file. Both of them. On Monday.
This is the rule that decides whether you have a project. A group that discovers on Thursday evening that the API will not answer them has lost the week. A group that discovers it on Monday morning has lost an hour and can ask for help.
The rules¶
Only your two sources. The ones in your tab, and nothing else. If you want to add a third, ask first.
Be polite on every request. A User-Agent header that names you and gives a
way to contact you. One request at a time. Roughly one a second, slower if the
source asks for slower. There is no prize for finishing the download first and
the cost of going fast is paid by somebody else.
Cache everything. The first time you fetch something, write it to disk. Every run after that reads your copy. You will run your parsing code fifty times before it is right and the server does not need to see any of that.
Commit the raw files. They are small. They are also the only proof of what the source said on the day you asked.
Write the data contract. One table, like this:
| column | type | unit | source | known gaps |
|---|---|---|---|---|
race_date |
date | Jolpica API | none | |
circuit_length |
float | km | Wikipedia circuits table | 4 circuits missing |
grid_position |
int | position | Jolpica API | blank when the car started from the pit lane |
Use real probability. Across your two questions, at least two of these four:
- A range worked out by simulation, not by a formula you found.
- A comparison against what pure chance would produce, also by simulation.
- A fitted distribution, with an honest look at where it does not fit.
- Base rates or conditional probability.
It must run with the wifi off. Restart the kernel, disconnect, run the notebook top to bottom from your cached files. If it fails, it is not finished.
When the source misbehaves¶
It will. Plan for it rather than panicking on Thursday.
- Cache first, so that a source going down costs you nothing you had already collected.
- The other group on your dataset hit the same wall. Compare notes. Helping each other collect is allowed and sensible. Sharing analysis is not.
- A 403 or a 429 is information, not a wall. Slow down, set a proper
User-Agent, and read what the response body actually says. - If you are still stuck after an hour, message the channel. Losing a day to something somebody could have told you in two minutes is the one avoidable failure in this project.
What a range actually buys you¶
This is the concept the whole project turns on, so it is worth being precise about it.
You met this on the Probability page as sampling variation. Here it is the difference between a finding and a sentence.
ML connection
In three weeks you will be comparing models and reading a difference of one percentage point as an improvement. The habit you build this week, asking how wide the range is before deciding whether the difference is real, is the one that stops you from doing that.
What everyone does¶
Same tasks for every group. What they mean is in your tab.
| Task | |
|---|---|
| 0 | Access check: one successful request to each source, saved to disk |
| 1 | Collect from the API, with caching |
| 2 | Scrape the Wikipedia table |
| 3 | Join, clean, and write the data contract |
| 4 | Explore: what is actually in this table |
| 5 | Probability question one |
| 6 | Probability question two |
| 7 | Build the talk |
Dataset one: Formula 1¶
Seventy-five years of race results, and a sport that has argued with itself for all of them about whether the driver or the machinery wins races.
Two groups work on this data from different angles. You will be standing next to each other on Monday.
| Source | What it gives you | How to behave |
|---|---|---|
| Jolpica-F1 API | Results, qualifying, pit stops, laps, drivers, constructors, 1950 to now. Ergast-compatible JSON, no key. | 4 requests a second and 500 an hour. Pages are 30 rows by default, 100 maximum. |
| List of Formula One circuits | Circuit type, length, country, years active. | Name yourself in the User-Agent. Use the /wiki/ URL with no query string. |
The question. Teammates drive the same machinery. So when one of them finishes ahead of the other, that is as close to a fair fight as this sport offers. How much of a driver's result is the driver?
Question one. Take teammate pairs across a season. How often does the stronger one finish ahead? Simulate what a coin flip between two equal drivers would produce over the same number of races, and compare.
Question two. Does the size of the teammate gap depend on the kind of circuit? Street circuits against permanent ones, or short against long. Report a range, not a number.
Watch out for. A head-to-head only means something when both cars finished. Drivers change teams mid-season. And a car that broke is not a driver who was slower, so decide what to do about retirements before you count anything, and write the decision down.
If you finish early. Do the same comparison on qualifying times instead of race results, where mechanical retirements cannot interfere, and see whether the answer changes.
The question. Everybody says qualifying matters. Put a number on it, and then find out whether that number is the same everywhere.
Question one. Given pole position, what is the chance of winning? Work out a range by simulation, and say plainly whether the range is narrow enough to be useful.
Question two. What does each grid position buy you? Chance of a podium from P1, P2, P5, P10. Then pick one circuit where the answer is unusual and explain why.
Watch out for. Grid penalties mean the starting grid is not the qualifying order. The number of races at any one circuit is small, so your per-circuit ranges will be wide and you should say so rather than hide it. And the sport in 1955 is not the sport in 2025: cars finish now, and they did not then.
If you finish early. Split your answer by era. Has the value of pole position gone up or down since the 1980s?
Dataset two: Albanian weather¶
Daily weather for anywhere in Albania, back to 1940, and a set of published "normals" on Wikipedia claiming what the weather is supposed to be.
Two groups work on this data from different angles. You will be standing next to each other on Monday.
| Source | What it gives you | How to behave |
|---|---|---|
| Open-Meteo archive API | Daily maximum and minimum temperature, precipitation, wind, for any coordinate, 1940 to now. ERA5 reanalysis, no key. | Free for non-commercial use. Data is CC BY 4.0, so credit it. Ask for long date ranges in one request rather than looping over days. |
| Wikipedia city pages, climate boxes: Tirana, Shkodër, Vlorë, Korçë | The published monthly normals: average high, average low, rainfall, rainy days. | Name yourself in the User-Agent. One request per city, cached. |
The question. Everybody says the summers are worse than they used to be. Everybody also remembers selectively. Settle it for one city.
Question one. The chance of a day above 35 degrees in June, July and August, for 1961 to 1990 and for 1991 to 2020. Both with simulated ranges. Do the ranges overlap?
Question two. Did the average move, or did the tails move? Those are different claims. Compare the shift in the mean against the shift in the 95th percentile, and say which one actually changed.
Watch out for. ERA5 is a model reconstruction on a grid of roughly 25 kilometres, not a thermometer standing in Skanderbeg Square. If you compare your numbers against a published normal, check what years that normal covers before you treat the difference as a finding.
If you finish early. Run the same analysis for a second city and see whether your conclusion travels.
The question. The climate box on Wikipedia says August has two rainy days. Can you plan a wedding on that?
Question one. The chance of a rain day by month, for one city, with a range. You will have to choose what counts as a rain day. Defend the threshold you pick.
Question two. Streaks. How long is the longest dry spell you should expect in July? How often does a week of rain happen in September? At least one of these should be answered by simulation.
Watch out for. Consecutive days are not independent: rain tomorrow is more likely if it rained today, so anything you calculate by multiplying a daily probability together will be wrong. And when you compare against the Wikipedia normals, look carefully at the small print under each climate box. They are not all measuring the same thing.
If you finish early. Compare a coastal city with an inland one and say which is easier to plan around.
Before you touch any data¶
Each group writes down, on paper, what you expect to find. One or two sentences. Fold it up and give it to me.
It gets opened on Monday, next to your answer.
Why this is worth doing
If you form your expectations after exploring, you will form ones you already know are true, and you will have proved nothing to anybody including yourself.
Being wrong on Monday is the best outcome available to you. It means the data told you something you did not already believe, which is the only reason to go and get data in the first place.