Skip to content

Python warm up

60 min Foundations Needs: Python running, from the last tutorial

Seven small tasks on realistic data. Loops, functions, lists, and dictionaries. Nothing here is new material, and that is the point. These are the moves you will use every day for the rest of the programme.

Why this matters

Most machine learning work is not modelling. It is getting data out of an awkward shape and into a useful one. Before a model sees anything, somebody loops over records, pulls out the fields that matter, groups them, counts them, and checks what is missing.

That somebody is you. The exercises below are that job in miniature.

The data

You will use the sample users from JSONPlaceholder. It is fake data shaped like a real web API, which means it is nested. Fields inside fields.

The shape of one user record Data is a list of users. Each user has name, username and email at the top level, plus an address object containing street, suite, city and zipcode, and a company object containing name. data a list of 10 users data[0] one user, a dictionary "name", "username", "email" plain text, one level down "address" holds "street", "suite", "city", "zipcode" a dictionary inside the dictionary "company" holds "name"

So a city is not user["city"]. It is user["address"]["city"]. Getting that wrong is the single most common error in these tasks, and reading the diagram above is faster than guessing.

Getting the data

import requests

data = requests.get("https://jsonplaceholder.typicode.com/users").json()

If Python complains that requests is missing, install it first:

pip install requests

Open the URL in your browser, copy everything, and paste it into your file as a list:

data = [
    {"id": 1, "name": "Leanne Graham", ...},
]

This works offline and is a fine choice.

Before you write anything else, look at one record:

print(data[0])
print(data[0]["address"]["city"])

Always do this with unfamiliar data. Ten seconds of looking saves twenty minutes of guessing at key names.

The tasks

Create a file called user_processing.py. Each task is one function.

Task 1: Print every name

Loop through data and print each user's name.

Leanne Graham
Ervin Howell
...

Task 2: Collect the .biz emails

Return a list of every email address ending in .biz.

Return the list, do not print it inside the function. A function that returns a value can be used by other code. A function that only prints is a dead end.

Task 3: Find users by city

Take a city name as an argument and print the name and email of everyone living there. Remember the nesting: user["address"]["city"].

Task 4: Count users per company

Build a dictionary where the key is a company name and the value is how many users work there.

{'Romaguera-Crona': 1, 'Deckow-Crist': 1, ...}
Stuck on the counting pattern?

The shape is always the same. Check whether the key exists, add it with a starting value if it does not, then increase it.

counts = {}
for item in things:
    key = item["something"]
    counts[key] = counts.get(key, 0) + 1

dict.get(key, 0) returns zero instead of crashing when the key is missing. You will use this pattern constantly.

Task 5: List the unique zip codes

Pull every zip code out of the address section, remove duplicates, and print them sorted.

A set removes duplicates for you. sorted() turns it back into an ordered list.

Task 6: Group users by the first letter of their username

Build a dictionary where each key is a first letter and each value is a list of names.

{'B': ['Leanne Graham'], 'A': ['Ervin Howell', 'Chelsey Dietrich'], ...}

This is grouping, and it is the same operation as groupby in pandas, which you will meet in the Data module. Doing it by hand once makes the pandas version obvious later.

Task 7: Print a full address

Take a username and print that person's address on one line:

Leanne Graham: Apt. 556, Kulas Light, Gwenborough, 92998-3874

Decide what your function does when the username does not exist. Printing a clear message beats crashing, and thinking about the missing case is a habit worth building now.

How to hand it in

Put every task in its own function and call them from a main block:

def print_all_names(data):
    """Print the name of every user."""
    for user in data:
        print(user["name"])


if __name__ == "__main__":
    print_all_names(data)

The if __name__ == "__main__": line means the code below it runs when you run the file directly, but not when another file imports your functions. It is standard in Python and you should use it from now on.

Then commit and push:

git add user_processing.py
git commit -m "Add user processing exercises"
git push

Check yourself

Before you submit, confirm each of these:

  1. Every task is a separate function with a name that says what it does.
  2. Tasks 2 and 6 return their result rather than only printing it.
  3. Running python3 user_processing.py produces output for all seven tasks with no errors.
  4. Each function has a one line docstring.
  5. Your work is pushed to GitHub and you can see it in the browser.

One test of whether you have really got this

Close the file, open a new one, and write Task 4 again from memory. If you can, the pattern is yours. If you cannot, that is useful information and it is better to find out now than in the Data module.

You have finished Tools

Your accounts work, your code is backed up on GitHub, you can experiment in notebooks, and you have a Linux environment for real programs.

The next module starts with data. Watch the cohort channel for when it opens.