Python for Data Analysis

Python is the language where your analysis stops being limited by spreadsheet rows or one window — where you can clean 200,000 rows, rerun an analysis instantly with new data, and hand a colleague a script that reproduces your whole pipeline. Even though you will reach for Excel and Power BI for many tasks, Python is what job postings mean by "programming", and it is the bridge into EDA and machine learning next.

This topic teaches the analyst's Python — not a computer-science course. You will learn just enough language fundamentals to read and write script-like code, then spend most of your time in pandas, the library that makes Python behave like a supercharged spreadsheet with a memory.

Basics through SQL give the concepts; Data Cleaning gives the workflow you will re-express here. No programming background needed — this starts from zero and moves at analyst speed.

Excel and Power BI are fantastic — until the data is huge, the task is repetitive, or you need the exact same analysis every week with zero clicking. Python's analyst strengths:

• Scale and speed — read and transform millions of rows in seconds
• Automation — one script does Monday's, Tuesday's and next month's job
• Reproducibility — the script is your documented methodology
• Ecosystem — pandas, plus plotting (matplotlib, seaborn) and, later, statistics/scikit-learn

In this path, Python supersizes your data-cleaning and prep work and becomes the workhorse for Exploratory Data Analysis. You will still use Power BI for dashboards and Excel for quick checks — professionals use all three.

You need three pieces: Python (the 3.x runtime), a code environment, and pandas (plus NumPy). The analyst-standard environment is Jupyter notebooks — a document of cells mixing markdown notes and runnable Python, ideal for analysis that tells a story top to bottom.

# terminal:
python --version                 # verify Python 3.8+
pip install pandas numpy jupyter # install your analysis stack
# then:
jupyter notebook               # opens the notebook UI in your browser

In a notebook: run a cell with Shift+Enter; an In [5] counter appears. The golden cell habit: one idea per cell, run top to bottom, and re-run everything after any change — a cell run out of order is the #1 source of "but it worked a minute ago".

Don't learn a programming language — learn a toolbox. These few pieces carry 95% of analyst code:

# Variables & types
name      = "Cairo"            # str
price     = 49.99              # float
orders    = 1200               # int
in_stock  = True               # bool

# Collections
cities = ["Cairo", "Alex", "Giza"]   # list (ordered)
info   = {"region": "North", "target": 100000}  # dict (key → value)

# Loop + conditional
for city in cities:
    if city.startswith("C"):
        print(f"Starts with C: {city}")   # f-string

# Function
def margin(revenue, cost):
    return (revenue - cost) / revenue

• Zero-indexing: cities[0] is the first element; slicing cities[1:3] takes positions 1–2
• Methods — actions on values: " Hi ".strip(), cities.append("Mansoura"), name.lower()
• Indentation is syntax — four spaces under if/for/def; inconsistent indentation is the beginner bug

You will not write a lot of loops in pandas (the library vectorizes them for you) — but you must be able to read them and debug them.

NumPy gives you arrays — grids of numbers where every operation applies to the whole grid at once (vectorized). You rarely call NumPy directly, but it powers pandas, and understanding the "operate on everything" idea unlocks everything else.

import numpy as np

amounts = np.array([100, 250, 400, 700])   # a numeric vector
discounted = amounts * 0.9                 # applies to every element!
print(discounted)                          # [ 90. 225. 360. 630.]
print(amounts.mean())                      # 362.5
print(amounts[amounts >= 300])             # boolean filter: array([400, 700])

Notice there is no loop: the operation runs on all rows because the array knows itself as one object. That same speed and simplicity is exactly how pandas handles whole columns.

pandas gives you two objects: a Series (one labeled column) and a DataFrame (labeled rows × columns — think "a spreadsheet in code").

import pandas as pd

df = pd.read_csv("orders.csv")   # load CSV

df.head()      # first 5 rows
df.shape       # (rows, columns)
df.columns     # column names
df.info()      # types + nulls — your profile in one call
df.describe()  # count/mean/min/max/percentiles for numeric columns

df["region"].value_counts()   # categorical profile

With df.head(), df.info() and df.describe() you have just replaced your Excel/SQL profiling step — in three lines, on any size file. That is the moment the power of code becomes obvious.

Two selection styles cover everything:

# By label:      df.loc[row_selector, column_selector]
# By position:   df.iloc[row_position, column_position]

df.loc[:, ["order_id", "amount"]]          # pick columns by name
df.loc[df["region"] == "North"]            # rows where region is North

# Boolean mask → the SQL WHERE
df.loc[(df["amount"] >= 500) & (df["status"] == "shipped")]

# Or the query() sugar
df.query("amount >= 500 and status == 'shipped'")

# New derived column (vectorized, like a calculated column)
df["revenue"] = df["quantity"] * df["price"]
df["order_size"] = df["revenue"].apply(lambda v: "Large" if v >= 500 else "Small")

• & (and) and | (or) — never Python's and/or on arrays; parentheses around each condition are required
• .apply() runs row-wise logic when a vectorized expression is not easy

This mirrors Excel's filter + XLOOKUP + IF, all evaluated for the entire table at once.

Every trick from your data-cleaning topic now has a one-liner:

# Missing values
clean = df.dropna(subset=["order_id"])            # drop rows missing a key column
clean = df.fillna({"region": "UNK"})             # fill a column with a default

# Duplicates — define the key first
clean = clean.drop_duplicates(subset=["order_id"], keep="last")

# Text standardization
clean["region"] = clean["region"].str.strip().str.title()

# Types
clean["order_date"] = pd.to_datetime(clean["order_date"])     # true dates
clean["price"] = clean["price"].astype(float)

# Replace / recode categories
clean["segment"] = clean["segment"].replace({"prem": "premium", "stnd": "standard"})

Notice the method chain pattern: read → drop → fill → dedupe → standardize → type, all as script steps. This is your Raw → Clean pipeline as executable, reviewable code — run it on a new file tomorrow and it recreates the exact same result.

Your SQL muscle memory transfers directly:

# GROUP BY in pandas
df.groupby("region")["amount"].sum()          # revenue by region
df.groupby(["region", "month"]).agg(
    orders   = ("amount", "count"),
    revenue  = ("amount", "sum"),
    avg      = ("amount", "mean"),
).reset_index()

# SQL-style JOINs
merged = orders.merge(customers, on="customer_id", how="left")  # LEFT JOIN

# Pivot — like a PivotTable
df.pivot_table(index="region", columns="month",
               values="amount", aggfunc="sum")

After every merge, check the shape: merged.shape — a many-to-many join can silently duplicate rows (the same trap as SQL).

Finish like a professional:

clean.to_csv("orders_clean.csv", index=False)   # export, no row numbers

# save a single frame for quick inspection
summary = df.groupby("region")["amount"].sum().reset_index()
summary.to_excel("region_report.xlsx", index=False)

And follow notebook hygiene that colleagues (and future-you) will thank you for: explain each section with a markdown cell (title, source, question); keep one table → one analysis step; name DataFrames meaningfully; don't print the whole table; and when an analysis grows beyond a page, split reusable logic into functions.

The unspoken rule: your notebook is your methodology document. A reviewer should be able to re-run it end-to-end and get the same numbers — that is reproducibility, and it is the difference between "I found this" and "here is the evidence".

Reuse the messy CSV from your data-cleaning topic — this time by code.

1. Load it with pandas and profile it in three lines (shape, isna-sum, nunique).
2. Clean it in a method chain: drop dupes on the key, fill nulls (sensible defaults), standardize text, convert dates and numbers.
3. Verify the cleaning (isna().sum() and duplicated().sum() = 0 on the key).
4. Answer 4 business questions with groupby/agg and merge (e.g. revenue by region × month, top category, salesperson rank).
5. Export the clean table and one summary and confirm both re-run from a fresh kernel.

Python appears in analytics interviews both as takehome code and as screening questions: "load this, clean it, answer these three questions, email the script". What they grade is exactly what this topic trained: correct pandas idioms, a clean reproducible notebook, thoughtful cleaning decisions, and clearly communicated results. On the job, Python is your automation arm — scheduled ETL scripts, exploratory studies, ad-hoc analyses too big for Excel — and it is the prerequisite for the EDA topic next: filling notebooks with numbers, charts and honest conclusions before anything gets presented to a stakeholder.

Business problem: The growth team keeps getting different answers from different people for the same question. They ask you to produce one canonical weekly analysis, fully reproducible.

Dataset: any public retail CSV (or your cleaned data-cleaning export), 1,000+ rows, with date, region, category, salesperson and amount.

Objective: one clean, automated, reproducible pipeline + a concise weekly summary notebook.