Learn Any ML Library by Rebuilding It: NumPy, pandas, scikit-learn, PyTorch and sentence-transformer

Question

A. Introduction

1. Abstract

Some people learning machine learning reach a stage where they can run code copied from tutorials but can't write it themselves. This post presents a No Shortcuts addendum which argues that one cause is the way libraries are usually learned: function by function, from the outside, without the small data model and design conventions each library is built on. For NumPy, pandas, scikit-learn, PyTorch and sentence-transformers, the addendum identifies that core, rebuilds it as a miniature in plain Python, and checks the miniature against the real library, about 5,000 checks in all, rerun by CI on every change. Here you get the argument, the method, and one real result from each library. The full text, about 32,000 words with every miniature and its tests, lives in its own branch of the course repository. Every output shown was produced by running the code.

2. The question

Someone on r/learnmachinelearning asked a question a few days ago that I suspect more people have than say out loud. Paraphrasing: "I copy the code from the labs and tutorials and it works, but I can't write it myself. When I see fit() and predict() or model.encode(...) I understand what they do in that example, and I'm lost as soon as the situation is a little different."

I know that feeling very well. My first real ML job had very little to do with the ML I had studied. They gave me a table with 400 million rows and half the labels broken, the pipeline blew up every five minutes, and the courses I had paid for (three of them, my wallet still remembers xD) were useless for all of it. They had taught me which function to call. None of them had taught me what the function was doing, so the moment the data stopped looking like the tutorial's, I was copying code I didn't understand into a problem it wasn't written for.

The answer I ended up writing is far too long for one post (around 32,000 words), so it became the first addendum to my course, and this post is the door to it.

3. Where the copy-paste phase comes from

A common way to learn a library, and the way I learned my first ones, is from the outside, one function at a time. You see df.groupby("city").mean() in a tutorial, it works, you remember it. You see StandardScaler().fit_transform(X), it works, you remember that too. After a few months you have a long list of incantations and no model of what's underneath, so every new situation looks like a new incantation you haven't seen yet.

The thing is that each of these libraries is a small set of ideas, a data model plus a handful of design conventions, with thousands of functions built on top:

• NumPy is one idea: an array is a block of memory plus four numbers that say how to read it. Slicing, transposing, broadcasting, reshaping and "why did my original array change?" all come out of it.
• pandas adds one more: every column carries an index, and operations line up by label, not by position. Most pandas surprises are that alignment doing its job.
• scikit-learn is a contract. Hyperparameters go in the constructor, fit learns from data and stores the result in attributes that end with an underscore, predict and transform use what was learned. Pipelines and cross-validation are built on that contract.
• PyTorch is tensors (NumPy's idea again) plus a recorder that writes down every operation so it can compute gradients backwards, plus nn.Module, which is mostly a clever way of collecting parameters.
• sentence-transformers is a tokenizer, a transformer, an average and a normalization. model.encode() is about fifteen lines once you know that.

Once you know the idea, the documentation turns from a wall of functions into a list of things whose behavior you can mostly guess, and that's what lets you write code instead of copying it.

4. And now the code comes from an AI

There's a second cause, and it's newer. A few years ago the code you copied came from a tutorial or a Stack Overflow answer, and at least you had to find it, read it a bit, and adapt it. Now you ask an assistant, it hands you thirty lines that run, you paste them, and nobody ever types a single character of them. The code works, so it feels like progress, and if what you wanted was to learn the library, it mostly isn't.

This one has been measured. In a randomized experiment published in January 2026, developers learned a new Python library (Trio, for asynchronous code), some with an AI assistant and some without (Shen and Tamkin, "How AI Impacts Skill Formation"; both authors work at an AI company, which is worth knowing when you read it). The group with AI understood the library worse, read and debugged code worse, and wasn't significantly faster on average. Those who handed the whole task to the assistant learned the least. The ones who kept their heads in it (asking for explanations, checking what they got) kept most of the learning. It's one study with 52 people and one library, so don't treat it as a law of nature, but it points the same way as decades of memory research.

That research has a name for the effect: the generation effect. You remember what you produce much better than what you only read. It was described in 1978 (Slamecka and Graf), and a meta-analysis of 86 studies later found the advantage again and again, close to half a standard deviation on average (Bertsch et al., 2007). Pasting code is reading it, at best. Writing it from memory, and only then checking, is generating it, and typing it sits somewhere in between.

Nobody can stop you from pasting, and at work you'll probably use assistants a lot, which is fine. But while you're learning a library, type every character yourself. I'd go further: write some of it with pencil and paper. I know it sounds old-fashioned, and to be honest about the evidence, the famous "the pen beats the laptop" study about note-taking didn't hold up well when others repeated it (Morehead, Dunlosky and Rawson, 2019), so I won't sell you paper as a superpower. What paper does is take away autocomplete, copy-paste and the assistant, so the only way to get the code onto the page is to generate it, which is the part the evidence supports.

5. Why this is a No Shortcuts addendum

No Shortcuts is a free course on data, ML, deep learning and AI engineering that I've been writing (github.com/BlueRingsLabs/no-shortcuts). The course rebuilds the algorithms from scratch before using the libraries: linear models before scikit-learn, an autograd engine before PyTorch, a GPT before Hugging Face. What it takes for granted is the libraries themselves, and that's exactly where the copy-paste phase lives. So the addendum, Learning ML Libraries by Rebuilding Them, does to the tools what the course does to the algorithms. It isn't part of the course, and you don't need the course to read it.

Note: Every snippet below was run, and every output is the real output, on Python 3.13.17, NumPy 2.5.4, pandas 3.1.0, scikit-learn 1.9.1, PyTorch 2.15.0 (CPU) and sentence-transformers 6.2.0. On another machine, floating-point results can differ in the last digit or two.

B. The method, in one page

1. Learn the layers in a different order

Every serious library can be read at five levels: the problem it solves, its data model (the one or two objects everything else operates on), its conventions (what gets modified in place, how things are named, what comes back), its API (the thousands of functions) and its implementation. Tutorials start at the API, and that's usually where learners stop too. I think that's backwards. The data model and the conventions take an afternoon or two per library, and after that a function you've never seen is much less of a mystery: you already know what kind of object it takes, whether it'll touch your data and what shape its result has, because those are decided by the conventions.

2. Read the right kind of documentation

A big reason documentation feels overwhelming is reading the wrong kind for what you need. Daniele Procida's Diátaxis framework (diataxis.fr) splits it into four kinds:

Kind	It answers	Read it when
Tutorial	"Show me, step by step, so I get something working."	You've never used the library. Once.
How-to guide	"How do I do this specific thing?"	You have a task and know roughly what you need.
Reference	"What exactly does this function take and return?"	You're writing code and need a precise detail.
Explanation	"Why does it work this way? What's the idea?"	You want to understand the library.

Going straight to the reference to learn is like learning a language from its dictionary. The explanation pages ("Copies and views" in NumPy, "Developing scikit-learn estimators" in scikit-learn, the autograd notes in PyTorch) are where the data model and the conventions live. They're short, and they're the easiest pages to skip.

3. Build it small, check it against the real thing

This is the loop every chapter of the addendum follows:

1. Find the core: read the explanation pages until you can say in two sentences what the data model is and what the main conventions are.
2. Write a miniature from memory, twenty to a hundred lines of plain Python that implement that core and nothing else. Getting stuck is information.
3. Test it against the real library on many random inputs, with the library's own tools when it has them (np.testing, pandas.testing, scikit-learn's check_estimator, torch.autograd.gradcheck).
4. Read every failure, because each one is something you believed about the library that isn't true.
5. Read the real source of whatever you got wrong. Now you know what you're looking for.
6. Use the real library on a small project, without a tutorial.

The miniatures are slow and incomplete on purpose, so that you can hold the whole thing in your head. And they pay off when they disagree with the library, as you'll see in section G below.

4. Type it, write it on paper, use the assistant as a tutor

Type everything you study, including the code in this post. Predict what each line prints before you run it. The next day, write the miniature again on paper, from memory, then type it and run the test. And use assistants as a tutor, not as a ghostwriter: "ask me five questions about scikit-learn's estimator contract and don't give me the answers until I've tried" is retrieval practice with infinite questions, while "write the training loop for me" skips exactly the part that builds the skill.

The full chapter adds introspection (inspect.signature, __mro__, reading a library's source), reading a traceback from the bottom up, what to memorize, and how libraries warn you before they change, including why Python hides most of those warnings from your own modules: chapter B in the addendum.

C. NumPy: the bug that doesn't crash

A NumPy array prints like a nested list, but underneath it's a flat buffer plus a dtype, a shape, strides (how many bytes to jump along each axis) and an offset. Slicing computes new strides over the same buffer, which is why a slice is a view. Broadcasting gives the stretched axis a stride of zero, which is why np.broadcast_to(np.arange(4), (3, 4)).strides is (0, 8). The miniature, MiniArray, is about 120 lines, and it agrees with NumPy on 805 random cases.

Once you see broadcasting that way, this bug stops being mysterious. You want to subtract each row's mean from that row:

import numpy as np

X = np.array([[1., 2., 3.], [10., 20., 30.], [100., 200., 300.]])
print(X.mean(axis=1))
print(X - X.mean(axis=1))                  # wrong, and no error
print(X - X.mean(axis=1, keepdims=True))   # right

[  2.  20. 200.]
[[  -1.  -18. -197.]
 [   8.    0. -170.]
 [  98.  180.  100.]]
[[  -1.    0.    1.]
 [ -10.    0.   10.]
 [-100.    0.  100.]]

X.mean(axis=1) has shape (3,), and broadcasting aligns shapes on the right, so it's compared with the columns of X. With a (5, 3) matrix NumPy would refuse with an error. With a square one it happily subtracts row 0's mean from column 0, row 1's mean from column 1, and so on: garbage with the right shape, which is how a bug survives into a model.

Tip: After any reduction you plan to broadcast back (mean, sum, max, std along an axis), use keepdims=True.

The chapter also covers views and copies, why c[c > 2][0] = 99 silently does nothing, einsum as named loops, integer overflow and the NumPy 2 promotion rules, why vectorized code is 1,400 times faster than a loop here, and random generators: chapter C.

D. pandas: a join that almost doubles your revenue

pandas is NumPy plus labels. Every binary operation between two Series is an outer join on the index, which is where the NaNs that "appear out of nowhere" come from, and the miniature, MiniSeries, does it with one dictionary lookup per label. Joins are also where some of the most expensive bugs live, because they don't raise anything:

import pandas as pd

orders = pd.DataFrame({"customer": ["ana", "ana", "bo"], "amount": [10, 20, 5]})
emails = pd.DataFrame({"customer": ["ana", "ana", "bo"], "email": ["a@x", "ana@y", "b@x"]})
m = orders.merge(emails, on="customer")
print(len(orders), len(m))
print(orders["amount"].sum(), m["amount"].sum())

3 5
35 65

Ana has two email addresses, so each of her orders matched twice. Three orders became five rows, and the revenue went from 35 to 65. A model trained on m sees Ana's orders twice, and a dashboard built on it reports almost double the revenue. Telling pandas what kind of join you think you're doing turns the silence into an error:

orders.merge(emails, on="customer", validate="many_to_one")

MergeError: Merge keys are not unique in right dataset; not a many-to-one merge

The chapter rebuilds alignment, labels versus positions, copy-on-write (the pandas 3 change that broke old tutorials, built as a buffer that counts its readers) and split-apply-combine, and checks it all on 1,700 random cases. And it shows what read_csv guesses when nobody tells it anything:

import io

raw = """customer_id,zip,signup,amount,vip
000123,02134,2026-01-05,19.99,yes
000124,10001,2026-01-06,,no
000125,94105,2026-01-07,"1,200.50",yes
"""
print(pd.read_csv(io.StringIO(raw)).dtypes)

customer_id    int64
zip            int64
signup           str
amount           str
vip              str
dtype: object

Every single column is a little wrong, and not one of them raised a damn thing. The IDs and the zip codes lost their leading zeros ("02134", a Boston zip code, is now 2134). amount is text because one value has a thousands separator, so .sum() glues the strings together into '19.991,200.50'. The dates are text too. The fixes (dtype=, parse_dates=, thousands=) and the date trap that dayfirst=True sets are in chapter D.

E. scikit-learn: 85% accuracy on coin flips

scikit-learn is mostly a contract, and you can write estimators that follow it and let scikit-learn grade them. A first, naive linear regression isn't even graded until it inherits from BaseEstimator, and then it fails 17 of the library's own checks (check_estimator). After renaming the learned attributes, adding check_is_fitted and validating the input with validate_data, the miniature passes all 51. The contract is also what makes pipelines work, and pipelines are what protect you from this:

import numpy as np
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline

rng = np.random.default_rng(0)
X = rng.normal(size=(100, 5000))          # 100 samples, 5,000 features of pure noise
y = rng.integers(0, 2, size=100)          # labels that have nothing to do with X

# wrong: choose the 20 "best" features using all the data, then cross-validate
X_selected = SelectKBest(f_classif, k=20).fit_transform(X, y)
wrong = cross_val_score(LogisticRegression(), X_selected, y, cv=5)

# right: the selection is part of the model, so it's refitted inside every fold
model = make_pipeline(SelectKBest(f_classif, k=20), LogisticRegression())
right = cross_val_score(model, X, y, cv=5)

print(f"selection outside CV: {wrong.mean():.2f}")
print(f"selection inside CV:  {right.mean():.2f}")

selection outside CV: 0.85
selection inside CV:  0.48

The labels are coin flips, and the wrong version reports 85% accuracy. Out of 5,000 noise features, some are correlated with the coin flips by chance in this sample, and the selection found them using all 100 labels, including the ones the cross-validation later pretends not to know. Inside the pipeline, the selection is refitted on each training fold and the score drops to 48%, which is what a coin flip deserves. It's literally a textbook example, the "wrong and right way to do cross-validation" in The Elements of Statistical Learning, section 7.10.2, and a survey found leakage in 294 published papers across 17 scientific fields (Kapoor and Narayanan, 2023).

Warning: Anything that learns from data is part of the model and must be fitted only on training data: scalers, imputers, encoders, feature selection, PCA, oversampling. Put it in a Pipeline.

The chapter builds the estimator contract step by step, shows why the constructor must not touch its arguments (get_params reads your __init__ signature, and clone trusts it), writes a pipeline and a grid search in a few dozen lines that match scikit-learn's exactly, and handles real tables with ColumnTransformer: chapter E.

F. PyTorch: half a network that never trains

A tensor is NumPy's array again, autograd is a tape that records every operation and replays it backwards, and nn.Module finds your parameters by watching what you assign to it: it overrides __setattr__. The miniature builds all three on top of NumPy, plus SGD and Adam, and matches PyTorch on 359 checks, gradients and optimizer trajectories included. Knowing how registration works explains this one:

import torch
import torch.nn as nn


class HalfBroken(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden = [nn.Linear(4, 4)]          # a plain list: not registered
        self.head = nn.Linear(4, 1)              # registered

    def forward(self, x):
        return self.head(torch.relu(self.hidden[0](x)))


torch.manual_seed(0)
model = HalfBroken()
print([n for n, _ in model.named_parameters()])

before = model.hidden[0].weight.detach().clone()
opt = torch.optim.SGD(model.parameters(), lr=0.1)
x, y = torch.randn(64, 4), torch.randn(64, 1)
for _ in range(50):
    opt.zero_grad()
    loss = ((model(x) - y) ** 2).mean()
    loss.backward()
    opt.step()

print(torch.equal(before, model.hidden[0].weight), model.hidden[0].weight.grad is not None)

['head.weight', 'head.bias']
True True

The hidden layer got gradients (the second True), but the optimizer was given model.parameters(), which doesn't include it, so its weights are exactly what they were before training. The loss still goes down a bit, because the head is learning, and nothing crashes. nn.ModuleList is the fix, and sum(p.numel() for p in model.parameters()) is the one-line check.

The same model explains why every training loop starts with zero_grad: a leaf tensor adds each new gradient to the one it already has. Here's a one-feature linear regression (the true weight is 3 and the bias −2) trained for 30 steps with and without it, showing the loss at steps 0, 9, 19 and 29, then the learned weight and bias:

with zero_grad    ([9.5716, 2.6276, 0.9376, 0.4191], [1.957, -1.892])
without zero_grad ([9.5716, 6.6228, 8.9934, 8.1757], [0.497, -4.482])

No error, just a model that doesn't learn, and a bug that looks like "bad hyperparameters".

The chapter also covers in-place operations and the version counter that protects your gradients, train() versus eval(), what state_dict really saves, and a DataLoader miniature whose first test failure revealed a detail of how PyTorch seeds its workers: chapter F.

G. sentence-transformers: when the miniature disagrees

model.encode() is four steps: a WordPiece tokenizer, a BERT encoder, a masked average of the token vectors, and a normalization to length 1. The chapter rebuilds all four down to plain Python and NumPy, with the real vocabulary and the real weights of all-MiniLM-L6-v2 (a 90.9 MB model), and the vectors match model.encode within 2.6e-07. Along the way it shows two silent behaviors. The first is padding. In a batch, short texts are padded to the length of the longest one, and the transformer computes vectors for the padding positions too. With the miniature's functions, average them in by accident and see what happens to "short" when it shares a batch with a long sentence:

ids, mask = tok.batch(["short", "a much longer sentence that forces a lot of padding onto the first one"])
tokens = bert(W, ids, mask)
good = normalize(mean_pool(tokens, mask))
naive = normalize(tokens.mean(axis=1))
alone = model.encode(["short"])[0]
print(float(good[0] @ alone), float(naive[0] @ alone))

1.0 0.6889

With the masked mean, "short" gets the same vector it gets alone. With the plain mean, its similarity to itself is 0.69, and the vector of a text now depends on which other texts happened to be in its batch. The second is that the 256-token limit is a wall:

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
text = " ".join(["The experiment was a success."] * 50)        # 302 tokens with [CLS] and [SEP]
a = model.encode([text + " Actually, it failed completely."])
b = model.encode([text])
print(np.array_equal(a, b))

True

The conclusion changed completely, and the embedding didn't change at all, because everything after token 256 is cut off before the transformer sees it, without a warning.

Then the best part. I sent every Unicode code point from U+0020 to U+2FFFF through my tokenizer and through the one sentence-transformers actually runs:

48575 of 194528 code points disagree
 47750  unassigned in Unicode 15.1.0
   419  other (category Mn)
   256  CJK U+2B820 to U+2B91F

(plus 150 more, spread over smaller categories). My rules are the ones in Google's original BERT tokenizer. The library behind sentence-transformers today is Hugging Face's Rust tokenizers, and its source (version 0.23.3) explains all of it: a Unicode table from 2016, a function documented as covering unassigned characters that doesn't check them, and a typo, 0x2B920 where Google's code has 0x2B820. For real text it barely matters, and all 2,000 realistic test texts matched exactly. But "the same tokenizer" turns out to mean "the same implementation", and if you ever train with one and serve with another, test them on your own text.

Full chapter, BERT in about forty lines of NumPy included: chapter G.

H. From copying to writing

The miniatures give you the model. The addendum's chapter H is about using it on problems no tutorial covered, with six habits that cost nothing: predict before you run, rewrite yesterday's code from memory, translate between libraries, break things on purpose, read the source of one function a week, and write a three-line test for anything you believe about a library. It also suggests an order to learn things in, six projects that each have a "done when" (all of them run on a laptop without a GPU), a setup for a modest Windows machine (WSL2 and the CPU-only PyTorch build, which saved me about 3 GB of NVIDIA libraries that would never run), and prompts that keep an assistant in the tutor's chair: chapter H.

I. Getting the addendum

Everything is in the branch addendum1/learning-ml-libraries-by-rebuilding-them of the course repository: eleven chapters (the introduction, the method, the five libraries, the habits, an FAQ with a cheat sheet that has every command on one page, a few final words and twenty references), the miniatures with their tests, and the exact versions. The chapters stand on their own, so if fit_transform is what bothers you, start at chapter E. To run every check yourself, on a CPU, in a few minutes:

git clone --branch addendum1/learning-ml-libraries-by-rebuilding-them https://github.com/BlueRingsLabs/no-shortcuts
cd no-shortcuts
pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
cd code && python test_mini_numpy.py

805 checks against NumPy 2.5.4: all passed

and the same for the other four (test_mini_pandas.py, sk/test_mini_sklearn.py, torch_ch/test_mini_torch.py, st/test_mini_st.py). GitHub's CI runs all of them on every push to the branch.

J. FAQ

Should I start with the docs, a tutorial, a video or a project?

One short tutorial, to get something running and see the library's shape. Then the explanation pages for the data model and the conventions. Then a miniature, and then a small project without a tutorial. Videos are fine for motivation, but they're the most passive of all of these, and the copy-paste phase feeds on passive.

Is it normal to look up syntax all the time?

Completely normal, even after years. Memorize the data model and the conventions of each library you use, and let the everyday functions stick by themselves through use.

Which libraries should I learn, and in what order?

NumPy, pandas, enough matplotlib to look at your data, scikit-learn, PyTorch, then Hugging Face and the RAG tools. The order matters more than the speed.

Isn't using an AI assistant just the modern way of learning?

It's the modern way of working, and you should get good at it. While you're learning a library, though, the evidence points the same way as common sense: when the assistant produces the code, you learn less. Use it to quiz you and to explain, and keep the writing for yourself.

What if my miniature disagrees with the real library?

That's a good outcome. Either you've misunderstood something, and now you know exactly what, or you've found a corner of the library that surprised you, which is worth knowing before it surprises you in production. Section G above is the extreme case.

Do I need the No Shortcuts course to use this?

No. The addendum stands on its own. The course is where these libraries get used for real, and the end of each chapter points to the relevant lessons.

K. Final thoughts

If something in here, or in the addendum, is wrong, and in something this long something probably is, tell me in the comments or open an issue on the repository. I'd much rather fix it than leave it. And if this helped you, the best thanks is sharing it with someone who's stuck copy-pasting. Thanks for reading :B

L. References

Listed in order of first appearance.

\[1\] J. H. Shen and A. Tamkin, "How AI impacts skill formation," arXiv:2601.20245, 2026. https://arxiv.org/abs/2601.20245

\[2\] N. J. Slamecka and P. Graf, "The generation effect: Delineation of a phenomenon," Journal of Experimental Psychology: Human Learning and Memory, vol. 4, no. 6, pp. 592–604, 1978. https://doi.org/10.1037/0278-7393.4.6.592

\[3\] S. Bertsch, B. J. Pesta, R. Wiscott and M. A. McDaniel, "The generation effect: A meta-analytic review," Memory & Cognition, vol. 35, no. 2, pp. 201–210, 2007. https://doi.org/10.3758/BF03193441

\[4\] K. Morehead, J. Dunlosky and K. A. Rawson, "How much mightier is the pen than the keyboard for note-taking? A replication and extension of Mueller and Oppenheimer (2014)," Educational Psychology Review, vol. 31, no. 3, pp. 753–780, 2019. https://doi.org/10.1007/s10648-019-09468-2

\[5\] G. L. Romero, "No Shortcuts: Data, Machine Learning, Deep Learning and AI Engineering, the long way round," GitHub, 2026. https://github.com/BlueRingsLabs/no-shortcuts

\[6\] D. Procida, "Diátaxis: a systematic approach to technical documentation authoring." https://diataxis.fr/

\[7\] T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning, 2nd ed., Springer, 2009, sec. 7.10.2. https://hastie.su.domains/ElemStatLearn/

\[8\] S. Kapoor and A. Narayanan, "Leakage and the reproducibility crisis in machine-learning-based science," Patterns, vol. 4, no. 9, art. 100804, 2023. https://doi.org/10.1016/j.patter.2023.100804