Glossary (exhaustive)¶

This appendix is the single alphabetical reference for every term used across the book, from cgroups to the Bellman equation. Each entry does three things: it defines the term precisely, it explains why it matters in the workflow the book teaches, and — whenever the term has a mathematical basis — it derives the formula and computes it on real numbers, so you never have to take a symbol on faith. Full multi-step derivations (SVD, backpropagation, KL divergence family, Bellman optimality) live in Appendix D; this glossary gives you the compressed, load-bearing version you can recall in a design review or an interview.

Mental model. The book's arc is DevOps → MLOps → AI: you first learn to run any workload reliably (Linux, networking, git, containers, Kubernetes, IaC, CI/CD, security, observability — the platform layer); then you learn to run data and model workloads reliably on that platform (Python data stack, data engineering, DVC, MLflow, MLOps patterns — the ML platform layer); then you learn the models themselves (classical ML, deep learning, NLP/LLMs, computer vision, generative AI, reinforcement learning, time series — the applied AI layer). Every term below belongs to exactly one of these layers, and most of the confusion between "the ML person" and "the platform person" on a team comes from silently assuming the other one means the same thing by a shared word (state, policy, pipeline, deployment). After this appendix you will be able to: (1) place any unfamiliar term into the right layer in one glance, (2) reconstruct the formula behind it without looking it up, and (3) spot when two team members are using the same word for two different concepts.

flowchart TD
    subgraph Platform["Platform layer — DevOps"]
        L1["Linux & Shell (Ch.1)"] --> L2["Networking (Ch.2)"]
        L2 --> L3["Git (Ch.3)"]
        L3 --> L4["Docker (Ch.4)"]
        L4 --> L5["AWS (Ch.5)"]
        L5 --> L6["Kubernetes (Ch.6)"]
        L6 --> L7["Terraform (Ch.7)"]
        L7 --> L8["CI/CD (Ch.8)"]
        L8 --> L9["Security (Ch.9)"]
        L9 --> L10["Observability (Ch.10)"]
    end
    subgraph MLPlatform["ML platform layer — MLOps"]
        L11["Python data stack (Ch.11)"] --> L12["Data engineering (Ch.12)"]
        L12 --> L13["ML foundations (Ch.13)"]
        L13 --> L14["DVC (Ch.14)"]
        L14 --> L15["MLflow (Ch.15)"]
        L15 --> L16["MLOps patterns (Ch.16)"]
    end
    subgraph AppliedAI["Applied AI layer"]
        L17["Classical ML (Ch.17)"]
        L18["Deep learning (Ch.18)"]
        L19["NLP & LLMs (Ch.19)"]
        L20["Computer vision (Ch.20)"]
        L21["Generative AI (Ch.21)"]
        L22["Reinforcement learning (Ch.22)"]
        L23["Time series (Ch.23)"]
    end
    Platform --> MLPlatform --> AppliedAI --> Capstone["Capstone (Ch.24)"]

How to use this glossary¶

Entries are grouped into four alphabetical bands (A–C, D–H, I–O, P–Z) purely to keep each band scrollable; within a band, terms are strictly alphabetical. Each entry ends with the chapter(s) that develop it in full — click through when you need the complete treatment, code, and exercises. A recurring pattern you will see below is define → derive → compute: take Little's law as the smallest possible illustration. It states that in steady state, the average number of items in a system equals the arrival rate times the average time an item spends in the system:

\[L = \lambda W\]

Here $L$ is the average number of concurrent requests "in flight," $\lambda$ is the arrival rate (requests per second), and $W$ is the average time each request spends in the system (seconds). If your API receives $\lambda = 50$ requests/s and each takes on average $W = 0.2\,\text{s}$ to answer, then on average $L = 50 \times 0.2 = 10$ requests are being processed concurrently at any instant — which is exactly the number of worker threads/pods you must be able to run in parallel to avoid queueing. Every math-bearing entry below follows this same three-step discipline.

A–C¶

ACID — Atomicity, Consistency, Isolation, Durability: the four guarantees a transactional data store makes so that concurrent writes never leave the database in a half-updated, contradictory state. Atomicity means a transaction is all-or-nothing; Isolation means concurrent transactions don't see each other's half-finished work; Durability means a committed write survives a crash. Lakehouse formats (Delta Lake, Iceberg, Hudi) exist precisely to bring ACID semantics to files sitting on object storage, which historically had none. (Ch. 12)
Advantage $A(s,a)$ — how much better than the average action a specific action $a$ is in state $s$: $A(s,a) = Q(s,a) - V(s)$, where $Q(s,a)$ is the expected return of taking $a$ in $s$ and $V(s)$ is the expected return of state $s$ under the current policy. Worked example: if $Q(s,a)=10$ and $V(s)=7$, then $A(s,a)=3$: this action beats the state's average outcome by 3 units of return, so a policy-gradient update should increase its probability. Using the advantage instead of the raw return $Q(s,a)$ subtracts a state-dependent baseline, which does not bias the gradient's expectation but sharply reduces its variance — the single most important variance-reduction trick in policy-gradient methods. (Ch. 22)
AMI (Amazon Machine Image) — the frozen template (root filesystem + boot config) an EC2 instance is launched from. Bake application dependencies into a custom AMI (via Packer, for example) to cut cold-boot time versus provisioning everything with a startup script every time. (Ch. 5)
ANN (Approximate Nearest Neighbor) search / vector database — the retrieval engine behind RAG (below): embeddings (below) are high-dimensional dense vectors, and finding the true $k$ nearest neighbors of a query vector by brute force is $O(n)$ per query over $n$ stored vectors — far too slow once $n$ is in the millions. ANN indexes (HNSW — Hierarchical Navigable Small World graphs, or IVF — Inverted File index with product quantization, as used by FAISS) trade a small, tunable amount of recall for query time close to $O(\log n)$ by pre-building a graph or clustering structure that lets search skip most of the vector space. A vector database (Pinecone, Weaviate, pgvector, Qdrant) packages an ANN index with metadata filtering, persistence, and horizontal scaling into a queryable service. Similarity is usually cosine similarity, $\cos(\theta) = \dfrac{u\cdot v}{\lVert u\rVert\lVert v\rVert}$: worked example, $u=[1,2]$, $v=[2,1]$ give $u\cdot v = 1\times2+2\times1=4$, $\lVert u\rVert=\lVert v\rVert=\sqrt5\approx2.236$, so $\cos(\theta)=4/(2.236\times2.236)=4/5=0.8$ — fairly similar, but not identical, direction. (Ch. 19)
API design (REST vs. gRPC vs. GraphQL) — three common contracts for a service boundary. REST models a system as resources addressed by URLs and manipulated with HTTP verbs (GET/POST/PUT/DELETE), is human-readable (usually JSON) and cacheable by ordinary HTTP infrastructure, but tends to either over-fetch (an endpoint returns fields the client doesn't need) or under-fetch (the client must chain several calls). GraphQL flips this: the client sends a query describing exactly the fields/relations it wants, and the server resolves precisely that shape in one round trip — solving over/under-fetching at the cost of losing simple HTTP caching and needing query-complexity limits to prevent a single request from fanning out into an expensive resolver tree. gRPC compiles a strongly-typed schema (Protocol Buffers) into client/server stubs in every target language, communicates over HTTP/2 with binary serialization (much smaller and faster to parse than JSON), and natively supports bidirectional streaming — the default choice for low-latency internal service-to-service calls, at the cost of not being directly browser- or curl-friendly. (Ch. 2)
ARIMA(p, d, q) — AutoRegressive Integrated Moving Average, the classical linear model for a stationary (after differencing $d$ times) time series: it regresses the current value on its own past $p$ values (AR term) and on past forecast errors' $q$ terms (MA term). It remains a strong, cheap, interpretable baseline that every deep-learning forecaster must beat before being trusted in production. (Ch. 23)
ARN (Amazon Resource Name) — the globally unique identifier of any AWS resource, e.g. arn:aws:s3:::my-bucket/key. IAM policies grant or deny actions on ARNs (with wildcards), never on human-friendly names, which is why a typo'd ARN in a policy silently grants access to nothing rather than erroring. (Ch. 5)
Attention — a content-based routing mechanism that lets each output position look at every input position and weigh them by relevance, rather than only at a fixed nearby window. Scaled dot-product attention is $$\text{Attention}(Q,K,V) = \text{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,$$ where $Q$ (queries), $K$ (keys), $V$ (values) are learned linear projections of the input, and $\sqrt{d_k}$ rescales the dot products so softmax doesn't saturate for large key dimension $d_k$. Worked toy example: with $d_k=2$, query $Q=[1,0]$ and keys $K_1=[1,0]$, $K_2=[0,1]$, the raw scores are $Q\cdot K_1/\sqrt2 = 1/1.414 = 0.707$ and $Q\cdot K_2/\sqrt2 = 0$. Softmax over $[0.707, 0]$ gives $e^{0.707}=2.028$ and $e^{0}=1$, sum $=3.028$, so the attention weights are $[0.670,\,0.330]$: token 1 receives 67% of the attention, token 2 the remaining 33%. This is the core primitive of the Transformer. (Ch. 19)
AUC / ROC curve — the Receiver Operating Characteristic plots the true-positive rate against the false-positive rate as a classifier's decision threshold sweeps from 0 to 1; the Area Under that Curve is a single threshold-independent number between 0.5 (random) and 1.0 (perfect ranking). AUC equals the probability that a randomly chosen positive example is scored higher than a randomly chosen negative one — useful when the operating threshold isn't fixed yet, but it can look deceptively good on very imbalanced data (prefer precision-recall AUC there). (Ch. 13, Ch. 17)
Autoencoder — a network trained to reconstruct its own input through a narrow bottleneck, forcing it to learn a compressed representation. The plain (deterministic) autoencoder learns compression only; the Variational Autoencoder (VAE, below) turns the bottleneck into a probability distribution so you can sample new, never-seen data from it. (Ch. 18, Ch. 21)
Autoscaling (HPA / VPA / CA) — the family of Kubernetes controllers that add/remove replicas (Horizontal Pod Autoscaler), resize a pod's CPU/memory requests (Vertical Pod Autoscaler), or add/remove worker nodes (Cluster Autoscaler) in response to observed load. HPA and VPA should not target the same metric on the same workload — they will fight each other. (Ch. 6)
AZ (Availability Zone) — one or more physically isolated datacenters within an AWS Region, each with independent power/cooling/networking. Spreading instances across $\ge 2$ AZs is the cheapest form of high availability: an AZ-wide outage (power, fiber cut) takes down at most half your fleet. (Ch. 5)
Backoff / retry / jitter — the standard pattern for calling a flaky or overloaded remote dependency: on failure, retry, but wait progressively longer between attempts (exponential backoff, $\text{wait}_n = \min(\text{cap},\, \text{base}\times 2^n)$) so a struggling downstream service isn't hit harder the moment it starts failing. Naive exponential backoff still synchronizes many clients' retries into the same instant (they all failed at once and all back off by the same schedule), which can produce a "retry storm" wave — jitter breaks this by randomizing the wait within a range, e.g. $\text{wait}_n = \text{random}(0,\, \text{base}\times 2^n)$ ("full jitter"). Worked example: with base $=100\,\text{ms}$ and cap $=10\,\text{s}$, attempt $n=5$ gives an un-jittered wait of $\min(10000, 100\times2^5)=\min(10000,3200)=3200\,\text{ms}$; full jitter instead draws uniformly from $[0, 3200]\,\text{ms}$, spreading a thundering herd of simultaneous retries across more than three seconds instead of concentrating them at exactly 3.2 s. Always pair retries with a maximum attempt count and, ideally, a circuit breaker (below) so a permanently down dependency doesn't retry forever. (Ch. 2)
Backpropagation — the algorithm that computes $\partial L/\partial\theta$ for every parameter $\theta$ in a network with $L$ layers, by applying the chain rule from the loss backward to the inputs, reusing intermediate derivatives instead of recomputing them per-parameter (which would be exponential). It is the engine that makes gradient descent (below) tractable for millions/billions of parameters. (Ch. 18)
Batch normalization — a layer that re-centers and re-scales its input's activations to zero mean and unit variance per mini-batch (then applies a learned scale $\gamma$ and shift $\beta$), which stabilizes and speeds up training by keeping activation distributions from drifting layer to layer ("internal covariate shift"). At inference time it uses a running average of mean/variance collected during training rather than the current (possibly batch-size-1) batch statistics — a common production bug is forgetting to switch the layer to eval mode, which silently reverts to batch statistics on a single request. (Ch. 18)
Batch size / epoch / iteration — a batch is the subset of training examples used to compute one gradient update; an iteration (or "step") is one such update; an epoch is one full pass over the entire training set. With 10,000 examples and batch size 100, one epoch is 100 iterations. Larger batches give a less noisy gradient estimate (lower variance) but each step is more expensive and, empirically, very large batches can generalize slightly worse without a matching learning-rate increase. (Ch. 18)
Bayesian optimization — a hyperparameter-search strategy that fits a probabilistic surrogate model (typically a Gaussian Process) over "hyperparameters → validation score" observed so far, then picks the next configuration to try by maximizing an acquisition function (e.g. Expected Improvement) that trades off exploring uncertain regions against exploiting known-good ones. It needs far fewer trials than grid or random search to reach a good optimum, which matters when each trial is a multi-hour training run. (Ch. 16, Ch. 17)
Bellman equation — the recursive identity that ties the value of a state to the value of its successor states, the foundation of dynamic programming and of almost every RL algorithm: $$V^\pi(s) = \mathbb{E}_{a\sim\pi}\!\left[R(s,a) + \gamma\, V^\pi(s')\right].$$ It says: the value of being in state $s$ under policy $\pi$ equals the immediate reward plus the (discounted) value of wherever you land next. Q-learning and value iteration are both fixed-point iterations on this equation. (Ch. 22)
Bias–variance decomposition — the identity that splits a model's expected test error into three additive terms: $\text{Error} = \text{Bias}^2 + \text{Variance} + \text{Irreducible noise}$. High bias is underfitting (the model class is too simple to capture the pattern — e.g. a line through curved data); high variance is overfitting (the model chases noise specific to the training sample and won't generalize). Regularization (below) and more data both primarily attack variance; a bigger/more expressive model attacks bias. (Ch. 13)
Blue-green / Canary deployment — two zero/low-downtime release strategies. Blue-green keeps two full environments ("blue" = current, "green" = new) and switches all traffic atomically once green is validated — instant rollback is just switching back. Canary instead routes a small percentage of live traffic (e.g. 5%) to the new version, watches error/latency metrics, then ramps up gradually — it limits the blast radius of a bad release rather than avoiding downtime. (Ch. 8)
Boosting (Gradient Boosting, XGBoost, LightGBM) — an ensemble technique that builds trees sequentially, each new tree trained to predict the current ensemble's residual errors (technically, the negative gradient of the loss with respect to the current predictions), so the ensemble as a whole keeps reducing training loss step by step. It typically outperforms a Random Forest on tabular data at the cost of being more sensitive to overfitting and requiring more careful learning-rate/early-stopping tuning. Contrast with bagging (Random Forest), which builds trees independently and in parallel on bootstrap resamples and reduces variance by averaging, without the residual-fitting step. (Ch. 17)
BPE (Byte-Pair Encoding) — the standard subword tokenization algorithm for LLMs: start from individual bytes/characters, repeatedly merge the most frequent adjacent pair into a new token, until a target vocabulary size is reached. This lets a fixed vocabulary (e.g. 50k tokens) represent any string, including misspellings and unseen words, by falling back to smaller pieces. (Ch. 19)
Broadcasting — NumPy/PyTorch/TensorFlow's rule for applying an elementwise operation between arrays of different shapes by virtually stretching the smaller one along size-1 axes, without copying memory. Adding a shape-$(3,)$ vector to a shape-$(4,3)$ matrix broadcasts the vector across all 4 rows; the rule that makes this legal is that trailing dimensions must match or be 1. (Ch. 11)
CAP theorem — a distributed data store facing a network partition must choose between Consistency (every read sees the most recent write, or an error) and Availability (every request gets a non-error response, possibly stale); you cannot have both during a partition ($P$ is not really a choice — networks do partition). Postgres/etcd (used for Kubernetes' own cluster state, above) are CP: they refuse to serve a read/write rather than risk an inconsistent answer when a quorum is unreachable. DynamoDB and Cassandra default to AP: they keep answering during a partition and reconcile divergent replicas afterward (eventual consistency). Neither choice is "better" in the abstract — a payments ledger wants CP, a shopping-cart "add to cart" button wants AP. (Ch. 12)
cgroups (control groups) — the Linux kernel feature that limits and accounts for a process group's CPU, memory, I/O, and network usage. Docker/Kubernetes resource limits/requests are a thin, friendly API over cgroups; an OOMKilled container (below) is the kernel enforcing a cgroup memory limit. (Ch. 1, Ch. 4)
Chaos engineering — deliberately injecting failure into a running system (kill a random pod, add latency to a network call, exhaust a disk) in production or a production-like environment, on a controlled schedule, to verify that the redundancy/failover you designed on paper actually works under real conditions rather than only in the architecture diagram. Netflix's Chaos Monkey (randomly terminating instances) is the canonical example; the discipline generalizes to "game days" that rehearse an incident response before a real one forces the rehearsal to happen live. (Ch. 10)
CIDR (Classless Inter-Domain Routing) — the notation for an IP address block by prefix length, e.g. 10.0.0.0/16 means the first 16 bits are fixed (the network part) and the remaining 16 bits are host addresses, giving $2^{16}=65{,}536$ addresses. A /24 gives $2^{8}=256$ addresses (254 usable after the network and broadcast addresses) — the size you'll see on most VPC subnets. (Ch. 2, Ch. 5)
Circuit breaker — a client-side pattern that stops calling a downstream dependency once it has failed too often recently, failing fast (locally, instantly) instead of piling up slow timeouts against a dependency that is already down. It behaves like a state machine: closed (calls flow normally, failures counted) → trips to open (calls fail immediately without even attempting the network call) after the failure rate crosses a threshold in a rolling window → after a cooldown, half-open (let a single probe call through) → closed again if it succeeds, back to open if it doesn't. Combined with backoff/retry (above), a circuit breaker is what prevents one slow dependency from cascading into a full outage by exhausting every caller's thread pool waiting on it. (Ch. 2)
CloudWatch — AWS's native metrics, logs, and alarms service: every managed AWS resource (EC2, RDS, Lambda, ALB…) emits metrics here automatically, and you can define alarms that page you or trigger autoscaling when a metric crosses a threshold. It's the AWS-native alternative/complement to Prometheus + Grafana. (Ch. 5, Ch. 10)
CNI (Container Network Interface) — the plugin interface Kubernetes uses to give every pod a routable IP address and wire up pod-to-pod networking (Calico, Cilium, Flannel are common implementations); the choice of CNI also determines whether NetworkPolicy (below) is enforced at all. (Ch. 6)
ConfigMap / Secret (Kubernetes) — key-value objects mounted into pods as environment variables or files; a ConfigMap holds non-sensitive configuration, a Secret holds sensitive values (base64-encoded, not encrypted, by default — encryption at rest must be configured separately). Neither is versioned: changing one does not automatically restart pods that already read it. (Ch. 6)
Confusion matrix — the 2×2 (or $k\times k$) table of predicted vs. actual class counts that every binary classification metric is computed from: True Positives (TP), False Positives (FP), False Negatives (FN), True Negatives (TN). Worked example: TP=40, FP=10, FN=20, TN=930 (a 1000-example imbalanced set) gives precision $=40/(40+10)=0.80$, recall $=40/(40+20)=0.667$ — see F1 / precision / recall below for the combined score. (Ch. 13)
Container — a process (or process group) isolated by Linux namespaces (its own view of PIDs, network, filesystem mounts) and resource-limited by cgroups; unlike a VM, it shares the host kernel, which is why containers start in milliseconds instead of tens of seconds. (Ch. 4)
Convolution — a sliding, weight-shared linear filter applied over a spatial (or temporal) input: the same small kernel of weights (e.g. 3×3) is applied at every position, which gives the network translation equivariance (a shifted input produces a correspondingly shifted output) and drastically fewer parameters than a fully-connected layer over the same input. It is the core primitive of CNNs for images and 1D-convolutional models for sequences. (Ch. 18, Ch. 20)
CPCV (Combinatorial Purged Cross-Validation) — a cross-validation scheme for time series/financial data that (a) never trains on data that comes after a test fold chronologically without purging the overlap, and (b) additionally embargoes a buffer of samples immediately around each test fold, because overlapping labels (e.g. a triple-barrier label spanning several future bars) leak information across adjacent folds. Plain k-fold CV on time series without purging/embargo silently overstates performance — the single most common cause of backtests that "worked" and then failed live. (Ch. 23)
Cross-entropy — the standard classification loss, measuring how surprised a predicted distribution $\hat p$ is by the true label $y$: $$L = -\sum_{k} y_k \log \hat p_k.$$ For a one-hot true label (only class $k^*$ has $y_{k^*}=1$), this collapses to $L = -\log \hat p_{k^*}$ — the loss only depends on the probability mass the model put on the correct class. Worked example: a 3-class model outputs $\hat p = [0.1, 0.2, 0.7]$ and the true class is the third one; the loss is $L = -\log(0.7) = 0.357$ nats. If the model had instead been confidently wrong ($\hat p_{k^*}=0.01$), the loss would jump to $-\log(0.01)=4.605$ nats — cross-entropy punishes confident mistakes much more than cautious ones. (Ch. 13, Appendix D)
Cross-validation — estimating a model's generalization performance by repeatedly splitting the data into train/validation folds (commonly $k=5$ or $10$, "k-fold CV") and averaging the validation score across folds, so every example serves as validation data exactly once. For i.i.d. tabular data this is the default; for time series it must be replaced by walk-forward or purged/embargoed schemes (see CPCV) because plain k-fold shuffles time order and leaks the future into the past. (Ch. 13, Ch. 17)
CRD / Operator — a Custom Resource Definition extends the Kubernetes API with a new object kind (e.g. ExternalSecret); an Operator is a controller that watches instances of that kind and drives real-world state toward what they declare (a control loop, exactly like the built-in Deployment controller, but for a domain-specific concept). (Ch. 6, ESO — Ch. 9)
CUDA / GPU — CUDA is NVIDIA's parallel-computing platform/API that lets frameworks (PyTorch, TensorFlow) offload dense linear algebra (matrix multiplies, convolutions) to the GPU's thousands of simple cores, which is what makes training networks with billions of parameters feasible in practice — the same operation on a CPU's tens of cores would take orders of magnitude longer. GPU memory (VRAM), not compute, is usually the binding constraint on how large a model/batch you can train. (Ch. 18)
CVD (Cumulative Volume Delta) — a market-microstructure signal that running-sums (buy volume − sell volume) over time; a rising CVD alongside a flat or falling price is a classic divergence signal used in order-flow-based trading strategies. (Referenced in the book's trading case studies.)

D–H¶

DAG (Directed Acyclic Graph) — a graph with directed edges and no cycles, the natural data structure for anything with ordering dependencies and no circular waits: Airflow/dbt pipeline steps, CI/CD job graphs, and a neural network's computation graph are all DAGs. "Acyclic" is what guarantees a valid execution order (a topological sort) always exists. (Ch. 8, Ch. 12)
DaemonSet (Kubernetes) — ensures exactly one copy of a pod runs on every (or every matching) node in the cluster — the standard pattern for node-level agents like log shippers, CNI plugins, or Prometheus node-exporter, as opposed to a Deployment's arbitrary-N-replicas-anywhere model. (Ch. 6)
Data augmentation — synthesizing additional, label-preserving training examples by transforming existing ones (random crop/flip/rotation/color-jitter for images; back-translation or synonym swap for text; time-warping or noise injection for signals) so a model sees more variation than the raw dataset contains, without the cost of collecting/labeling new data. It is one of the cheapest and most reliable regularizers for deep learning, effectively fighting the variance side of the bias–variance trade-off (above) by widening the training distribution rather than by penalizing weights. (Ch. 18, Ch. 20)
Data lineage — the traceable record of where a dataset or feature came from and every transformation applied to it (source table → cleaning step → join → feature → model input), essential for debugging a bad prediction back to its root data cause and for audits/compliance. Tools like dbt and OpenLineage capture this automatically from pipeline definitions. (Ch. 12)
Data warehouse vs. data lake vs. lakehouse — a warehouse stores structured, schema-on-write data optimized for SQL analytics (Redshift, Snowflake, BigQuery); a lake stores raw files of any format cheaply on object storage, schema-on-read (S3 + Parquet); a lakehouse (Delta Lake, Iceberg, Hudi) adds ACID transactions, schema enforcement, and time travel on top of lake storage, aiming to get warehouse guarantees at lake cost. (Ch. 12)
Decision tree / Random forest — a decision tree recursively splits the feature space on the attribute/threshold that most reduces impurity (Gini or entropy for classification, variance for regression) at each node; a single tree overfits easily. A Random forest trains many trees on bootstrap-resampled data and random feature subsets per split, then averages their predictions (bagging) — the randomization decorrelates the trees' errors, so the ensemble's variance drops roughly by a factor of the number of trees (for uncorrelated errors) without increasing bias. (Ch. 17)
Deflated Sharpe Ratio (DSR) — the Sharpe ratio (below) corrected for the fact that testing many strategy variants and reporting only the best one inflates the apparent Sharpe purely by selection — exactly the multiple-comparisons problem in statistics. DSR asks: given that you tried $N$ independent variants, what Sharpe would you expect the best one to show by pure luck, and is your observed Sharpe meaningfully above that? A widely used back-of-envelope approximation for the expected maximum Sharpe under the null (no real skill) over $N$ independent trials is $\mathbb E[\max \widehat{SR}] \approx \sigma_{SR}\sqrt{2\ln N}$. Illustrative example: with $N=50$ backtested variants and a per-trial Sharpe standard error $\sigma_{SR}\approx 1$, the expected noise-only maximum is $\sqrt{2\ln 50} = \sqrt{2 \times 3.912} = \sqrt{7.824} \approx 2.80$ — so an observed Sharpe of 1.2 across 50 variants is not distinguishable from noise, even though 1.2 looks respectable in isolation. The exact Bailey & López de Prado formula (which also accounts for skew and kurtosis of returns) is derived in Appendix D. (Ch. 23)
Deployment (Kubernetes) — the controller that manages a ReplicaSet of identical pods and drives rolling updates/rollbacks: change the pod template's image tag, and the Deployment controller creates new pods and terminates old ones gradually according to its maxSurge/maxUnavailable strategy, keeping the service available throughout. (Ch. 6)
Diffusion model — a generative model trained to reverse a fixed process that gradually adds Gaussian noise to data over $T$ steps until it becomes pure noise; generation runs that reversal from pure noise back to a sample, denoising a little at each of the $T$ steps. It currently produces the highest-fidelity images of any generative family, at the cost of many sequential forward passes to sample. (Ch. 21)
Discount factor $\gamma$ — the per-step multiplier ($0 \le \gamma < 1$) that shrinks the value of future rewards in the return $G_t = \sum_{k=0}^{\infty}\gamma^k R_{t+k+1}$, both for numerical convergence (an infinite undiscounted sum can diverge) and to encode a preference for sooner rewards. With $\gamma=0.99$, a reward 100 steps away is worth only $0.99^{100}\approx 0.366$ of its face value today; with $\gamma=0.9$ the same reward is worth $0.9^{100}\approx 0.0000266$ — nearly discarded. Choosing $\gamma$ is choosing the agent's effective planning horizon. (Ch. 22)
Docker Compose — a tool/file format (docker-compose.yml) that defines and runs a multi-container application (app + database + cache, say) as a single unit on one host, wiring up a shared network and named volumes between them — the right tool for local development and single-host demos, not for production orchestration (that's Kubernetes' job). (Ch. 4)
Dockerfile — the declarative recipe (FROM, COPY, RUN, CMD…) that builds a container image layer by layer; each instruction creates a new, cached, read-only layer, which is why instruction order matters for build speed (put rarely-changing steps like dependency installation before frequently-changing steps like copying source code). (Ch. 4)
Drift (data / concept) — data drift is a shift in the input feature distribution (e.g. average transaction amount doubles); concept drift is a shift in the true relationship between inputs and target (the same input now implies a different label/outcome). Both degrade a deployed model's accuracy silently — no error is thrown, predictions just get worse — which is why production ML needs Population Stability Index (PSI, below), KL divergence, or Kolmogorov–Smirnov (KS) tests monitoring the live input distribution against the training distribution. (Ch. 16)
DNS (Domain Name System) — the hierarchical, cached name → IP-address resolution system; each record has a Time-To-Live (TTL) controlling how long resolvers may cache it, which is why lowering a TTL before a planned cutover (so the change propagates fast) is standard practice. (Ch. 2)
Dropout — a regularization technique that randomly zeroes a fraction $p$ of a layer's activations during each training step (and rescales the rest by $1/(1-p)$ to keep the expected sum constant), which prevents units from co-adapting to each other's specific quirks — a crude but effective approximation of training an ensemble of exponentially many thinned networks and averaging them. Dropout is disabled at inference time. (Ch. 18)
DVC (Data Version Control) — git-for-data: DVC replaces a large file/directory in git with a small pointer file (a content hash), and stores the actual bytes in a configured remote (S3, GCS, a plain server). dvc pull/dvc push sync data the same way git pull/git push sync code, so a git commit can pin an exact dataset version alongside the exact code version that produced a model. (Ch. 14)
Early stopping — a regularization technique that monitors validation loss during training and stops (or restores the best checkpoint) once it stops improving for a configured number of epochs ("patience"), rather than training for a fixed, arbitrarily chosen number of epochs. It directly targets the bias–variance trade-off (above): training loss keeps falling as the model increasingly memorizes the training set, but validation loss eventually turns upward — early stopping cuts training at that turning point, before the memorization phase dominates. (Ch. 18)
EC2 (Elastic Compute Cloud) — AWS's raw virtual-machine service: you pick an instance type (CPU/RAM/ GPU ratio), an AMI, and a network placement (VPC/subnet/security groups), and get a billed-by-the-second VM. It's the substrate under many higher-level AWS services (EKS worker nodes are EC2 instances, for example). (Ch. 5)
ECR (Elastic Container Registry) — AWS's managed Docker image registry; access tokens returned by aws ecr get-login-password are short-lived (typically 12 hours) and must be refreshed before any docker push/pull, and any Kubernetes imagePullSecret built from that token expires on the same schedule. (Ch. 5, Ch. 6)
EKS (Elastic Kubernetes Service) — AWS's managed Kubernetes control plane: AWS runs and patches the API server/etcd, you run and scale the worker nodes (EC2 or Fargate) that actually host your pods. (Ch. 5, Ch. 6)
ELBO (Evidence Lower BOund) — the objective a VAE (below) maximizes as a tractable stand-in for the true (intractable) data likelihood $\log p(x)$: $$\text{ELBO} = \mathbb{E}_{q(z\mid x)}\!\left[\log p(x\mid z)\right] - D_{\mathrm{KL}}\!\big(q(z\mid x)\,\|\,p(z)\big).$$ The first term rewards accurate reconstruction; the second penalizes the learned encoder distribution $q(z\mid x)$ for straying from a simple prior $p(z)$ (usually a standard Gaussian), which keeps the latent space smooth and sample-able. Worked example of the KL term alone: if $q(z\mid x) = \mathcal N(0.5, 1)$ and the prior is $p(z)=\mathcal N(0,1)$, the closed-form KL between two univariate Gaussians is $D_{\mathrm{KL}} = \ln\frac{\sigma_0}{\sigma_1} + \frac{\sigma_1^2 + (\mu_1-\mu_0)^2}{2\sigma_0^2} - \frac12$; with $\sigma_0=\sigma_1=1$ and $\mu_1=0.5,\mu_0=0$, this is $0 + \frac{1+0.25}{2} - 0.5 = 0.125$ nats — a small, tolerable penalty for an encoder that only shifted the mean slightly. (Ch. 21)
ELB / ALB (Elastic / Application Load Balancer) — AWS's managed load balancers: ALB operates at layer 7 (HTTP/HTTPS), routing by host/path and terminating TLS, and is what a Kubernetes Ingress on EKS typically provisions under the hood via the AWS Load Balancer Controller. (Ch. 5, Ch. 6)
Embedding — a dense, low-dimensional vector representation of a discrete object (a word, a user, an item) learned so that geometric proximity (cosine similarity or Euclidean distance) reflects semantic similarity. The famous example: $\text{embedding}(\text{king}) - \text{embedding}(\text{man}) + \text{embedding}(\text{woman}) \approx \text{embedding}(\text{queen})$ — arithmetic on meaning. (Ch. 19)
Ensemble methods — combining several models' predictions to get a result more accurate/robust than any single member: bagging averages independently trained models to reduce variance (Random Forest), boosting sequentially corrects residual errors to reduce bias (XGBoost), and stacking trains a meta-model on the base models' outputs. (Ch. 17)
Error budget — the allowed amount of unreliability implied by an SLO: if the SLO is 99.9% availability over 30 days, the error budget is $1 - 0.999 = 0.001$, i.e. $0.001 \times 30 \times 24 \times 60 = 43.2$ minutes of downtime you're allowed to "spend" — on risky deploys, experiments, maintenance — before you must freeze releases and focus purely on reliability. (Ch. 10)
ESO (External Secrets Operator) — a Kubernetes operator that syncs secrets from an external secret manager (AWS Secrets Manager, Vault…) into native Kubernetes Secret objects on a refresh interval, so the source of truth lives outside the cluster and secrets are never committed to git manifests. (Ch. 9)
etcd — the distributed, consistent key-value store that holds all Kubernetes cluster state (every object's desired spec and current status); it is the single most critical component to back up, because losing etcd means losing the cluster's memory of what should be running. (Ch. 6)
Exploration–exploitation trade-off — the fundamental RL dilemma: exploiting the best action known so far maximizes immediate expected reward, but exploring untried actions might reveal something even better and is necessary to ever discover the true optimum. Concrete mechanisms: $\varepsilon$-greedy (act randomly with probability $\varepsilon$, greedily otherwise), or entropy bonuses added to the loss (as in PPO, below) that reward the policy for staying stochastic. (Ch. 22)
F1 / precision / recall — from the confusion matrix (above): precision $= \text{TP}/(\text{TP}+ \text{FP})$ answers "of everything I flagged positive, how much was actually positive?"; recall $= \text{TP}/(\text{TP}+\text{FN})$ answers "of everything actually positive, how much did I catch?"; F1 is their harmonic mean, $F_1 = 2 \cdot \frac{\text{precision}\cdot\text{recall}}{\text{precision}+\text{recall}}$, which (unlike the arithmetic mean) heavily penalizes a model that is great on one and terrible on the other. Worked example (continuing the confusion-matrix entry, TP=40, FP=10, FN=20): precision $=0.80$, recall $=0.667$, so $F_1 = 2 \times 0.80 \times 0.667 / (0.80+0.667) = 1.067/1.467 \approx 0.727$. (Ch. 13)
Fargate — a serverless container runtime for ECS/EKS: you specify CPU/memory and a container image, AWS provisions and manages the underlying compute with no node to patch or scale yourself, billed per vCPU-second and GB-second actually used. (Ch. 5)
Feature engineering — transforming raw data into the inputs a model actually consumes: encoding categoricals (one-hot, target/mean encoding), scaling numerics, extracting date parts, building interaction terms, or computing domain signals (e.g. a rolling volatility for a trading model). Even in the deep-learning era, feature engineering remains decisive for tabular data, where raw deep nets routinely lose to a well-engineered gradient-boosted tree. (Ch. 13, Ch. 17)
Feature flag — a runtime on/off (or percentage-rollout) switch read from configuration rather than hardcoded, letting a team merge and deploy code continuously while controlling exposure separately from deployment: a half-finished feature can sit dark in production behind a flag, and a risky change can be enabled for 1% of traffic, then 10%, then 100%, watching metrics at each step — the same progressive- exposure idea as a canary deployment (above), but controlled by a config toggle instead of a separate release/rollback. (Ch. 8)
Feature store — a centralized system (e.g. Feast) that computes, stores, and serves features consistently for both offline training (batch, joined over history) and online inference (low-latency, point-in-time-correct lookup), eliminating training/serving skew caused by two independently maintained feature pipelines silently drifting apart. (Ch. 16)
FID (Fréchet Inception Distance) — a generative-image-quality metric that runs both real and generated images through a pretrained Inception network, fits a Gaussian to each set's feature activations, and measures the distance between the two Gaussians (mean and covariance); lower FID means the generated distribution is closer to the real one, both in average appearance and in diversity. (Ch. 21)
Fine-tuning vs. prompt engineering vs. RAG — three ways to specialize a pretrained model to a task without training from scratch: fine-tuning further updates the model's weights on task-specific data (full or parameter-efficient, e.g. LoRA below); prompt engineering only changes the input text at inference time, no weights touched; RAG (below) supplies the model with retrieved documents as context instead of, or in addition to, either. They compose — you can RAG a fine-tuned model with a carefully engineered prompt template. (Ch. 19)
for_each vs. count (Terraform) — both create multiple resource instances from one block, but count addresses them by numeric index (aws_instance.this[0]), so removing an item from the middle of the list shifts every later index and can force destroy/recreate of unrelated resources; for_each addresses them by a stable map/set key (aws_instance.this["prod"]), so removing one item only affects that item's address. Prefer for_each whenever the collection isn't a fixed, ordered count. (Ch. 7)
GAN (Generative Adversarial Network) — two networks trained against each other: a generator $G$ maps noise to fake samples, a discriminator $D$ tries to tell real from fake, and $G$ is trained to fool $D$. The minimax objective is $$\min_G \max_D\; \mathbb{E}_{x\sim p_{\text{data}}}[\log D(x)] + \mathbb{E}_{z\sim p_z}[\log(1-D(G(z)))].$$ At the (theoretical) equilibrium, $D$ outputs $0.5$ everywhere — it can no longer distinguish real from fake at all — which is also why GAN training is notoriously unstable (it's a saddle-point search, not a simple minimization). (Ch. 21)
GitOps — the practice of making a git repository of declarative manifests the single source of truth for a system's desired state, with a controller (ArgoCD, Flux) continuously reconciling the live cluster to match that repo — so a deployment is a git commit/merge, and git revert is your rollback mechanism, fully auditable in history. (Ch. 8)
Gradient boosting — see Boosting, above.
Gradient clipping — capping the magnitude of the gradient vector before the optimizer step, either by clipping each element to a fixed range (rare) or, far more commonly, by rescaling the whole gradient vector so its L2 norm never exceeds a threshold $c$: if $\lVert g\rVert_2 > c$, replace $g$ with $g \cdot c / \lVert g\rVert_2$. This prevents a single unlucky batch (or the compounding multiplicative effect of many layers, in a recurrent network or a very deep one) from producing an exploding gradient that destroys the model's weights in one step. Worked example: $g=[3,4]$ has norm $\sqrt{3^2+4^2}=5$; clipping to $c=1$ rescales it to $[3,4]\times(1/5)=[0.6,0.8]$, same direction, unit norm. Standard practice for training Transformers and RNNs; rarely needed for well-behaved CNNs with batch normalization. (Ch. 18)
Gradient descent — the core optimization loop of nearly all of machine learning: repeatedly step the parameters $\theta$ a small amount in the direction that most decreases the loss $L$, $$\theta \leftarrow \theta - \eta \nabla_\theta L,$$ where $\eta$ (the learning rate) controls the step size and $\nabla_\theta L$ is the gradient (vector of partial derivatives) of the loss with respect to every parameter. Worked example: minimize the toy loss $L(\theta) = (\theta - 3)^2$ starting at $\theta_0 = 0$ with $\eta = 0.1$. The gradient is $\nabla L = 2(\theta - 3)$; at $\theta_0=0$, $\nabla L = -6$, so $\theta_1 = 0 - 0.1\times(-6) = 0.6$. At $\theta_1=0.6$, $\nabla L = 2(0.6-3)=-4.8$, so $\theta_2 = 0.6 - 0.1\times(-4.8) = 1.08$. Each step overshoots less and less, converging geometrically toward the true minimum $\theta^* = 3$. In practice, plain gradient descent is replaced by an optimizer — see the entry below — that adapts the step per parameter. (Ch. 13, Appendix D)
Grafana — the open-source dashboarding front-end typically paired with Prometheus (query via PromQL, below) or any other metrics/logs/traces backend, used to build the RED/USE dashboards described in observability (below). (Ch. 10)
Helm — the package manager for Kubernetes: a "chart" bundles a set of templated manifests plus a values.yaml of overridable parameters, so helm install myapp ./chart -f prod-values.yaml deploys a whole parameterized application (Deployment, Service, Ingress, ConfigMap…) in one command, and helm upgrade/helm rollback version the release as a unit. (Ch. 6)
HPA (Horizontal Pod Autoscaler) — see Autoscaling, above; specifically the controller that scales a Deployment/StatefulSet's replica count up or down based on an observed metric (commonly CPU utilization, but any custom metric via the metrics API), keeping it near a target value you configure. (Ch. 6)
Hyperparameter — any configuration value chosen before training that is not learned from data by gradient descent itself (learning rate, batch size, number of trees, regularization strength, network depth). Tuned via grid search (exhaustive), random search (usually more efficient per trial than grid for the same budget), or Bayesian optimization (above, most sample-efficient). (Ch. 13, Ch. 16)

I–O¶

IAM (Identity and Access Management) — AWS's system for defining who (users, roles, services) can do what (actions) on which resources (ARNs), expressed as JSON policy documents attached to principals. Nearly every AWS security incident traces back to an overly broad IAM policy ("Action": "*", "Resource": "*") rather than a broken encryption algorithm. (Ch. 5)
IaC (Infrastructure as Code) — describing infrastructure (networks, compute, IAM, DNS) in a declarative, version-controlled language (Terraform's HCL, CloudFormation, Pulumi) instead of clicking through a console, so infrastructure changes go through the same review/diff/history discipline as application code. Terraform's plan step is IaC's core safety net: it shows exactly what will change before anything is touched. (Ch. 7)
Idempotency — an operation that produces the same end state no matter how many times it's applied; PUT /users/42 {"name": "Ana"} is idempotent (repeating it changes nothing further), POST /users {"name": "Ana"} typically is not (each call creates a new user). Terraform apply and Kubernetes apply are both designed to be idempotent — safe to re-run after a partial failure or a retry. (Ch. 2, Ch. 7)
Image / layer (Docker) — an image is an ordered stack of read-only filesystem layers plus metadata (entrypoint, exposed ports, env); layers are content-addressed and cached, so two images sharing a base layer (e.g. the same python:3.12-slim) don't duplicate that layer's storage or transfer time. (Ch. 4)
Immutable infrastructure — the practice of never patching a running server/container in place; instead, build a new image/AMI with the change baked in and replace the old instance wholesale. This eliminates configuration drift (two "identical" servers that silently diverged after years of manual patches) and makes rollback trivial (redeploy the previous image). (Ch. 4, Ch. 7)
Inference (batch vs. online) — batch inference scores a large dataset all at once on a schedule (nightly churn scores for every customer); online inference serves one prediction per incoming request with a latency budget (a fraud check on a live payment). The two impose very different infrastructure: batch optimizes for throughput (a big Spark/Ray job), online optimizes for p99 latency (a warm, autoscaled model server). (Ch. 16)
Ingress (Kubernetes) — the object that routes external HTTP(S) traffic to internal Services by host and/or path, and typically also terminates TLS; it requires an Ingress controller (nginx-ingress, Traefik, AWS Load Balancer Controller) actually running in the cluster to do anything — the Ingress object alone is just a routing rule. (Ch. 6)
Init container / Sidecar container (Kubernetes) — two patterns for extra containers sharing a pod (above) with the main "app" container. An init container runs to completion before any app container starts (e.g. run a migration, wait for a dependency to be reachable) — if it fails, the pod doesn't start at all. A sidecar container runs alongside the app container for the pod's whole lifetime, sharing its network namespace and volumes (a log shipper tailing the app's log file, a service-mesh proxy intercepting all its traffic, a config-reloader watching a mounted ConfigMap) — the pattern that lets you bolt on cross-cutting infrastructure concerns without modifying the app container's image at all. (Ch. 6)
IoU (Intersection over Union) — the overlap metric for two bounding boxes (or segmentation masks): the area of their intersection divided by the area of their union, $$\text{IoU} = \frac{|A \cap B|}{|A \cup B|}.$$ Worked example: box $A = [0,0,10,10]$ and box $B = [5,5,15,15]$ (both axis-aligned, coordinates in pixels). Their intersection is the rectangle $[5,5,10,10]$, area $= 5\times5=25$. Their union is $|A|+|B|-|A\cap B| = 100+100-25=175$. So $\text{IoU} = 25/175 \approx 0.143$ — well below the typical $0.5$ threshold used to call a detection a "match," so this pair would be scored as a miss in most detection benchmarks. (Ch. 20)
Job / CronJob (Kubernetes) — a Job runs a pod to completion (retrying on failure up to a limit) for one-off/batch work, rather than keeping it running forever like a Deployment; a CronJob wraps a Job with a cron schedule (0 2 * * * = every day at 2am) to run it repeatedly. (Ch. 6)
JWT (JSON Web Token) — a compact, self-contained, digitally signed token made of three base64url-encoded, dot-separated parts (header.payload.signature): the header names the signing algorithm, the payload carries claims (sub = subject, exp = expiry, custom claims), and the signature (HMAC or RSA/ECDSA) lets any holder of the public key/shared secret verify the token wasn't tampered with without calling back to the issuer. This statelessness is exactly why OIDC (below) uses JWTs for identity tokens and why an API gateway can authorize a request by verifying a signature locally instead of a database round trip per request — at the cost that a JWT cannot be individually revoked before its exp without extra infrastructure (a deny-list), so short expiries plus refresh tokens are the standard mitigation. (Ch. 9)
Kafka — a distributed, partitioned, append-only log used as the backbone of streaming data pipelines: producers append events to a topic's partitions, consumers read at their own pace tracking an offset, and the log retains events for a configured window regardless of whether they've been consumed — which decouples producers from consumers entirely and lets multiple independent consumer groups replay the same stream. (Ch. 12)
Kernel — in machine learning, a kernel is a function $k(x,x')$ that computes the inner product of two points as if they had been mapped into a much higher- (even infinite-) dimensional feature space, without ever computing that mapping explicitly (the "kernel trick"); this is what lets an SVM draw a nonlinear decision boundary in the original space while only ever solving a linear problem in feature space. In the operating system sense, the kernel is the privileged core of the OS that manages processes, memory, and hardware — cgroups and namespaces (containers' foundation) are kernel features. (Ch. 1, Ch. 17)
KL divergence — a (non-symmetric) measure of how different a probability distribution $q$ is from a reference distribution $p$: $$D_{\mathrm{KL}}(p\,\|\,q) = \sum_k p_k \log\frac{p_k}{q_k}.$$ It is zero iff $p=q$ everywhere, and it is not a true distance (in general $D_{\mathrm{KL}}(p\|q) \ne D_{\mathrm{KL}}(q\|p)$). Worked example: $p=[0.5,0.5]$, $q=[0.9,0.1]$: $D_{\mathrm{KL}}(p\|q) = 0.5\ln(0.5/0.9) + 0.5\ln(0.5/0.1) = 0.5\times(-0.588) + 0.5\times(1.609) = -0.294 + 0.805 = 0.511$ nats. This is exactly the quantity that appears in the VAE's ELBO (above) and in monitoring data drift (above) against a training-time reference distribution. (Ch. 16, Ch. 21, Appendix D)
Kubeconfig — the YAML file (~/.kube/config by default) holding cluster API endpoints, certificate authority data, and user credentials that kubectl reads to know which cluster to talk to and as whom; it can define multiple contexts (cluster+user+namespace triples) and switch between them with kubectl config use-context. (Ch. 6)
Kubernetes — the declarative container orchestration platform: you describe the desired state (how many replicas, which image, which resources) as objects, and a set of control loops continuously reconcile the actual cluster state toward it — self-healing (a crashed pod is recreated), rolling updates, service discovery, and autoscaling all fall out of this one reconciliation pattern applied to different object kinds. (Ch. 6)
Kustomize — a template-free way to customize raw Kubernetes YAML: instead of parameterizing manifests with a templating language (Helm's approach), you write a base set of plain manifests plus small "overlay" patches per environment (dev/staging/prod), and Kustomize merges base + overlay at apply time. It ships built into kubectl (kubectl apply -k), which makes it a lighter-weight alternative to Helm when you don't need Helm's packaging/versioning/release-rollback machinery, only environment-specific tweaks (replica count, image tag, resource limits) on top of a shared base. (Ch. 6)
Lakehouse — see Data warehouse vs. data lake vs. lakehouse, above.
Lambda (AWS) — a serverless compute service that runs your function code in response to an event (HTTP request, queue message, schedule) with no server to provision, billed per invocation and per GB-second of execution time; the trade-off versus a long-running server is per-invocation cold-start latency and a maximum execution duration. (Ch. 5)
Latent space — the (typically low-dimensional, continuous) space a generative model's internal representation lives in (a VAE's $z$, a diffusion model's noise seed, a GAN's input noise); moving smoothly through latent space and decoding at each point produces smoothly morphing outputs, which is both a debugging tool (visualize what the model has learned) and a creative one (latent-space interpolation, arithmetic). (Ch. 21)
Learning rate schedule (warmup, cosine annealing) — the policy that varies the learning rate $\eta$ (above, under Gradient descent) over the course of training rather than holding it fixed. Warmup linearly ramps $\eta$ up from near zero over the first few hundred/thousand steps, avoiding a large, destabilizing update while the model's weights are still near their random initialization (critical for Transformers, above). Cosine annealing then decays $\eta$ smoothly to (near) zero following $\eta_t = \eta_{\min} + \tfrac12(\eta_{\max}-\eta_{\min})\big(1+\cos(\pi t/T)\big)$, where $t$ is the current step and $T$ the total number of steps — the cosine shape spends more steps near the peak learning rate (fast early progress) and tapers gently near the end (fine-grained convergence) compared to a linear decay. Worked example: $\eta_{\max}=1\text{e-}3$, $\eta_{\min}=0$, at the halfway point $t/T=0.5$: $\cos(\pi\times0.5)=\cos(90°)=0$, so $\eta = 0 + 0.5\times(1\text{e-}3)\times(1+0) = 5\text{e-}4$ — exactly half the peak rate at the midpoint, as expected from the cosine shape. (Ch. 18, Ch. 19)
Least privilege — the security principle of granting a principal (user, role, service account) only the exact permissions it needs to do its job, nothing more — the single most effective, and most often skipped, mitigation against the blast radius of any credential leak or compromised workload. (Ch. 5, Ch. 9)
Little's law — see the worked example in "How to use this glossary," above: $L=\lambda W$, the steady-state relationship between concurrency, arrival rate, and time-in-system, central to sizing worker pools and connection limits. (Ch. 2)
LoRA (Low-Rank Adaptation) — a parameter-efficient fine-tuning technique that freezes the pretrained weight matrix $W$ and learns only a low-rank update $\Delta W = BA$ (with $B \in \mathbb R^{d\times r}$, $A \in \mathbb R^{r\times k}$, rank $r \ll \min(d,k)$) added at inference time; this cuts trainable parameters and optimizer memory by orders of magnitude versus full fine-tuning, at a small quality cost, and lets you swap task-specific LoRA adapters in and out of one frozen base model cheaply. (Ch. 19)
Loss function — the single scalar a training run is trying to minimize; it operationalizes "wrong" into a differentiable number so gradient descent has something to descend (mean squared error for regression, cross-entropy for classification, the GAN minimax objective, the RL policy-gradient objective — every learning algorithm in this book reduces to "define a loss, compute its gradient, step the parameters"). (Ch. 13)
mAP (mean Average Precision) — the standard object-detection benchmark score: for each class, plot precision against recall as the confidence threshold sweeps down, compute the area under that curve (Average Precision), then average AP across classes. "AP@0.5" means a predicted box only counts as a match if its IoU (above) with a ground-truth box exceeds 0.5. (Ch. 20)
MDP (Markov Decision Process) — the mathematical formalism underlying reinforcement learning: a tuple $(S, A, P, R, \gamma)$ of states, actions, transition probabilities $P(s'\mid s,a)$, a reward function $R(s,a)$, and a discount factor $\gamma$ (above); "Markov" means the next state depends only on the current state and action, not on the full history — which is what lets the Bellman equation (above) be written recursively at all. (Ch. 22)
MLflow — an open-source platform with four components used across this book: Tracking (log parameters/metrics/artifacts per run), Projects (packaged, reproducible runs), Models (a standard packaging format any serving tool can load), and the Model Registry (stage a specific run's model as Staging/Production, with lineage back to the exact run, code version, and data version that produced it). (Ch. 15)
Multi-stage build (Docker) — a Dockerfile with several FROM stages, where an early stage compiles/ builds (with all the heavy build tools) and a final stage COPY --from=<earlier-stage> only the compiled artifact into a minimal runtime base image — "build fat, ship lean," which shrinks the final image and its attack surface. (Ch. 4)
Namespace — in Kubernetes, a virtual sub-cluster used to scope names, apply resource quotas, and isolate RBAC (below) between teams/environments sharing one physical cluster. In Linux, a namespace is the kernel primitive that gives a process its own isolated view of some global resource (PIDs, network interfaces, mounts) — the actual mechanism containers (above) are built from. (Ch. 1, Ch. 6)
NAT Gateway — a managed AWS resource that lets instances in a private subnet initiate outbound connections to the internet (to pull packages, call an external API) while remaining unreachable from the internet inbound; billed per hour plus per GB processed, which makes it a common line item worth watching on an AWS bill. (Ch. 5)
NetworkPolicy (Kubernetes) — a pod-level firewall: by default every pod can reach every other pod in the cluster, and a NetworkPolicy is an explicit allow-list (by pod/namespace label selectors and ports) that, once any policy selects a pod, switches that pod to default-deny for anything not explicitly allowed. Requires a CNI (above) that implements NetworkPolicy enforcement. (Ch. 6)
Node affinity / anti-affinity / PodDisruptionBudget (Kubernetes) — three controls over where and how many pods may be evicted at once. Node affinity attracts a pod toward nodes matching a label (e.g. gpu=true), a softer, more expressive cousin of taints/tolerations (below) that expresses a preference from the pod's side rather than a repulsion from the node's side. Pod anti-affinity does the opposite between pods — e.g. "never schedule two replicas of this Deployment on the same node," so a single node failure can't take out every replica at once. A PodDisruptionBudget (PDB) caps how many pods of a set may be voluntarily evicted simultaneously (during a node drain for cluster upgrade, say) — e.g. minAvailable: 2 on a 3-replica Deployment guarantees the cluster autoscaler/upgrade process never drains the third replica until one of the first two is back, protecting availability during routine maintenance the way anti-affinity protects it during a hardware failure. (Ch. 6)
Normal equations — the closed-form solution that directly minimizes ordinary least-squares loss without any iterative gradient descent: $$\theta = (X^\top X)^{-1} X^\top y.$$ Worked example (simple linear regression, one feature plus intercept): three points $x=[1,2,3]$, $y=[1,3,3]$. Means: $\bar x = 2$, $\bar y = 2.333$. The slope is $b_1 = \dfrac{\sum (x_i-\bar x)(y_i-\bar y)}{\sum (x_i - \bar x)^2}$; the deviations are $x-\bar x = [-1,0,1]$ and $y-\bar y = [-1.333, 0.667, 0.667]$, giving numerator $(-1)(-1.333)+0\times0.667+1\times0.667 = 1.333+0+0.667 = 2.0$ and denominator $1+0+1=2.0$, so $b_1 = 2.0/2.0 = 1.0$. The intercept is $b_0 = \bar y - b_1\bar x = 2.333 - 1.0\times2 = 0.333$. The fitted line is $\hat y = 0.333 + 1.0\,x$ — check: at $x=1$, $\hat y = 1.333$, close to the observed $y=1$; the residual variance is what the model leaves unexplained. This closed form only exists for linear regression under squared loss; every other model in this book (logistic regression onward) needs iterative gradient descent instead. (Ch. 17)
OIDC (OpenID Connect) — an identity layer on top of OAuth2 that adds a signed, verifiable identity token (a JWT) proving who authenticated, not just that an access token was issued; it's what Keycloak/oauth2-proxy use for human login (Ch. 9) and what GitHub Actions/GitLab CI use for keyless, short-lived AWS credentials via AssumeRoleWithWebIdentity (Ch. 8) — no long-lived AWS access key ever has to be stored in CI secrets. (Ch. 8, Ch. 9)
OOMKilled — the Kubernetes/Docker status meaning the container's process was killed by the Linux kernel's OOM (Out-Of-Memory) mechanism for exceeding its cgroup memory limit — not a crash in the application's own code, and raising the memory limit (or fixing an actual leak) is the fix, not restarting the pod. (Ch. 1, Ch. 6)
Optimizer (SGD, Momentum, Adam, RMSProp) — the algorithm that turns the raw gradient into an actual parameter update, on top of plain gradient descent's $\theta \leftarrow \theta - \eta\nabla L$. SGD with momentum accumulates a running average of past gradients (a "velocity") so the update keeps moving through small local bumps in the loss surface instead of stopping at every one. RMSProp and Adam additionally keep a per-parameter running average of squared gradients and divide the step by its square root, effectively giving each parameter its own adaptive learning rate — parameters with consistently large gradients get smaller effective steps, and vice versa. Adam (Momentum + RMSProp combined, with bias-correction terms) is the default choice for most deep-learning training in this book. (Ch. 18, Appendix D)

P–Z¶

Parquet — a columnar, typed, compressed binary file format: because values of the same column are stored contiguously, a query that only needs 3 of 50 columns reads a fraction of the bytes a row-oriented format (CSV, JSON) would require, and per-column encoding (dictionary, run-length) compresses far better than mixed-type rows. The default choice for any dataset that will be queried by column. (Ch. 11, Ch. 12)
PCA (Principal Component Analysis) — a dimensionality-reduction technique that finds the orthogonal directions (principal components) of maximum variance in the data, obtained as the eigenvectors of the data's covariance matrix, ordered by their eigenvalues (the variance each direction explains). Worked example: a 2-feature dataset with covariance matrix $\Sigma = \begin{pmatrix}2 & 1\\1 & 2\end{pmatrix}$ has eigenvalues found from $\det(\Sigma - \lambda I)=0 \Rightarrow (2-\lambda)^2 - 1 = 0 \Rightarrow \lambda = 3 \text{ or } 1$. Projecting onto just the first principal component (eigenvalue 3) retains $3/(3+1) = 75\%$ of the total variance in a single dimension instead of two — the practical payoff of PCA: fewer dimensions for a controlled loss of information. It is computed in practice via the SVD (below) of the (centered) data matrix rather than by explicitly forming the covariance matrix, for numerical stability. (Ch. 17, Appendix D)
Perplexity — the standard language-model quality metric, defined as the exponential of the average cross-entropy (above) per token: $\text{PPL} = \exp(H)$ where $H$ is the mean cross-entropy in nats. Intuitively, perplexity is "the effective number of equally likely choices the model is confused among" at each position. Worked example: a model with average cross-entropy $H = 1.5$ nats per token has perplexity $\exp(1.5) \approx 4.48$ — as confused, on average, as if it were guessing uniformly among about 4–5 equally likely next tokens, even though the real vocabulary might have 50,000 entries. Lower is better; a perplexity near the vocabulary size means the model has learned almost nothing. (Ch. 19)
Pod (Kubernetes) — the smallest deployable unit: one or more containers that always share the same network namespace (one IP) and can share volumes, scheduled together onto the same node and living/dying together. A Deployment doesn't manage containers directly — it manages Pods, which in turn hold containers. (Ch. 6)
Policy — in IAM, a JSON document listing allowed/denied actions on resources, attached to a user, role, or group. In reinforcement learning, a policy $\pi(a\mid s)$ is the (possibly stochastic) mapping from states to a distribution over actions that the agent follows — the object that RL training is actually trying to improve. (Ch. 5, Ch. 22)
Positional encoding — since self-attention (above) treats its input as an unordered set of tokens (permuting the input permutes the output identically), a Transformer must inject explicit position information; the original design adds a fixed sinusoidal vector per position (varying frequency across dimensions) directly to each token's embedding before the first attention layer, so nearby positions get similar codes and the model can learn to attend by relative offset. (Ch. 19)
PPO (Proximal Policy Optimization) — the most widely used policy-gradient RL algorithm in practice, which maximizes expected advantage (above) while clipping the probability ratio between the new and old policy to a band $[1-\varepsilon, 1+\varepsilon]$ (commonly $\varepsilon=0.2$) so a single update can't move the policy too far from the one that collected the data — trading a bit of per-step optimality for training stability. Worked example: if the raw ratio $r(\theta) = \pi_{\text{new}}(a\mid s)/\pi_{\text{old}}(a\mid s) = 1.5$ (the new policy would have made this action 50% more likely) and $\varepsilon=0.2$, PPO clips $r(\theta)$ to $\min(1.5, 1.2) = 1.2$ before multiplying by the advantage, capping how aggressively any single good (or bad) action can push the policy in one update. (Ch. 22)
Probes (readiness / liveness / startup) — Kubernetes health checks on a container: a startup probe gates when the other two probes even begin (for slow-booting apps); a liveness probe failing gets the container killed and restarted (it's deemed permanently stuck); a readiness probe failing only removes the pod from Service load-balancing (it's temporarily not ready, e.g. warming a cache) but does not restart it. Conflating liveness with readiness is a common cause of restart-crash-loops on pods that were merely slow, not broken. (Ch. 6)
PromQL (Prometheus Query Language) — the query language for Prometheus's time-series metrics; e.g. rate(http_requests_total[5m]) computes the per-second average request rate over a trailing 5-minute window from a monotonically increasing counter — rate() on a counter, not a raw difference, is the idiomatic pattern because it correctly handles counter resets (process restarts). (Ch. 10)
PSI (Population Stability Index) — a data-drift metric that buckets a feature's values (deciles, typically) and compares the proportion of examples in each bucket between a reference (training) period and a current (production) period: $$\text{PSI} = \sum_i (p_i - q_i)\ln\frac{p_i}{q_i},$$ the same functional form as a symmetrized KL divergence (above). A widely used rule of thumb: PSI $< 0.1$ means no significant shift, $0.1$–$0.25$ means moderate shift worth investigating, $> 0.25$ means the feature distribution has changed enough to warrant retraining or a model audit. (Ch. 16)
PV / PVC / StorageClass (Kubernetes) — a PersistentVolume (PV) is a piece of real storage (an AWS EBS volume, say) provisioned in the cluster; a PersistentVolumeClaim (PVC) is a pod's request for storage matching some size/access-mode criteria, bound to a matching PV; a StorageClass parameterizes how PVs get dynamically provisioned on demand (which backend, which disk type) so you rarely create PVs by hand. (Ch. 6)
Q-learning — an off-policy, model-free RL algorithm that learns the optimal action-value function $Q^*(s,a)$ directly via the Bellman optimality update $Q(s,a) \leftarrow Q(s,a) + \alpha\big[r + \gamma\max_{a'}Q(s',a') - Q(s,a)\big]$, where $\alpha$ is a learning rate; "off-policy" means it can learn the optimal policy's values while acting according to a different, more exploratory policy (e.g. $\varepsilon$-greedy) — a key reason it's compatible with replay buffers of old experience. (Ch. 22)
Quantization — reducing the numerical precision used to store/compute a model's weights and/or activations (e.g. 32-bit floats down to 8-bit integers, or the 4-bit schemes common for serving large LLMs), trading a small, usually acceptable accuracy loss for large reductions in memory footprint and inference latency/cost. (Ch. 19)
RAG (Retrieval-Augmented Generation) — grounding an LLM's answer in retrieved documents rather than relying purely on what it memorized during pretraining: embed the knowledge base into vectors, embed the query the same way, retrieve the nearest documents (cosine similarity search), and stuff them into the prompt as context before generation — the standard mitigation for hallucination and for giving an LLM knowledge of private/recent data it was never trained on. (Ch. 19)
Rate limiting (token bucket) — protecting a service from being overwhelmed by capping how many requests a client (or the service as a whole) may make per unit time. The token bucket algorithm is the standard implementation: a bucket holds up to $B$ tokens (the "burst" capacity), refills at a steady rate $r$ tokens/second, and each incoming request consumes one token — if the bucket is empty, the request is rejected (HTTP 429) or queued. This allows short bursts up to $B$ requests instantly while enforcing a long-run average rate of $r$/s, unlike a naive fixed-window counter which allows up to $2B$ requests in a short window straddling two window boundaries. Worked example: $B=100$, $r=10$/s; a client that has been idle accumulates up to 100 tokens and can fire 100 requests instantly, but is then limited to 10/s thereafter until the bucket refills — sized so a legitimate retry burst passes but a sustained scraping script does not. (Ch. 2)
RBAC (Role-Based Access Control) — Kubernetes' native authorization model: a Role (namespaced) or ClusterRole (cluster-wide) lists allowed verbs (get/list/watch/create/delete) on resource kinds, and a RoleBinding/ClusterRoleBinding grants that role to a user, group, or service account. Every pod that calls the Kubernetes API itself (an operator, a CI runner) does so as a service account bound to a role — least privilege (above) applies here exactly as it does in IAM. (Ch. 6)
RDS (Relational Database Service) — AWS's managed relational database (Postgres, MySQL, etc.): AWS handles patching, automated backups/snapshots, and (with Multi-AZ) synchronous standby failover, in exchange for less low-level control than self-hosting the database on EC2. (Ch. 5)
RED / USE methods — two complementary dashboarding philosophies: RED (for request-driven services) tracks Rate, Errors, Duration per endpoint; USE (for resources: CPU, disk, network) tracks Utilization, Saturation, Errors. A healthy on-call dashboard usually has one RED panel per service and one USE panel per critical resource, not an undifferentiated wall of graphs. (Ch. 10)
Region (AWS) — a fully isolated AWS geography (e.g. eu-west-3 / Paris) containing multiple Availability Zones (above); resources in one region are, by default, invisible to and unaffected by another region — replication across regions is always an explicit, deliberate choice. (Ch. 5)
Regularization (L1 / L2) — a penalty term added to the loss to discourage overly complex models and fight the variance side of the bias–variance trade-off (above). L2 (ridge) adds $\lambda \sum_j \theta_j^2$, shrinking all weights smoothly toward zero without forcing any to exactly zero. L1 (lasso) adds $\lambda\sum_j |\theta_j|$, which — because of the non-differentiable kink at zero — drives some weights to exactly zero, performing implicit feature selection. Larger $\lambda$ means stronger regularization (more bias, less variance). (Ch. 13, Ch. 17)
ReLU / sigmoid / softmax — the standard activation functions. ReLU $f(x)=\max(0,x)$ is the default for hidden layers: cheap, and its constant gradient of 1 for $x>0$ avoids the vanishing-gradient problem that plagued sigmoid-based deep nets. Sigmoid $\sigma(x)=1/(1+e^{-x})$ squashes to $(0,1)$, used for a single-output binary-classification probability. Softmax $\text{softmax}(x)_k = e^{x_k}/\sum_j e^{x_j}$ generalizes sigmoid to $K>2$ classes, turning a vector of raw scores ("logits") into a valid probability distribution — this is exactly what feeds cross-entropy (above) at a classifier's output layer. (Ch. 18)
Residual connection — a shortcut $y = f(x) + x$ that adds a layer (or block)'s input directly to its output, so the block only needs to learn the residual (the difference) rather than the full transformation from scratch; this keeps gradients flowing directly through the +x path during backpropagation, which is what made training networks with 50–150+ layers (ResNet) practical where plain stacks of layers had previously degraded with depth. (Ch. 18, Ch. 20)
ResNet / ViT — ResNet is a deep convolutional network built from residual blocks (above); ViT (Vision Transformer) instead slices an image into fixed-size patches, embeds each patch as a token, and applies a standard Transformer encoder (attention, above) over the patch sequence — trading the CNN's built-in translation-equivariance for the Transformer's greater capacity, at the cost of needing more training data to reach the same accuracy. (Ch. 20)
Reward model / RLHF — a reward model is a learned function (trained on human preference comparisons — "response A is better than response B") that scores a generated output the way a human rater would; RLHF (Reinforcement Learning from Human Feedback) then fine-tunes a language model with an RL algorithm (typically PPO, above) to maximize that learned reward model's score, which is how base LLMs are turned into helpful, instruction-following assistants. (Ch. 19, Ch. 22)
Rolling update — the default Kubernetes Deployment rollout strategy (above): replace old-version pods with new-version pods a few at a time (governed by maxSurge/maxUnavailable), keeping the service continuously available, as opposed to a "recreate" strategy that tears everything down before bringing the new version up. (Ch. 6, Ch. 8)
Route 53 — AWS's managed DNS (above) service, also supporting health-check-based failover routing and weighted/latency-based routing policies across regions/endpoints. (Ch. 5)
S3 (Simple Storage Service) — AWS's object storage: durable, effectively infinitely scalable key-value storage for arbitrary blobs, billed by GB stored, requests, and egress bandwidth; it is the substrate under most data lakes (above), Terraform remote state, and DVC/MLflow artifact storage in this book. (Ch. 5, Ch. 7, Ch. 12)
SAST / DAST — Static Application Security Testing scans source code (or dependency manifests) for known-vulnerable patterns/libraries without running the application; Dynamic Application Security Testing probes a running instance of the application (fuzzing inputs, checking headers) for exploitable behavior — the two catch different classes of issues and are typically both wired into CI/CD's security gate. (Ch. 8, Ch. 9)
Security Group vs. NACL — an AWS Security Group is a stateful firewall attached to an instance/ENI (an allowed inbound connection's return traffic is automatically allowed, no matching outbound rule needed); a Network ACL is a stateless firewall attached to a subnet (return traffic must be explicitly allowed by a separate rule) evaluated in numbered-rule order. Most designs rely on Security Groups for day-to-day rules and NACLs only for coarse subnet-wide deny rules. (Ch. 5)
Service (Kubernetes) — a stable virtual IP address and DNS name (myapp.namespace.svc.cluster.local) that load-balances traffic across the current, ever-changing set of pods matching a label selector — the layer that lets other components address "the app" without ever needing to know an individual pod's (ephemeral) IP. (Ch. 6)
Sharpe ratio (and Sortino, Calmar) — the classic risk-adjusted return metric, $$\text{Sharpe} = \frac{\mathbb E[r] - r_f}{\sigma_r},$$ the mean excess return (over a risk-free rate $r_f$, often approximated as 0 for short horizons) divided by the return's standard deviation. Worked example: a strategy has mean daily return $\mu = 0.001$ (0.1%/day) and daily standard deviation $\sigma = 0.015$ (1.5%/day); the daily Sharpe is $0.001/0.015 = 0.0667$. Annualizing (assuming i.i.d. daily returns and 252 trading days) scales by $\sqrt{252}\approx 15.87$: $\text{Sharpe}_{\text{annual}} \approx 0.0667 \times 15.87 \approx 1.06$. Sortino replaces $\sigma_r$ with downside deviation only (volatility from losing days doesn't hurt a strategy the way volatility from winning days does — Sortino doesn't penalize it). Calmar instead divides annualized return by maximum drawdown, directly answering "how much return per unit of the worst peak-to-trough loss experienced." A high Sharpe reported after testing many variants should always be checked against the Deflated Sharpe Ratio (above). (Ch. 23)
SLI / SLO (/ SLA) — a Service Level Indicator is the raw measured metric (e.g. "fraction of requests under 200ms"); a Service Level Objective is the internal target for that indicator (e.g. "99.5% of requests under 200ms, measured over 28 days"); a Service Level Agreement is the external, often contractual, commitment (typically looser than the internal SLO, leaving margin) with a business consequence (credits, penalties) if missed. The error budget (above) is derived directly from the SLO. (Ch. 10)
SNS / SQS — Simple Notification Service is AWS's pub/sub fan-out (one message delivered to many subscribers — email, Lambda, SQS queues); Simple Queue Service is AWS's point-to-point queue (one message consumed by one worker among a pool), used to decouple producers from consumers and absorb load spikes. (Ch. 5)
Stationarity (and the ADF test) — a time series is (weakly) stationary if its mean, variance, and autocorrelation structure don't change over time; most classical forecasting models (ARIMA, above) and many statistical tests assume it. The Augmented Dickey-Fuller (ADF) test checks the null hypothesis "the series has a unit root" (is non-stationary); rejecting the null (typically at $p<0.05$) supports treating the series as stationary, while failing to reject usually means you should difference it (subtract each value from the previous one, the "I" in ARIMA) before modeling. (Ch. 23)
State (Terraform) — the JSON file (terraform.tfstate) mapping every resource block in your .tf code to the real-world resource ID it created, which is how terraform plan computes a diff between code and reality. It must live in a locked, remote backend (S3 + DynamoDB lock table, or Terraform Cloud) shared by the whole team — a local, unlocked state file is both a single point of failure and a race condition waiting to corrupt itself under concurrent applys. (Ch. 7)
StatefulSet (Kubernetes) — like a Deployment, but for workloads that need a stable identity: each replica gets a fixed ordinal name (db-0, db-1, db-2) and its own PersistentVolumeClaim that follows it across rescheduling, and scale-up/down happens strictly in order — the right controller for databases and other clustered stateful software, where Deployment's interchangeable, identity-less pods would be actively wrong. (Ch. 6)
STS / AssumeRole — AWS Security Token Service issues short-lived, temporary credentials when a principal "assumes" an IAM role, which is the mechanism behind cross-account access, EC2/EKS instance roles, and OIDC-based CI credentials (above) — none of which require a long-lived access key ever to exist. (Ch. 5)
SVD (Singular Value Decomposition) — the matrix factorization $X = U\Sigma V^\top$ that underlies PCA (above): $U$'s columns are the left singular vectors, $V$'s columns (the principal-component directions) are the right singular vectors, and $\Sigma$'s diagonal holds the singular values, whose squares are proportional to the variance explained by each component — computing PCA via SVD directly on the (centered) data matrix is more numerically stable than explicitly forming and eigen-decomposing the covariance matrix. Full derivation in Appendix D. (Ch. 17)
Taints / tolerations (Kubernetes) — a taint on a node repels pods from scheduling there unless the pod carries a matching toleration — the mechanism behind dedicating nodes to a specific workload (GPU nodes tainted so only GPU-requesting pods land there) without needing an allow-list on every other pod in the cluster. (Ch. 6)
Terraform module / workspace — a module is a reusable, parameterized bundle of .tf resource blocks (inputs as variable, outputs as output) called from elsewhere with module "name" { source = ...; ... }, the standard way to avoid copy-pasting the same VPC/EKS-cluster/RDS boilerplate across environments. A workspace is a named, isolated instance of Terraform state (above) for the same configuration — terraform workspace new staging gives you a separate state file so dev/staging/prod don't collide, without duplicating the .tf code itself. Workspaces are a lighter-weight alternative to fully separate state backends per environment; many teams prefer separate backends (or separate root modules) for environments with meaningfully different blast radius, reserving workspaces for short-lived, throwaway variants (a per-developer sandbox, a per-PR preview environment). (Ch. 7)
TLS / X.509 / CA — Transport Layer Security encrypts and authenticates a connection; an X.509 certificate binds a public key to an identity (a domain name) and is itself signed by a Certificate Authority, forming a chain of trust up to a root CA that clients trust by default; mTLS (mutual TLS, below under Zero trust) additionally requires the client to present a certificate too. (Ch. 2, Ch. 9)
Tokenization — splitting raw text into the discrete units (tokens) a language model actually consumes; modern LLMs use subword tokenization (BPE, above, or similar) rather than whole words, which bounds vocabulary size while still being able to represent any string, at the cost of common words costing 1 token and rare/foreign words costing several. (Ch. 19)
Transfer learning — reusing a model already trained on a large, general dataset (a "pretrained" model) as the starting point for a new, usually smaller, more specific task, instead of training from randomly initialized weights. The pretrained model's early/middle layers have already learned broadly useful representations (edges and textures for vision, syntax and world knowledge for language), so fine-tuning (above) only needs to adapt the last layers (or a low-rank update, as in LoRA, below) to the new task's specifics — dramatically reducing both the labeled data and the compute a new task requires versus training from scratch. Nearly every practical deep-learning system in this book (a fine-tuned LLM, a ResNet backbone reused for a custom detector) is an instance of transfer learning. (Ch. 18, Ch. 19, Ch. 20)
Transformer — the attention-based (above) sequence architecture that replaced recurrence (RNNs/LSTMs) as the default for text, and increasingly for images (ViT, above) and other modalities: stacked blocks of self-attention (mix information across positions) plus a position-wise feed-forward network (transform each position independently), with residual connections (above) and layer normalization around each sub-layer. Its key practical advantage over RNNs is that all positions can be processed in parallel during training (no sequential recurrence to unroll). (Ch. 19)
VAE (Variational Autoencoder) — a generative autoencoder (above) whose encoder outputs the parameters of a distribution (mean and variance of a Gaussian) over the latent code rather than a single point, trained by maximizing the ELBO (above); because nearby points in its latent space decode to similar, plausible outputs, you can sample new data by sampling latent points from the prior and decoding. (Ch. 21)
Vanishing / exploding gradient — the failure mode where backpropagation (above) through many layers multiplies many partial derivatives together, and if those derivatives are consistently $<1$ in magnitude the product shrinks toward zero (vanishing — early layers barely update, training stalls), while if they are consistently $>1$ it grows without bound (exploding — weights blow up to NaN). Worked example: the sigmoid activation's derivative $\sigma'(x)=\sigma(x)(1-\sigma(x))$ has a maximum value of exactly $0.25$ (at $x=0$); through just 10 sigmoid layers, the best-case gradient magnitude is multiplied by $0.25^{10} \approx 9.5\times10^{-7}$ — effectively zero, which is precisely why deep sigmoid networks were nearly untrainable before ReLU (above), residual connections (above), and batch normalization (above) became standard: ReLU's derivative is exactly 1 for any positive input (no shrinkage), residual connections give the gradient a +x path that bypasses the multiplicative chain entirely, and gradient clipping (above) caps the exploding side directly. (Ch. 18, Appendix D)
Vectorization — expressing a computation as array-level operations (NumPy/PyTorch ufuncs, matrix multiplies) executed by compiled, often SIMD- or GPU-parallelized code, instead of an explicit Python for loop over elements; the same logical computation can run 10–1000× faster purely from avoiding per-element Python interpreter overhead, with no change in what is mathematically computed. (Ch. 11)
VPC (Virtual Private Cloud) — a private, software-defined network within an AWS Region: you define its CIDR range (above), subnets (public — has a route to an internet gateway; private — doesn't), route tables, and gateways; almost everything else in AWS (EC2, RDS, EKS nodes) is placed inside a VPC's subnets. (Ch. 5)
Walk-forward validation / backtesting — the time-series-correct alternative to shuffled cross- validation: train on data up to time $t$, evaluate on the next window after $t$, then slide the whole window forward and repeat — never evaluating on data that precedes the training window chronologically. A backtest is the broader practice of simulating a trading (or any sequential-decision) strategy over historical data as if it had been run live, and is only trustworthy when it respects this same no-peeking-into-the-future discipline (see also CPCV, above). (Ch. 23)
Watermark (stream processing) — a streaming engine's declared bound on "how late can an event still arrive and be counted," e.g. "no event older than 10 minutes behind the current processing time will be included in a window's aggregate"; it's the mechanism that lets a streaming aggregation ever close a time window and emit a final result, at the cost of silently dropping (or routing to a "late data" side-output) anything arriving after the watermark has passed. (Ch. 12)
Weight decay — an optimizer-level implementation of L2 regularization (above) that shrinks every weight by a small multiplicative factor at each step, independently of the gradient: $\theta \leftarrow (1-\lambda\eta)\theta - \eta\nabla_\theta L$. In plain SGD this is mathematically identical to adding $\lambda\lVert\theta\rVert_2^2$ to the loss, but for adaptive optimizers like Adam (above) the two are not equivalent — Adam's per-parameter adaptive step size interacts badly with an L2 penalty folded into the gradient, which is why AdamW (Adam with decoupled weight decay, applying the shrinkage directly to the weights outside the adaptive-gradient computation) is the standard optimizer for training Transformers rather than plain Adam with an L2 term. (Ch. 18, Ch. 19)
Well-Architected Framework (AWS) — AWS's five-pillar design checklist for reviewing an architecture: Operational Excellence, Security, Reliability, Performance Efficiency, and Cost Optimization (a sixth pillar, Sustainability, was added later); used less as a rulebook and more as a structured set of questions to ask about any nontrivial design before it ships. (Ch. 5)
Zero-shot / few-shot / in-context learning — three ways to get an LLM to perform a task without any gradient update to its weights, purely through the prompt (contrast with fine-tuning, above, which does update weights). Zero-shot gives only an instruction ("Classify this review as positive or negative"). Few-shot additionally includes a handful of worked examples directly in the prompt before the real query — the model conditions its next-token predictions on those examples' pattern without any training step, purely by the Transformer's attention (above) over the prompt's tokens; this ability to pick up a task's pattern from a few in-prompt examples alone is called in-context learning, and it emerges as a capability mainly in models above a certain scale rather than being explicitly programmed. Few-shot prompting typically improves accuracy over zero-shot at the direct cost of a longer prompt (more tokens billed and a larger share of the context window consumed before the actual query even appears). (Ch. 19)
Zero trust / mTLS — a security model that assumes no implicit trust from network location alone (being "inside the VPC" grants nothing by itself); every request is authenticated and authorized on its own merits, commonly via mutual TLS (mTLS) — both sides of a connection present and verify a certificate — a pattern that service meshes (Istio, Linkerd) automate transparently between pods. (Ch. 9)

Cross-cutting maps¶

Two relationships recur so often across chapters that they deserve a diagram of their own rather than being scattered across a dozen separate bullets above.

The training loop, which every entry from Loss function to Optimizer to Backpropagation is a piece of:

flowchart LR
    A["Data batch"] --> B["Forward pass<br/>(model prediction)"]
    B --> C["Loss function<br/>(cross-entropy, MSE...)"]
    C --> D["Backpropagation<br/>(∂L/∂θ via chain rule)"]
    D --> E["Optimizer step<br/>(SGD / Adam)"]
    E --> F["Updated weights θ"]
    F -.next batch.-> A

The identity/access triangle, which every entry from IAM to OIDC to RBAC to ESO is a piece of:

flowchart TD
    Human["Human user"] -->|"OIDC login (Keycloak)"| Proxy["oauth2-proxy / Ingress"]
    CI["CI/CD pipeline"] -->|"OIDC token exchange"| STS["AWS STS AssumeRoleWithWebIdentity"]
    STS --> Temp["Short-lived AWS credentials"]
    Pod["Kubernetes pod"] -->|"Service Account token"| RBAC["Kubernetes RBAC"]
    Pod -->|"synced by"| ESO["External Secrets Operator"]
    ESO -->|"reads from"| SM["AWS Secrets Manager"]
    Temp --> IAMPolicy["IAM Policy (least privilege)"]
    RBAC --> IAMPolicy

Both diagrams point at the same underlying lesson: every layer of the stack re-implements the same two patterns — a reconciliation loop (desired state vs. actual state, whether that's Kubernetes' controllers or gradient descent driving a loss to zero) and a chain-of-trust for identity (whether that's an X.509 certificate chain or an IAM AssumeRole chain). Recognizing the pattern is more valuable than memorizing each instance of it.

Pitfalls & best practices¶

Same word, different meaning across teams

state, policy, deployment, and pipeline each mean something different depending on whether the speaker means it in the Terraform/infra sense or the Kubernetes/RL/CI sense. In a cross-functional design review, the single highest-leverage clarifying question is: "which <term> do you mean — the infra one or the ML one?" This appendix's per-domain sub-definitions (e.g. Kernel, Policy, Namespace) exist specifically to make that ambiguity visible rather than silently assumed away.

Don't let a glossary go stale. A term whose definition no longer matches how the codebase actually uses it is worse than no definition — it actively misleads. Treat this file like any other doc: update it in the same pull request that introduces or repurposes a term, not "later."
Precision over brevity when it's load-bearing. "Roughly the loss" is not a definition — a reviewer who reads "~cross-entropy~" without the formula cannot verify whether your code computes it correctly. Every entry above that has a formula also has the formula, not a hand-wave.
Don't compute a metric you can't also compute by hand once. If you cannot reproduce a Sharpe ratio, an IoU, or a KL divergence on a 3-number toy example with pencil and paper, you cannot sanity-check your code's output against a bug — you can only trust it blindly. Every worked example above is small enough to redo by hand in under a minute; do that at least once for any metric you rely on operationally.
Multiple-comparisons blindness. Reporting the best of $N$ tuned models/strategies without correcting for $N$ (Deflated Sharpe Ratio, above, is the finance-specific instance; the same logic applies to hyperparameter search leaderboards and A/B test dashboards with many simultaneous variants) systematically overstates how good the winner really is.
Confusing liveness with readiness, confusing a Security Group with a NACL's statefulness, or patching a creationPolicy: Owner ESO-managed Secret by hand are three of the most common operational glossary-adjacent mistakes — each looks like it works immediately and fails silently later (a restart loop, an unexpectedly open port, a secret key that vanishes on the next sync).

Exercises¶

Little's law, inverted. A batch-inference service must sustain $\lambda = 200$ requests/s. Your SLO requires an average of at most $L=20$ requests in flight at any time (to bound memory). What is the maximum average time-in-system $W$ you can afford? What does this imply about how fast each request's model call must complete?
Cross-entropy by hand. A 4-class classifier outputs $\hat p = [0.05, 0.15, 0.70, 0.10]$ for an example whose true class is the 4th. Compute the cross-entropy loss in nats, then recompute it assuming the model had instead put $\hat p_4 = 0.40$ with the other three probabilities rescaled proportionally. By how much did the loss change, and why is the relationship nonlinear?
KL divergence asymmetry. Using $p=[0.9,0.1]$ and $q=[0.5,0.5]$, compute both $D_{\mathrm{KL}}(p\|q)$ and $D_{\mathrm{KL}}(q\|p)$. Confirm they are not equal, and explain in one sentence why KL divergence is not a valid distance metric.
IoU threshold sensitivity. Two bounding boxes are $A=[0,0,20,20]$ and $B=[10,10,25,25]$. Compute their IoU. Would this pair count as a match under the common $\text{IoU} \ge 0.5$ detection threshold? At what threshold does it stop counting as a match?
Deflated Sharpe intuition. You backtested $N=100$ independent strategy variants and the best one shows an annualized Sharpe of 1.5, with per-trial Sharpe standard error $\sigma_{SR}=1$. Using the approximation $\mathbb E[\max \widehat{SR}] \approx \sigma_{SR}\sqrt{2\ln N}$, estimate the Sharpe you'd expect from pure luck alone with 100 trials. Is 1.5 still impressive? What would you need to do differently next time (see Ch. 23) to make a defensible claim of real edge?
Token bucket capacity. A rate limiter uses a token bucket with burst capacity $B=50$ and refill rate $r=5$ tokens/s. A client sends a burst of 50 requests instantly (bucket starts full), then keeps sending at a steady 8 requests/s. After the initial burst, how long until the client starts seeing 429s, and at what steady rate does it then get accepted vs. rejected once the bucket reaches equilibrium?
Cosine annealing by hand. Using $\eta_{\max}=2\text{e-}4$, $\eta_{\min}=0$, and $T=1000$ total steps, compute the learning rate $\eta_t$ at $t=250$, $t=500$, and $t=750$. Sketch (in words) why the rate falls faster in the middle of training than near the very start or very end.
One-page cross-functional map. Pick any two chapters that are non-adjacent in the book's arc (e.g. Terraform and Reinforcement learning). Find one term from this glossary that appears in both, in a different sense each time (hint: state, policy), and write two or three sentences explaining the two meanings to someone who only knows one of the two chapters.