Node.js & Express — Concept Notes
Plain language, real depth. Everything from "what is Node" to a production-ready API. Print this page for a PDF.
1. What Node.js Actually Is
Node is JavaScript running outside the browser. Same language, different place.
In the browser, JavaScript gets tools like document (the web page) and window (the browser tab). Node takes those away and gives different tools instead: read files on your computer, listen for network connections, talk to a database.
Node is called a runtime. A runtime is simply "the program that runs your code and gives it tools". Chrome is a runtime for browser JavaScript; Node is a runtime for server JavaScript.
Like the same person working from home instead of the office. Same skills, different equipment available.
What Node is not
- Not a language — it's JavaScript. There's no "Node syntax" to learn.
- Not a framework — a framework is a ready-made skeleton that tells you where to put your code. Node doesn't do that. Express (and others) do.
- Not a web server by itself — but it gives you the pieces to build one in a few lines.
What's inside it
- V8 — the JavaScript engine (the part that reads your JavaScript and turns it into machine instructions). Google built it for Chrome; Node reuses it.
- libuv — a C library (a bundle of ready-made code written in the C language) that Node uses to talk to the operating system. It handles all the slow stuff: files, network, timers. This is what makes Node good at waiting.
- Built-in modules — ready-made toolboxes that ship with Node, so you don't need to install them:
fs(files),http(web server),path(file paths),crypto(hashing and encryption) and friends.
What Node is good and bad at
| Great at | Poor at |
|---|---|
| APIs and web servers | Heavy number-crunching |
| Lots of connections at once | Video/image processing in-process |
| Real-time (chat, live updates) | Long-running calculations |
| Reading/writing files, proxying | Anything that blocks for seconds |
The pattern: Node is excellent at work that is mostly waiting — waiting for a database, a file, or another API to answer. It struggles with work that is mostly thinking — heavy maths done by the CPU.
Example: an API that loads a user from the database spends 99% of its time waiting for the database. Node is perfect for that. Resizing a 20MB photo is pure thinking, so Node is a poor fit. Section 3 explains exactly why.
2. How Node Runs Your Code
A thread is one line of work inside a program — one worker that can do one thing at a time. Node has one thread running your JavaScript.
But underneath, it has helpers. When you ask for a file or make a web request, Node hands that job to a helper and goes straight back to running your code. When the helper is done, your callback (the function you gave it, meaning "call this back when you're finished") is put in a queue to run.
A single waiter in a restaurant. They don't stand at the kitchen waiting for one meal — they take other orders, and come back when the bell rings.
Why this beats the traditional approach
Older servers gave each connection its own thread. A thread costs about 1MB of memory just to exist. The computer also has to keep switching between threads, which wastes time. Ten thousand connections meant ten thousand threads, and most of them were just sitting there waiting.
Node keeps one thread and a list of "things to do when ready". An idle connection costs almost nothing — it's just one entry that the operating system is watching.
Two words from the diagram: the operating system (OS) is macOS, Linux or Windows — the base software that controls the hardware. The thread pool is a small team of extra threads (4 by default) that libuv uses for jobs the OS can't do in the background, such as reading files or hashing passwords.
3. Blocking vs Non-Blocking
Blocking means "everything stops until this finishes". Node has only one thread for your code. So a blocking operation freezes every user's request, not just the one that caused it.
Non-blocking means "start this, and tell me when it's done". Node keeps serving everyone else in the meantime.
Blocking is the cashier who stops the whole queue to go and find a price. Non-blocking is sending a colleague to check while continuing to serve.
// ✗ BLOCKING — nobody else gets served while this file is read
const data = fs.readFileSync("big.json");
console.log(data);
// ✓ NON-BLOCKING — Node serves other requests while the file loads
const data = await fs.promises.readFile("big.json");
console.log(data);
The thing people get wrong
Adding the word async does not make code non-blocking. If the work is calculation (the CPU doing maths), it still runs on the one thread. It still freezes everything:
// Still blocks completely — async doesn't help here
app.get("/report", async (req, res) => {
let total = 0;
for (let i = 0; i < 1e10; i++) total += i; // ~10 seconds of pure thinking
res.json({ total });
});
async only helps when you're waiting for something outside your code (a database, a file, another server). It can't split a calculation across threads.
Think of a chef. async lets the chef start the oven and do other things while the cake bakes. It does not help if the chef has to chop 10,000 onions by hand — that still takes all of the chef's time.
Things that block (and surprise people)
- Any
…Syncfunction —readFileSync,execSync. "Sync" (synchronous) means "wait right here until it's done". JSON.parseon a very large string.- Big loops, sorting huge arrays, complex regular expressions (text-matching patterns like
/^a+b$/). - Password hashing done synchronously (
bcrypt.hashSync).
What to do about heavy work
- Worker threads — run the calculation on a separate thread, so the main thread stays free.
- A queue — push the job to Redis/BullMQ (a job list stored outside your app) and let a separate worker process pick it up and do it.
- Break it up — process in small chunks. Between chunks, yield (pause briefly and give control back to Node) so other requests get a turn.
// Break a long job into chunks so the server stays responsive
async function processAll(items) {
for (let i = 0; i < items.length; i++) {
process(items[i]);
if (i % 100 === 0) await new Promise(r => setImmediate(r)); // let others through
}
}
readFileSync to load a config file before your server starts listening is fine — no users are connected yet, so nobody is kept waiting. It only becomes a problem once requests are coming in.4. The Event Loop
The event loop is Node's to-do list manager. It keeps asking the same question, over and over: "Has anything finished? Then run its callback."
It works through phases. A phase is one stop on its round, and each stop holds a different kind of job — timers at one stop, file and network results at another. It clears one phase, moves to the next, and then starts the round again.
A postal sorting office doing rounds: first the letters, then the parcels, then the special delivery — round and round, all day.
nextTick cut in between phases, which is why they always run before timers.The classic ordering question
console.log("1 sync");
setTimeout(() => console.log("5 timeout"), 0);
setImmediate(() => console.log("6 immediate"));
Promise.resolve().then(() => console.log("4 promise"));
process.nextTick(() => console.log("3 nextTick"));
console.log("2 sync");
// 1 sync → 2 sync → 3 nextTick → 4 promise → 5 timeout → 6 immediate
How to read that result, step by step:
- Plain (synchronous) code runs first, top to bottom: "1 sync", then "2 sync".
- Then
process.nextTickcallbacks (highest priority): "3 nextTick". - Then promise callbacks, also called microtasks (small jobs that run as soon as the current code finishes, before the loop moves on): "4 promise".
- Then the event loop phases in order: timers ("5 timeout"), then check ("6 immediate").
One catch: at the top level of a file, the timeout-vs-immediate order isn't guaranteed. But inside a file or network callback, setImmediate always wins. Why? The loop is already past the timers phase at that point, and the check phase comes next.
5. Modules
A module is one file of code. Everything inside it is private unless you choose to share it with export.
Node has two systems for this. CommonJS (require) is the old one. ES Modules (import) is the modern standard, the same one browsers use. You'll meet both.
Each file is a room with a closed door. export puts something on a shelf outside the door. import/require is another room taking it from that shelf.
// CommonJS — the older style, still everywhere
const express = require("express");
module.exports = { createUser };
// ES Modules — the modern standard
import express from "express";
export { createUser };
Turning on ES Modules
// package.json
{ "type": "module" } // now .js files use import/export
Without that line, Node treats every .js file as CommonJS. You can also name one file .mjs to make just that file an ES module, or .cjs to force it to be CommonJS.
Two table words: top-level await means using await directly in a file, outside any function. Unused code removed (also called tree shaking) means a build tool can drop functions you never import, making the final bundle smaller.
| CommonJS | ES Modules | |
|---|---|---|
| Syntax | require / module.exports | import / export |
| Loads | When the line runs | Before anything runs |
Top-level await | No | Yes |
__dirname | Available | Must build it yourself |
| Unused code removed | No | Yes (bundlers can) |
// __dirname in ES Modules
import { fileURLToPath } from "url";
import { dirname } from "path";
const __dirname = dirname(fileURLToPath(import.meta.url));
Modules are cached
// db.js
console.log("connecting…");
export const db = createConnection();
// imported in five files → "connecting…" prints ONCE
Node loads each file once, remembers the result (the module cache), and hands the same copy to every file that asks for it. So a database connection or config object created in a module is shared by your whole app. That's useful — but worth knowing so it doesn't surprise you. For example, if one file changes a value on a shared config object, every other file sees the change.
6. Node.js Process Lifecycle
A process is one running copy of a program. When you type node app.js, the operating system starts a Node process. The lifecycle is everything that happens to that process, from the moment it starts to the moment it exits.
In short: Node starts its engine, loads your files, runs your code from top to bottom once, and then enters the event loop. It stays in the loop as long as there is something left to wait for (a timer, an open server, a pending file read). When nothing is left, it exits by itself.
A shop's day. Unlock the door and turn on the lights (start V8 and libuv). Stock the shelves (load modules). Do the opening checklist (top-level code). Then serve customers as they come in (the event loop). When there are no customers and nothing is booked, lock up and go home (exit). If the owner phones and says "close now", you finish with the current customer, then lock up (graceful shutdown).
Step by step
node app.js— the operating system starts a new process. Node reads settings such as environment variables and command-line flags (for example--env-file=.env).- V8 and libuv start. V8 (the JavaScript engine) gets ready to run code. libuv (the C library Node uses to talk to the operating system for files, network and timers) sets up the event loop and the thread pool.
- Module loading. Node reads
app.js, finds everyrequireorimport, and loads those files too. Each file runs once and is stored in the module cache, so the secondrequire("./db")gets the same object back, instantly. With ES Modules, all imports are found and loaded before any code runs. With CommonJS, eachrequireloads when its line is reached. - Top-level code runs. "Top-level" means code that is not inside a function — it runs straight away, top to bottom. This is where you call
app.listen(), start timers, and open database connections. These calls register work with the event loop; they don't wait for it. - Microtasks drain. Before the loop starts, Node runs every queued
process.nextTickcallback, then every promise callback. - The event loop starts and goes round its phases in order: timers → pending callbacks → idle/prepare → poll → check → close callbacks. Between every single callback, Node empties the
nextTickqueue and then the promise queue. - Exit. After each lap, Node asks "is anything still pending?" If there are no timers, no open servers or sockets, and no file reads in flight, it emits
'beforeExit', then'exit', and the process ends.
The phases, in order
| Phase | What runs here | Example |
|---|---|---|
| timers | Timer callbacks whose time is up | setTimeout, setInterval |
| pending callbacks | A few system-level callbacks delayed from the last lap | Some TCP connection errors |
| idle / prepare | Node's internal housekeeping — your code never runs here | — |
| poll | Results of I/O (input/output: files, network). Waits here if nothing else is due | fs.readFile callback, incoming HTTP request |
| check | Callbacks scheduled to run right after poll | setImmediate |
| close callbacks | Clean-up events for things that closed | socket.on("close") |
| between every callback | Microtask queues, emptied fully | process.nextTick first, then promise .then |
When does the process exit?
Node exits by itself when the event loop has nothing left to do. "Something to do" means any of these: a timer that hasn't fired yet, an open server or socket, or a file or network operation still in progress.
// Exits right away — nothing is pending after the log
console.log("hi");
// Exits after 2 seconds — the timer keeps the loop alive until it fires
setTimeout(() => console.log("bye"), 2000);
// Never exits on its own — an open server is always "pending"
http.createServer(handler).listen(3000);
// timer.unref() = "don't keep the process alive just for me"
setInterval(sendMetrics, 10000).unref();
It can also exit early: you call process.exit(code), an error crashes it, or the operating system sends a signal that stops it.
Process events
The process object is an EventEmitter (see Events & EventEmitter), so you can listen for moments in its life.
| Event | When it fires | Can you still do async work? |
|---|---|---|
'beforeExit' | The loop is empty and Node is about to exit on its own. Not fired for process.exit() or crashes | Yes — scheduling new work here keeps the process alive |
'exit' | The process is definitely ending, for any reason. You get the exit code | No — only synchronous code runs; timers and promises are ignored |
'uncaughtException' | An error was thrown and nothing caught it | Log it, then exit — the app is in an unknown state |
'unhandledRejection' | A promise failed and had no .catch / try…catch. Since Node 15 this crashes the process by default | Log it, then shut down |
'SIGINT' | A signal (a short message from the operating system) sent when you press Ctrl+C | Yes — run your shutdown code |
'SIGTERM' | The polite "please stop" signal sent by Docker, Kubernetes, PM2 or your host before a deploy | Yes — run your shutdown code (you usually get ~10–30 seconds) |
Exit codes: a number the process hands back to the system when it ends. 0 means "all good". Anything else (usually 1) means "something went wrong", which tells Docker, CI or your process manager to treat it as a failure.
Graceful shutdown
Graceful shutdown means stopping cleanly: stop taking new work, finish the work you already have, close connections, then exit. The opposite is being killed mid-request, which leaves users with errors and data half-saved.
const server = app.listen(3000);
let shuttingDown = false;
async function shutdown(signal) {
if (shuttingDown) return; // ignore a second Ctrl+C / SIGTERM
shuttingDown = true;
console.log(`${signal} received, shutting down…`);
// Safety net: if clean-up hangs, force exit after 10 s
setTimeout(() => process.exit(1), 10_000).unref();
server.close(async (err) => { // 1. stop accepting NEW connections;
// callback runs once in-flight requests finish
try {
await db.end(); // 2. close the database pool
await redis.quit(); // 3. close other connections
process.exit(err ? 1 : 0); // 4. exit: 0 = clean, 1 = problem
} catch (e) {
console.error(e);
process.exit(1);
}
});
}
process.on("SIGTERM", () => shutdown("SIGTERM")); // host / Docker / Kubernetes
process.on("SIGINT", () => shutdown("SIGINT")); // Ctrl+C
process.on("uncaughtException", (err) => {
console.error("Uncaught:", err);
process.exit(1); // state is unknown — don't carry on
});
process.on("unhandledRejection", (reason) => {
console.error("Unhandled rejection:", reason);
shutdown("unhandledRejection");
});
process.on("exit", (code) => {
console.log("exiting with code", code); // sync only — no await here
});
'exit' handlers can't wait for anything — an await db.end() inside one never finishes. Do async clean-up in your signal handler instead. (2) Calling process.exit() straight away in a SIGTERM handler throws away in-flight requests — always server.close() first. (3) SIGKILL (e.g. kill -9) can't be caught at all; that is what the host sends if you take too long.7. Events & EventEmitter
An EventEmitter lets one part of your code announce "this happened", and other parts listen for it. Neither side needs to know about the other.
To emit means to announce an event. A listener is a function that waits for a named event and runs when it happens. This pattern is everywhere in Node: servers, streams, and sockets (open network connections) are all EventEmitters underneath.
A doorbell. It doesn't know who's inside or what they'll do. It just rings, and whoever cares responds.
import { EventEmitter } from "events";
const orders = new EventEmitter();
orders.on("created", (order) => sendEmail(order)); // listener 1
orders.on("created", (order) => updateStats(order)); // listener 2
orders.emit("created", { id: 1 }); // both listeners run, in order
orders.once("ready", fn); // run at most once, then auto-remove
orders.off("created", fn); // remove a specific listener
orders.listenerCount("created"); // how many are attached
Why this matters
It decouples your code — meaning the parts don't depend on each other directly. The function that creates orders doesn't need to know that emails and statistics exist. Add a third listener later (say, "notify the warehouse") and you change nothing in the original code.
Two traps
// 1. Listeners run SYNCHRONOUSLY — a slow one blocks the others
orders.on("created", () => { heavyWork(); }); // everything waits
// 2. An "error" event with no listener CRASHES the process
emitter.on("error", (err) => log(err)); // always add this
Node also warns you when one event gets 11 or more listeners ("possible memory leak"). A memory leak is memory your app keeps using and never gives back, so it slowly grows until it crashes. The warning is usually a real bug: a listener is being added inside a function that runs again and again (for example, on every request), and never removed.
8. The File System
The fs ("file system") module reads and writes files. It comes in three flavours:
- Promise-based — use this. You write
await fs.readFile(…). - Callback-based — the older style. You pass a function to be called when it's done.
- Sync — waits and blocks the whole server. Avoid it once the server is running.
import fs from "fs/promises";
const text = await fs.readFile("notes.txt", "utf8"); // without "utf8" you get raw bytes
await fs.writeFile("out.txt", "hello"); // overwrites
await fs.appendFile("log.txt", "line\n"); // adds to the end
await fs.mkdir("uploads", { recursive: true }); // recursive = make parents too
const files = await fs.readdir("uploads");
await fs.rm("old.txt", { force: true }); // force = don't error if missing
const info = await fs.stat("file.txt");
info.size; info.isDirectory(); info.mtime;
Checking whether a file exists
// ✗ don't check-then-act — the file can change in between
if (await exists(path)) { await fs.readFile(path); }
// ✓ just try it and handle the failure
try {
const data = await fs.readFile(path, "utf8");
} catch (err) {
if (err.code === "ENOENT") return null; // not found
throw err;
}
Why not check first? Between your check and your read, another process could delete or change the file. This gap is called a race condition (two things "racing" to touch the same thing). Just trying and handling the error avoids the gap.
Error codes worth recognising: ENOENT (no such file or folder), EACCES (no permission), EISDIR (it's a folder, not a file), EEXIST (already there).
Always build paths with the path module
import path from "path";
path.join(__dirname, "uploads", "a.png"); // handles / vs \ across operating systems
path.resolve("uploads"); // → absolute path
path.extname("photo.png"); // ".png"
path.basename("/a/b/photo.png"); // "photo.png"
.. means "go up one folder", so a filename like ../../etc/passwd climbs out of your folder and reads a system file. This attack is called path traversal. Use path.basename() to strip the folder parts, and check that the final (resolved) path still starts with the folder you intended.9. Streams & Buffers
A stream handles data in small pieces (called chunks) as they arrive, instead of loading everything into memory first.
A buffer is a chunk of raw binary data — the actual bytes (0s and 1s grouped into numbers from 0 to 255), before they are turned into text or an image.
Reading a file normally is filling a bucket then carrying it. A stream is a hose — the water flows through continuously, and you never hold more than a trickle at once.
import { createReadStream, createWriteStream } from "fs";
import { pipeline } from "stream/promises";
import { createGzip } from "zlib";
// read → compress → write, in chunks, with errors handled
await pipeline(
createReadStream("big.log"),
createGzip(),
createWriteStream("big.log.gz")
);
Use pipeline() rather than chaining .pipe(). If one step fails, pipeline() closes all the other streams and gives you the error. .pipe() does not, so files can be left open and memory can leak.
Backpressure (in the diagram) means "slow down, I can't keep up". Example: reading from a fast disk while uploading over a slow network. Without backpressure, the unsent chunks would pile up in memory.
The four stream types
- Readable — you read from it (a file, an HTTP request body).
- Writable — you write to it (a file, an HTTP response).
- Duplex — both at once (a network socket: you send and receive).
- Transform — reads, changes, writes (gzip compression, encryption).
Buffers
const buf = Buffer.from("hello"); // text → bytes
buf.toString("utf8"); // bytes → text
buf.toString("base64"); // bytes → base64 text
buf.length; // size in BYTES, not characters
That last point matters: "café" is 4 characters but 5 bytes, because é takes two bytes in UTF-8 (the standard way of storing text as bytes). That's why file size limits should be checked in bytes. Base64 is a way of writing any bytes as plain letters and digits, so binary data (like an image) can travel inside JSON or a URL.
10. Paths, URLs & Process
process is a global object (available everywhere, no import needed) that describes the running program itself — its settings, its memory use, and messages from the operating system.
The most useful part is signals. A signal is a short message the operating system sends to a process, such as "please stop". When your hosting platform wants to stop your app, it politely asks first with a signal. Listening for that lets you finish what you're doing instead of being cut off mid-sentence.
process — information about the running program
process.env.NODE_ENV; // environment variables
process.argv; // command-line arguments
process.cwd(); // the folder you ran it from
process.memoryUsage();
process.exit(1); // stop, with an error code
process.on("SIGTERM", shutdown); // the host is asking you to stop
process.on("uncaughtException", handler);
process.on("unhandledRejection", handler);
Graceful shutdown — often missed, always asked about
const server = app.listen(3000);
async function shutdown() {
server.close(); // stop accepting NEW requests
await finishInFlightRequests(); // let current ones complete
await db.end(); // close the database cleanly
process.exit(0);
}
process.on("SIGTERM", shutdown);
process.on("SIGINT", shutdown); // Ctrl+C
Without this, a deploy kills the process in the middle of requests. Users see errors, and database changes can be left half-written. Every deployment system sends SIGTERM first and then waits a little while, so handling it is free reliability. (The full version, with timeouts and exit codes, is in Node.js Process Lifecycle.)
Environment variables
Environment variables are settings passed to your app from outside the code — database address, secret keys, port number. The same code can then run on your laptop and in production with different settings. A .env file is a simple text file that holds them during development.
# .env — never commit this file
DATABASE_URL=postgres://…
JWT_SECRET=…
import "dotenv/config"; // or: node --env-file=.env app.js
const url = process.env.DATABASE_URL;
if (!process.env.JWT_SECRET) { // fail loudly at startup, not at 3am
throw new Error("JWT_SECRET is missing");
}
URLs
const url = new URL("https://site.com/search?q=node&page=2");
url.pathname; // "/search"
url.searchParams.get("q"); // "node"
11. An HTTP Server Without Express
Node can serve web requests on its own — Express is optional. But you have to do everything by hand: work out which URL was asked for, collect the request body piece by piece, and set every header yourself (headers are small labels sent with a request or response, like Content-Type: application/json).
Writing one is the fastest way to understand what a framework is actually saving you.
Worth building once. It shows you exactly what Express is doing for you.
import http from "http";
const server = http.createServer((req, res) => {
// No routing — you check the method and URL yourself
if (req.method === "GET" && req.url === "/users") {
res.writeHead(200, { "Content-Type": "application/json" });
return res.end(JSON.stringify([{ id: 1 }]));
}
if (req.method === "POST" && req.url === "/users") {
// No body parsing — you collect the chunks yourself
let body = "";
req.on("data", chunk => { body += chunk; });
req.on("end", () => {
const user = JSON.parse(body); // and handle bad JSON yourself
res.writeHead(201, { "Content-Type": "application/json" });
res.end(JSON.stringify(user));
});
return;
}
res.writeHead(404);
res.end("Not found");
});
server.listen(3000);
What that reveals
Everything Express gives you is missing here:
- no routing (matching a URL to a function),
- no body parsing (turning the raw request text into a JavaScript object),
- no URL parameters (like the
42in/users/42) and no query parsing (like?page=2), - no middleware and no error handling.
You'd write all of it yourself — and the if chain becomes unmanageable after about five routes.
12. npm & package.json
npm (Node Package Manager) downloads and installs code other people have written. Each downloadable bundle is called a package. package.json is the list of packages your project uses, plus shortcuts (scripts) for running it.
A dependency is needed to run the app. A devDependency is only needed while building or testing it, so it isn't shipped to production.
package.json is a shopping list ("some bread, any brand from this year"). The lockfile is the receipt ("this exact loaf, this brand, this batch").
npm init -y # create package.json
npm install express # add a dependency
npm install -D nodemon # dev-only (not shipped to production)
npm ci # clean install from the lockfile — use this in CI
npm outdated # what's behind
npm audit fix # patch known vulnerabilities
{
"name": "my-api",
"type": "module",
"scripts": {
"dev": "node --watch server.js",
"start": "node server.js",
"test": "vitest"
},
"dependencies": { "express": "^4.19.0" },
"devDependencies": { "vitest": "^1.0.0" }
}
Version symbols
Versions follow semver (semantic versioning): MAJOR.MINOR.PATCH, e.g. 4.19.2. PATCH = bug fixes. MINOR = new features that don't break anything. MAJOR = breaking changes that may need you to change your code.
| Written | Accepts | Meaning |
|---|---|---|
^4.19.0 | 4.x.x | Bug fixes and new features, no breaking changes |
~4.19.0 | 4.19.x | Bug fixes only |
4.19.0 | exactly that | Locked |
The lockfile
package-lock.json records the exact version of every package, including the packages your packages depend on. Commit it to git. Without it, two developers running npm install on different days can end up with different code — the classic "works on my machine".
npm install vs npm ci: install may update the lockfile. ci installs exactly what the lockfile says, and fails if it disagrees with package.json. Use ci in automated builds (CI, continuous integration — the server that builds and tests your code on every push) so every build is identical, or reproducible.13. Why Express
Express is a thin layer on top of Node's HTTP server. It handles the boring, repetitive parts: matching URLs to functions, reading request bodies, and sending responses.
It's deliberately unopinionated — it doesn't tell you how to organise your project (which folders, which file names). That's freedom, and it's also why two Express codebases can look completely different.
Node's http module is a bare kitchen. Express adds labelled drawers and a recipe board — but you still decide what to cook and where to put things.
import express from "express";
const app = express();
app.use(express.json()); // parse JSON bodies
app.get("/users/:id", (req, res) => {
res.json({ id: req.params.id });
});
app.listen(3000, () => console.log("ready"));
Compare that to section 11: routing, parameters, body parsing and JSON responses all handled in a few lines.
Express 4 vs 5
Express 5 (2024) fixes the most annoying thing about Express 4: errors thrown inside async handlers are now caught automatically. In Express 4, such an error becomes an unhandled rejection (a failed promise that nobody caught), which crashes the process unless you wrap every handler.
// Express 4 — this UNHANDLED rejection crashes the app
app.get("/x", async (req, res) => { throw new Error("boom"); });
// Express 4 fix — a wrapper
const wrap = fn => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next);
app.get("/x", wrap(async (req, res) => { … }));
// Express 5 — just works
Alternatives
"Ecosystem" below means all the extra packages, tutorials and answers online built around a tool. "Edge runtimes" are small JavaScript environments that run on servers close to the user (e.g. Cloudflare Workers).
| Framework | Trade-off |
|---|---|
| Express | Huge ecosystem, most tutorials, most jobs. Slowest of these. |
| Fastify | Much faster, built-in validation and schemas. Smaller ecosystem. |
| NestJS | Structure decided for you (like Angular). Great for large teams, heavier to learn. |
| Hono | Tiny and fast; runs on edge runtimes as well as Node. |
14. Routing
Routing means deciding which function handles which URL — GET /users goes here, POST /orders goes there.
The method is the verb of the request: GET (read), POST (create), PUT/PATCH (update), DELETE (remove). The path is the part of the URL after the domain, like /users/42. Express matches the method and the path together, and checks your routes in the order you wrote them.
A receptionist with a list: "Deliveries → back door. Interviews → 3rd floor." They read the list from the top and send you to the first match.
app.get("/users", handler);
app.post("/users", handler);
app.put("/users/:id", handler);
app.patch("/users/:id", handler);
app.delete("/users/:id", handler);
app.all("/health", handler); // any method
Getting data out of a request
// URL parameter: /users/42
app.get("/users/:id", (req, res) => req.params.id); // "42" — always a string
// Query string: /search?q=node&page=2
app.get("/search", (req, res) => {
req.query.q; // "node"
req.query.page; // "2" — also a string, convert it yourself
});
// Body: needs express.json() first
app.post("/users", (req, res) => req.body.name);
req.params.id is "42", not 42. Comparing it with === to a number silently fails ("42" === 42 is false). Convert with Number(id) — and check the result isn't NaN ("Not a Number", what you get from Number("abc")).Route order matters
// ✗ "/users/new" never runs — ":id" matches "new" first
app.get("/users/:id", getUser);
app.get("/users/new", newUserForm);
// ✓ specific routes before dynamic ones
app.get("/users/new", newUserForm);
app.get("/users/:id", getUser);
Express checks routes top to bottom and stops at the first match. :id means "any value here", so it happily matches the word new. This trips people up constantly.
Patterns
app.get("/files/*", handler); // anything under /files
app.get("/posts/:year/:month", handler); // multiple params
app.get(/^\/ab+cd$/, handler); // a regular expression
15. Middleware — the heart of Express
Middleware is a function that sits in the middle — between the incoming request and your final handler (the function that actually answers the request). It can look at the request, change it, stop it, or pass it along.
Each one calls next() to hand over to the next function in line. Express runs them in the order you registered them (the order of your app.use lines).
Airport security: check-in, then passport, then bag scan, then the gate. Each station can wave you through or stop you.
// A middleware is just a function with these three arguments
function logger(req, res, next) {
console.log(req.method, req.url);
next(); // ← without this, the request hangs
}
app.use(logger); // runs for every request
app.use("/admin", requireAdmin); // only for paths under /admin
app.get("/users", requireAuth, getUsers); // only for this one route
The order you register them is the order they run
// ✗ auth runs AFTER the route — it never protects anything
app.get("/secret", handler);
app.use(requireAuth);
// ✓
app.use(requireAuth);
app.get("/secret", handler);
A typical setup, in the right order
Quick glossary for the code below: helmet adds security headers. CORS (Cross-Origin Resource Sharing) decides which other websites' browsers may call your API. morgan logs each request. Rate limiting caps how many requests one client can make in a time window.
app.use(helmet()); // security headers
app.use(cors({ origin: "https://app.com" })); // who may call this API
app.use(express.json({ limit: "10kb" })); // parse JSON, cap the size
app.use(morgan("dev")); // request logging
app.use(rateLimit({ windowMs: 60000, max: 100 }));
app.use("/api/users", userRoutes); // your routes
app.use(notFound); // 404 — after all routes
app.use(errorHandler); // errors — always LAST
Passing data between middleware
function requireAuth(req, res, next) {
const user = verifyToken(req.headers.authorization);
if (!user) return res.status(401).json({ error: "Not logged in" });
req.user = user; // attach it — later handlers can read it
next();
}
app.get("/me", requireAuth, (req, res) => res.json(req.user));
Attaching data to req (like req.user) is the standard way to share it with the functions further down the chain — the same req object travels through all of them. Note the return before res.status(401). Without it, the code carries on to next(), a second response is attempted, and you get "cannot set headers after they are sent".
16. Request & Response
req (request) holds everything the client (the browser, app or tool calling your API) sent — the URL, headers, and body. res (response) is how you reply.
The one rule that catches everyone: you get one response per request. Sending a second one throws an error.
A letter and its reply. You can only post one reply per letter — once it's in the postbox, you can't add another page.
Reading the request
req.params // URL parts: /users/:id
req.query // after the ?: ?sort=name
req.body // parsed body (needs express.json())
req.headers // all headers, lowercase keys
req.method // "GET"
req.path // "/users/42"
req.cookies // needs cookie-parser
req.ip
Sending the response
res.json({ ok: true }); // sets Content-Type automatically
res.status(201).json(newUser); // chain the status
res.status(204).end(); // no content
res.send("plain text");
res.redirect("/login");
res.sendFile(path.join(__dirname, "a.pdf"));
res.set("Cache-Control", "no-store");
res.cookie("session", id, { httpOnly: true, secure: true, sameSite: "lax" });
Status codes you'll actually use
| Code | When |
|---|---|
| 200 OK | Normal success |
| 201 Created | Something new was made |
| 204 No Content | Success, nothing to send back (often DELETE) |
| 400 Bad Request | The input was invalid |
| 401 Unauthorized | "I don't know who you are" — log in |
| 403 Forbidden | "I know who you are, and no" |
| 404 Not Found | That thing doesn't exist |
| 409 Conflict | Clashes with current state (duplicate email) |
| 422 Unprocessable | Valid format, failed business rules |
| 429 Too Many Requests | Rate limited |
| 500 Server Error | Your bug |
One response per request
// ✗ crashes: "Cannot set headers after they are sent"
if (!user) res.status(404).json({ error: "Not found" });
res.json(user); // runs anyway!
// ✓
if (!user) return res.status(404).json({ error: "Not found" });
res.json(user);
Why? res.status(404).json(…) sends the reply but does not stop your function. JavaScript keeps going to the next line, which tries to send a second reply. The return ends the function right there.
17. Express Request–Response Lifecycle
The request–response lifecycle is the journey one request takes through your app: it arrives, passes through each middleware in order, finds a matching route, runs your handler, and ends when a response is sent back.
Every step has only three choices: call next() to pass it on, send a response to finish, or call next(err) to report a problem. If a step does none of these, the request just sits there until the client gives up.
A parcel in a sorting centre. It goes down one conveyor belt past a series of stations — scan the label, check for danger, weigh it, sort by address — and finally a van delivers it. Any station can push it forward, reject it (an error), or deliver it early. If a station just holds on to it, the parcel never arrives.
Step by step
- Client sends a request — e.g.
POST /api/userswith a JSON body. - Node's http server receives it. It reads the raw bytes, works out the method, URL and headers, and creates two objects:
req(what came in) andres(how to answer). The body is not read yet — it is still a stream. - The Express app takes over. Your
appis really just a function that Node calls with(req, res). Express adds helpers likereq.params,res.json()andres.status(). - App-level middleware runs in order — everything registered with
app.use(), top to bottom:helmet(security headers) →cors(which websites may call you) →express.json()(reads the body stream and fillsreq.body) → a logger →auth(checks the token, setsreq.user). Each one callsnext(). - Router matching. Express compares the method and path against your routes, top to bottom, and picks the first match.
/users/:idfillsreq.params.id. - Route-level middleware runs — functions listed on that one route, such as
validate(schema)orrequireAdmin. - The route handler (controller) runs. It reads the input from
reqand calls the service. - Service and database. Business rules run and the database is queried. This is usually
await-ed — the event loop serves other requests while this one waits. res.json()/res.send()ends the cycle. Headers and body are written and the reply goes back to the client. After this, callingnext()or sending again is a bug.resemits'finish'once the last byte has been handed to the operating system. Loggers listen for this to record the status code and how long the request took.
Summary
| Stage | Registered with | Its job | Ends by |
|---|---|---|---|
| App-level middleware | app.use(fn) | Runs for every request: headers, CORS, body parsing, logging, auth | next() |
| Router matching | app.get(path, …), app.use("/api", router) | Find the first route whose method + path fit | Moves to that route, or falls through |
| Route-level middleware | app.post(path, mw, handler) | Checks only this route needs, e.g. validation | next() or an early 400/403 response |
| Route handler | last function on the route | Read input, call the service, reply | res.json() / res.send() |
| 404 handler | app.use(fn) after all routes | Catch requests no route matched | res.status(404).json(…) |
| Error handler | app.use((err, req, res, next) => …), last | Turn any error into one clean response | res.status(…).json(…) |
What can go wrong
next()is never called and no response is sent → the request hangs. Express has no timeout of its own; the client (or a load balancer) eventually gives up after 30–120 seconds. Common cause: anifbranch that forgets to call either.next(err)is called (or, in Express 5 / with a wrapper, an async function throws) → Express skips every normal middleware and route and jumps to the first error-handling middleware. Express recognises that function only because it has four parameters:(err, req, res, next).- No route matches → the request falls through to the end. Your 404 handler (a normal 3-argument middleware placed after all routes) answers it. Without one, Express sends its own plain "Cannot GET /x".
- Two responses are sent → "Cannot set headers after they are sent". You can check
res.headersSentif you're unsure.
A lifecycle you can watch
import express from "express";
const app = express();
// First middleware: start a timer and log when the response is finished
app.use((req, res, next) => {
req.start = Date.now();
console.log(`1. [app] ${req.method} ${req.url} arrived`);
res.on("finish", () => { // fires after the response is fully sent
console.log(`9. [finish] ${res.statusCode} in ${Date.now() - req.start}ms`);
});
next();
});
app.use(express.json()); // fills req.body
app.use((req, res, next) => { console.log("2. [app] body parsed", req.body); next(); });
// App-level auth
app.use((req, res, next) => {
console.log("3. [app] auth check");
req.user = { id: 7 }; // pretend we verified a token
next();
});
// Route-level middleware — only runs for this route
function validateUser(req, res, next) {
console.log("5. [route] validating");
if (!req.body?.name) return res.status(400).json({ error: "name is required" }); // ends here
next();
}
// Router matching + handler
app.post("/users", (req, res, next) => { console.log("4. [router] matched POST /users"); next(); },
validateUser,
async (req, res, next) => {
try {
console.log("6. [handler] calling service");
const user = await createUser(req.body); // service → DB
console.log("7. [handler] sending response");
res.status(201).json(user); // 8. cycle ends here (nothing logged)
} catch (err) {
next(err); // jump to error handler
}
});
// 404 — after all routes; only reached if nothing matched
app.use((req, res) => {
console.log("[404] no route matched");
res.status(404).json({ error: "Not found" });
});
// Error handler — 4 arguments, registered LAST
app.use((err, req, res, next) => {
console.log("[error]", err.message);
res.status(err.statusCode || 500).json({ error: "Something went wrong" });
});
app.listen(3000);
// POST /users {"name":"Ada"} prints:
// 1. [app] POST /users arrived
// 2. [app] body parsed { name: 'Ada' }
// 3. [app] auth check
// 4. [router] matched POST /users
// 5. [route] validating
// 6. [handler] calling service
// 7. [handler] sending response
// 9. [finish] 201 in 12ms
app.use. And remember: res.json() doesn't stop your function — use return res.json(…) when more code follows.18. Routers & Structure
A Router is a group of related routes bundled into their own file, which you then attach at one URL prefix.
Like departments in a building. The sign at the entrance says where each one is; inside, they organise themselves.
Putting every route in one file works until about 200 lines. A Router is a mini-app you can mount at a path — "mount" just means "attach under this URL prefix", so / inside the router becomes /api/users outside.
// routes/users.js
import { Router } from "express";
const router = Router();
router.get("/", listUsers); // → /api/users
router.get("/:id", getUser); // → /api/users/:id
router.post("/", createUser);
export default router;
// app.js
import userRoutes from "./routes/users.js";
app.use("/api/users", userRoutes); // the prefix lives in ONE place
Change the prefix to /api/v2/users and every route moves with it — you don't touch the router file at all.
Middleware on a whole router
router.use(requireAuth); // protects every route below this line
router.get("/", listUsers);
19. Project Layout
src/
app.js ← builds the Express app (no listen)
server.js ← starts it. Split so tests can import app alone
config/
env.js ← read + validate environment variables once
routes/
users.routes.js ← URL → controller
controllers/
users.controller.js ← HTTP only: read input, send response
services/
users.service.js ← the actual rules. Knows nothing about HTTP
repositories/
users.repo.js ← the only place that talks to the database
middleware/
auth.js error.js validate.js
utils/
AppError.js
The point of the layers is what each one doesn't know.
The controller knows about HTTP but not business rules. The service knows the rules but has never heard of req or res. The repository knows the database and nothing else.
Business rules are the decisions that make your app your app — "an email can only be registered once", "orders over £100 ship free". They're not about HTTP or databases.
A restaurant: the waiter (controller) takes the order and brings the plate. The chef (service) decides how to cook it. The storeroom keeper (repository) is the only one who goes into the storeroom. The chef never talks to customers; the waiter never cooks.
// controller — thin. Translate HTTP in, HTTP out.
export async function createUser(req, res, next) {
try {
const user = await userService.create(req.body);
res.status(201).json(user);
} catch (err) { next(err); }
}
// service — the rules. Testable without a server.
export async function create(data) {
const existing = await userRepo.findByEmail(data.email);
if (existing) throw new AppError("Email already registered", 409);
const hashed = await bcrypt.hash(data.password, 12);
return userRepo.insert({ ...data, password: hashed });
}
// repository — the only file that knows SQL
export function findByEmail(email) {
return db.query("SELECT * FROM users WHERE email = $1", [email]);
}
Why it pays off
- A scheduled job (code that runs on a timer, e.g. every night) can call
userService.create()— there's no fakereqto build. - Swapping Postgres for MongoDB touches only the repository.
- Testing the rules needs no HTTP server at all.
GET /users), this is more ceremony than it's worth. Add layers when a file becomes hard to read. Knowing when a pattern isn't worth it is a better interview answer than reciting it.20. Validation
Validation means checking that incoming data is what you expect before you use it. For example: is email really an email? Is age a positive whole number?
The frontend already checks? Doesn't matter. Anyone can call your API directly with curl or Postman (tools that send requests without a browser). The browser form is not a security boundary — it's not a wall an attacker has to go through.
A bouncer checks IDs at the club door even if the ticket website already asked for your age. The website only helps honest people; the bouncer stops everyone else.
import { z } from "zod";
const createUserSchema = z.object({
name: z.string().min(2).max(50),
email: z.string().email(),
password: z.string().min(8),
age: z.number().int().positive().optional(),
});
// reusable middleware
const validate = (schema) => (req, res, next) => {
const result = schema.safeParse(req.body);
if (!result.success) {
return res.status(400).json({
error: "Validation failed",
details: result.error.issues.map(i => ({
field: i.path.join("."), message: i.message,
})),
});
}
req.body = result.data; // now cleaned and correctly typed
next();
};
app.post("/users", validate(createUserSchema), createUser);
The code above uses zod, a library where you describe the shape of valid data once (a schema) and it checks any object against it. safeParse returns success or a list of problems instead of throwing.
Validate everything that comes from outside
req.body— the obvious one.req.params— is that id actually a number?req.query—?limit=99999999will happily try to load your whole database.- Uploaded files — type and size.
// A sensible pagination guard
const limit = Math.min(Number(req.query.limit) || 20, 100); // never above 100
const page = Math.max(Number(req.query.page) || 1, 1);
21. Error Handling
The aim is a single place that turns any failure into a sensible response. Without it, error formatting gets copy-pasted into every route, and the copies slowly drift apart.
Two kinds of failure matter. Ones you expected (bad input, not found): tell the user plainly. Ones you didn't (real bugs): log the detail for yourself and show the user something generic.
A customer-service desk. Staff anywhere in the shop can send a problem to the desk, and the desk decides what the customer is told — so every customer gets the same polite answer.
The goal: one place that decides what the client sees, so you never leak a stack trace (the list of files and line numbers where the error happened — useful to you, useful to attackers too) and never repeat yourself.
// utils/AppError.js — errors you threw on purpose
export class AppError extends Error {
constructor(message, statusCode = 400) {
super(message);
this.statusCode = statusCode;
this.isOperational = true; // "expected", not a bug
}
}
// middleware/error.js — registered LAST
export function errorHandler(err, req, res, next) {
const status = err.statusCode || 500;
logger.error({ err, url: req.originalUrl, userId: req.user?.id });
// Never send internal details for unexpected errors
const message = err.isOperational ? err.message : "Something went wrong";
res.status(status).json({
error: message,
...(process.env.NODE_ENV === "development" && { stack: err.stack }),
});
}
Two kinds of error
- Operational — expected and handled: invalid input, not found, duplicate email. Tell the user plainly.
- Programmer errors — actual bugs:
undefinedis not a function. Log it, alert someone, show a generic message.
Async errors in Express 4
// Every async handler needs its errors passed to next()
const wrap = fn => (req, res, next) =>
Promise.resolve(fn(req, res, next)).catch(next);
app.get("/users", wrap(async (req, res) => {
const users = await userService.list(); // if this throws, wrap catches it
res.json(users);
}));
Last-resort safety nets
process.on("unhandledRejection", (err) => { logger.fatal(err); shutdown(); });
process.on("uncaughtException", (err) => { logger.fatal(err); process.exit(1); });
After an uncaught exception (an error nothing caught), the process is in an unknown state — some work may be half-done, some memory may be broken. So the correct move is to log and restart, not to carry on. Your process manager (a tool like PM2, Docker or Kubernetes that watches your app) will start a fresh one.
22. Databases
Connection pooling — why you never connect per request
Opening a database connection is slow. There's a handshake (the two sides agreeing how to talk), a login, and memory set aside on the database server — often 20–100ms before any real work.
A pool opens a fixed set of connections once and lends them out. A request borrows one, runs its query, and gives it back.
A library's copies of a book. You borrow and return; the library doesn't buy a new copy for every reader.
import pg from "pg";
const pool = new pg.Pool({
connectionString: process.env.DATABASE_URL,
max: 20, // total connections this process will open
idleTimeoutMillis: 30000,
});
const { rows } = await pool.query("SELECT * FROM users WHERE id = $1", [id]);
Always use placeholders
// ✗ SQL injection — the input becomes part of the query
db.query(`SELECT * FROM users WHERE email = '${email}'`);
// ✓ the value travels separately and can never be read as SQL
db.query("SELECT * FROM users WHERE email = $1", [email]);
SQL injection is when user input gets mixed into a database command and changes what the command does. Example: an "email" of ' OR '1'='1 turns the first query into "find users where email is empty OR 1 equals 1" — which matches every user.
Placeholders aren't just a rule to memorise. The database reads (parses) the query first, with $1 as an empty slot, and only then puts the value in. There's no moment where your data could be mistaken for a command. This is also called a parameterised query.
The N+1 problem
"N+1" means: 1 query to get a list of N items, then N more queries — one for each item. 100 posts = 101 trips to the database. An ORM (Object-Relational Mapper, like Prisma or Mongoose) is a library that lets you query the database with JavaScript methods instead of raw SQL; its "include" option can fetch related data in one go (a join).
// 1 query for the posts, then one MORE per post = 101 queries
const posts = await Post.find();
for (const p of posts) p.author = await User.findById(p.userId);
// ✓ two queries, whatever the post count
const posts = await Post.find();
const users = await User.find({ _id: { $in: posts.map(p => p.userId) } });
const byId = new Map(users.map(u => [String(u._id), u]));
posts.forEach(p => { p.author = byId.get(String(p.userId)); });
// ✓ or let the ORM join
await prisma.post.findMany({ include: { author: true } });
It works perfectly with 10 test rows and collapses with 10,000 real ones. Log your queries in development — if one page triggers 200, you've found it.
Transactions
A transaction groups several database changes into one all-or-nothing unit. BEGIN starts it, COMMIT saves everything, ROLLBACK undoes everything. Example: a bank transfer must take money from one account and add it to the other — never just one of the two.
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query("UPDATE accounts SET balance = balance - $1 WHERE id = $2", [500, from]);
await client.query("UPDATE accounts SET balance = balance + $1 WHERE id = $2", [500, to]);
await client.query("COMMIT");
} catch (err) {
await client.query("ROLLBACK"); // both changes undone
throw err;
} finally {
client.release(); // ALWAYS give the connection back
}
Either both updates happen or neither does. Keep transactions short, and never call an external API inside one — you'd hold a connection open while waiting on someone else's server.
23. Authentication
Authentication is "who are you?" — proving identity by logging in.
Authorisation is "are you allowed to do this?" — a different question, checked separately, and the one people forget.
Authentication is showing ID at the door. Authorisation is whether your ticket covers the VIP area.
Passwords
import bcrypt from "bcrypt";
const hash = await bcrypt.hash(password, 12); // 12 = cost factor
const ok = await bcrypt.compare(password, hash);
hashSync blocks your whole server.JWT — a signed pass
A JWT (JSON Web Token, said "jot") is a small piece of text holding some data (like userId) plus a signature. The signature is made with your secret key, so if anyone changes the data, the signature no longer matches. The client sends it on each request in the Authorization: Bearer <token> header.
A festival wristband. The staff can see at a glance it's genuine (signed) and still valid (not expired), without phoning the ticket office each time.
import jwt from "jsonwebtoken";
const token = jwt.sign({ userId: user.id }, process.env.JWT_SECRET, { expiresIn: "15m" });
const payload = jwt.verify(token, process.env.JWT_SECRET); // throws if tampered/expired
export function requireAuth(req, res, next) {
const header = req.headers.authorization;
if (!header?.startsWith("Bearer ")) {
return res.status(401).json({ error: "Not authenticated" });
}
try {
req.user = jwt.verify(header.slice(7), process.env.JWT_SECRET);
next();
} catch {
return res.status(401).json({ error: "Invalid or expired token" });
}
}
Sessions vs JWT
With a session, the server stores "who is logged in" (in Redis or a database) and gives the browser only a random ID in a cookie. With a JWT, the information travels inside the token, so the server doesn't need to store anything. Stateless means exactly that: the server keeps no memory between requests.
| Session (cookie + store) | JWT | |
|---|---|---|
| Where state lives | Your server (Redis/DB) | In the token itself |
| Per-request cost | A lookup | Just verify a signature |
| Log someone out now | Delete the record ✓ | Can't — valid until it expires |
| Scaling | Needs a shared store | Works anywhere |
The usual compromise: a short-lived access token (15 min, sent with every request) plus a long-lived refresh token stored server-side. When the access token expires, the client uses the refresh token to get a new one. Because the refresh token is stored on the server, you can revoke it (cancel it) — for example when a user logs out or a phone is stolen.
Authorisation is not authentication
// ✗ logged in, but is this THEIR post?
app.delete("/posts/:id", requireAuth, async (req, res) => {
await Post.deleteOne({ _id: req.params.id }); // anyone can delete anything!
});
// ✓ check ownership
const post = await Post.findById(req.params.id);
if (!post) return res.status(404).json({ error: "Not found" });
if (String(post.userId) !== req.user.userId) {
return res.status(403).json({ error: "Not allowed" });
}
This is one of the most common real vulnerabilities (security holes) — checking that someone is logged in, but never whether they own the thing. Example: user 5 changes the URL from /posts/100 (their post) to /posts/101 (someone else's) and deletes it.
24. File Uploads
An upload arrives as a stream of bytes in a special format called multipart/form-data, not as a normal JSON body. That's why you need a separate tool (multer) to handle it — express.json() ignores it.
Treat every uploaded file as hostile (possibly dangerous) until checked: the name, the size and the claimed type are all chosen by whoever sent it.
A post room opening parcels from strangers. You don't trust the label that says "books" — you weigh it, check it, and give it your own reference number.
import multer from "multer";
const upload = multer({
storage: multer.memoryStorage(),
limits: { fileSize: 5 * 1024 * 1024 }, // 5MB cap
fileFilter: (req, file, cb) => {
const allowed = ["image/jpeg", "image/png", "image/webp"];
cb(allowed.includes(file.mimetype) ? null : new Error("Invalid file type"),
allowed.includes(file.mimetype));
},
});
app.post("/avatar", requireAuth, upload.single("avatar"), async (req, res) => {
const url = await uploadToS3(req.file.buffer, req.file.mimetype);
res.json({ url });
});
The rules
- Always set a size limit — otherwise one upload can exhaust your memory or disk.
- Never trust the filename. Generate your own;
../../etc/passwdis a real attack. - Don't trust the reported type (the MIME type, like
image/png) either — it's chosen by the client. For anything sensitive, check the actual file signature: the first few bytes of a file, which reveal its real type (every PNG starts with the same bytes). - Store files elsewhere (S3, Cloudinary — online file storage services), not on the server's disk. Disks fill up and don't survive a redeploy.
25. Security
Nearly every web vulnerability comes down to one mistake: trusting input that came from outside.
The defences all follow the same shape:
- keep data and commands apart (so data can never be run as a command),
- check what each person is allowed to do,
- limit how often anyone can try.
A bank: cash and instructions go through different slots, each teller checks your account before handing anything over, and three wrong PINs lock the card.
The baseline
import helmet from "helmet";
import rateLimit from "express-rate-limit";
import cors from "cors";
app.use(helmet()); // sensible security headers
app.use(cors({ origin: "https://app.com", credentials: true })); // not "*"
app.use(express.json({ limit: "10kb" })); // reject giant bodies
app.use("/api/auth/login", rateLimit({
windowMs: 15 * 60 * 1000,
max: 5, // 5 login attempts per 15 min per IP
}));
The main risks, and what stops them
Two terms for the table: XSS (Cross-Site Scripting) is when an attacker gets their JavaScript to run on your site in someone else's browser — e.g. a comment containing <script>. Escaping means turning special characters like < into harmless text (<) so the browser shows them instead of running them.
| Attack | What it is | Defence |
|---|---|---|
| SQL injection | Input treated as database commands | Parameterised queries, always |
| NoSQL injection | An object like {$gt:""} sent where text was expected | Validate types before querying |
| XSS | Attacker's script runs in another user's browser | Escape on output; sanitise stored HTML |
| Broken access control | Reading or deleting someone else's data | Check ownership on every action |
| Brute force | Guessing passwords repeatedly | Rate limiting, account lockout |
| Secrets in git | Keys committed by accident | .env in .gitignore, rotate if leaked |
The NoSQL injection people miss
In MongoDB, $gt means "greater than". An empty string is the smallest possible string, so "password greater than empty" is true for every password. The attacker logs in without knowing it.
// Attacker sends: { "email": "a@b.com", "password": { "$gt": "" } }
await User.findOne({ email, password }); // $gt matches ANY password!
// ✓ validate that they're actually strings first
if (typeof email !== "string" || typeof password !== "string") {
return res.status(400).json({ error: "Invalid input" });
}
Other habits
npm auditregularly; keep dependencies current.- Run as a non-root user in Docker (root = the all-powerful admin account; if an attacker breaks in, they get fewer powers).
- HTTPS everywhere; set
secureandhttpOnlyon cookies. - Never log passwords, tokens or card numbers — logs get read by a lot of people.
- Don't reveal which part of a login was wrong: "invalid email or password", not "no such user" (that confirms which emails are registered).
26. Performance
Use all your CPU cores
A CPU core is one independent worker inside the processor; most servers have 2–16. One Node process uses one core for your JavaScript. A 4-core server running one process leaves three cores idle.
Clustering runs several copies of your app and shares incoming requests between them.
One checkout open in a supermarket with four tills. Opening all four tills serves four times as many customers — as long as each till doesn't need to know what the others are doing.
# Simplest: let a process manager do it
pm2 start server.js -i max # one process per core, auto-restart
# Or a container per core, behind a load balancer
A load balancer is a front door that spreads incoming requests across several copies of your app. This only works if your app is stateless — no session stored in a local variable, no uploads on local disk, no in-memory cache that has to be the same everywhere. Each process has its own memory and knows nothing about the others. Example: if user A logs in and the session is saved in process 1's memory, their next request might land on process 2, which has never heard of them.
Caching
const cached = await redis.get(`user:${id}`);
if (cached) return JSON.parse(cached);
const user = await db.getUser(id);
await redis.set(`user:${id}`, JSON.stringify(user), "EX", 300); // 5 min
return user;
A cache is a fast, temporary copy of data you'd otherwise have to fetch slowly. Redis is an in-memory database often used for this. Cache things that are read far more often than they change. When the real data is updated, delete the cached key — or accept that it will be stale (out of date) until the timer (here 5 minutes) runs out.
Other wins, roughly in order of value
- Database indexes — usually the single biggest improvement. An index is like the index at the back of a book: it lets the database jump to the right rows instead of reading every page. A missing index on a column you filter by means scanning every row.
- Fix N+1 queries (section 22).
- Paginate — never
SELECT *on a table that grows. - Compression —
app.use(compression()), often 70% smaller JSON. - Move slow work to a queue — emails, thumbnails, reports.
- Stream large responses instead of building them in memory.
clinic.js or --prof, or just time your database queries. Most "Node is slow" problems turn out to be one unindexed query, not the runtime.27. Logging & Debugging
Structured logging means recording facts as labelled fields rather than as a sentence, so you can search and filter them later.
"User 7 failed to pay" is unsearchable across a million lines. { userId: 7, event: "payment_failed" } lets you find every failure for that user instantly.
import pino from "pino";
const logger = pino();
// ✗ unsearchable
console.log("User " + id + " failed to pay");
// ✓ structured — you can filter by userId across millions of lines
logger.error({ userId: id, orderId, requestId: req.id }, "payment failed");
Request IDs
import { randomUUID } from "crypto";
app.use((req, res, next) => {
req.id = req.headers["x-request-id"] || randomUUID();
res.setHeader("x-request-id", req.id);
next();
});
Include req.id in every log line for that request. Without it, requests happening at the same time (concurrent requests) get their log lines mixed together, and you can't tell which lines belong to which request. With it, one search shows a single request's entire journey. A UUID is a long random ID that is practically guaranteed to be unique.
Log levels
- error — something broke and needs attention.
- warn — unusual but handled (retry succeeded, deprecated route used).
- info — significant events (server started, user registered).
- debug — detail, switched off in production.
Debugging
node --inspect server.js # attach Chrome DevTools or VS Code
node --watch server.js # restart on change, no nodemon needed
node --env-file=.env server.js # load env vars without dotenv
A real debugger beats console.log for anything non-trivial. You set a breakpoint (a line where the program pauses), then look at every variable and step through the code line by line.
28. Testing
A unit test checks one function on its own. An integration test checks several parts working together — for an API, usually a real request hitting a real route.
The most valuable tests cover the paths where things go wrong (the "unhappy paths"), because that's where bugs actually live.
Testing a car: a unit test checks the brakes on a test bench. An integration test drives the car round the block. You need both.
// Unit test — the service, no HTTP involved
test("rejects a duplicate email", async () => {
userRepo.findByEmail = vi.fn().mockResolvedValue({ id: 1 });
await expect(userService.create({ email: "a@b.com" }))
.rejects.toThrow("Email already registered");
});
// Integration test — the real route, with supertest
import request from "supertest";
import app from "../src/app.js";
test("POST /api/users creates a user", async () => {
const res = await request(app)
.post("/api/users")
.send({ name: "Ada", email: "ada@test.com", password: "secret12" });
expect(res.status).toBe(201);
expect(res.body.email).toBe("ada@test.com");
expect(res.body.password).toBeUndefined(); // never leak the hash
});
In the unit test, vi.fn().mockResolvedValue(…) creates a mock — a fake function that returns whatever you tell it, so the test doesn't touch a real database. Note that supertest needs no running server — it starts the app in memory. That's exactly why app.js and server.js are separate files (section 19).
What's worth testing
- Business rules — the service layer. Highest value, cheapest to run.
- The unhappy paths — invalid input, missing records, wrong user. Bugs live here, not in the happy path.
- Auth boundaries — can user A reach user B's data? Test it explicitly.
- A handful of end-to-end tests (tests that drive the whole system like a real user would) over critical flows (signup, checkout).
29. Deployment
Deploying means running your app somewhere permanent (a cloud server), with the settings and secrets supplied from outside the code.
The recurring theme is assume it will restart — at any moment, for any reason. Anything held only in the process's memory or on its local disk will be lost.
Docker packages your app and everything it needs into an image — a sealed box that runs the same way on any machine. A running image is called a container. A Dockerfile is the recipe for building the image.
# Dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev # install BEFORE copying source → better layer caching
COPY . .
USER node # don't run as root
EXPOSE 3000
CMD ["node", "server.js"]
The pre-launch checklist
NODE_ENV=production— Express skips debug work and caches views.- All secrets from environment variables, never in the repository.
- Graceful shutdown on SIGTERM (section 10 and Node.js Process Lifecycle).
- A
/healthendpoint for the load balancer to check. - A process manager or orchestrator that restarts on crash.
- Structured logs going somewhere you can search.
- Rate limiting and
helmetenabled. - Database has indexes and a backup schedule.
app.get("/health", (req, res) => res.json({ status: "ok", uptime: process.uptime() }));
// A deeper check — is the database actually reachable?
app.get("/ready", async (req, res) => {
try { await db.query("SELECT 1"); res.json({ ready: true }); }
catch { res.status(503).json({ ready: false }); }
});
The distinction matters: health means "the process is alive"; ready means "it can actually serve traffic". A container that's alive but can't reach the database should be taken out of rotation (the load balancer stops sending it requests for now), not restarted — restarting won't fix a database that's down.
Interview Q&A — quick answers
Node fundamentals
- What is Node.js? A runtime that lets JavaScript run outside the browser, built on V8 and libuv, with tools for files, networking and processes.
- Is Node single-threaded? Your JavaScript is — it runs on one thread. Underneath, libuv uses the operating system and a small thread pool, so I/O (files, network) happens in parallel.
- Blocking vs non-blocking? Blocking stops the single thread, freezing every request. Non-blocking hands the work off and continues.
- Why is Node bad at CPU work? There's one thread for your code. A long calculation can't be interrupted, so every other request waits.
- How do you handle CPU-heavy work? Worker threads, a background queue, or breaking the job into chunks that yield.
- Explain the event loop. Phases — timers, pending, poll, check, close — run in a cycle.
nextTickand promises drain between every phase, so they always beatsetTimeout. - setImmediate vs setTimeout(fn, 0)? Different phases. Inside an I/O callback
setImmediatealways runs first; at the top level the order isn't guaranteed. - process.nextTick vs Promise?
nextTickhas higher priority and runs first. Overusing it can starve the loop. - CommonJS vs ES Modules?
requireloads when the line runs;importis resolved up front, before any code runs. ES Modules also support top-level await and tree shaking (build tools dropping unused code). - What is an EventEmitter? Node's publish/subscribe tool (one side announces, any number of others listen) — emit a named event, and every registered listener runs. Streams and servers are built on it.
- What are streams for? Processing data in chunks so memory stays flat regardless of size. Essential for large files and uploads.
- Buffer? Raw binary data. Note
.lengthis bytes, not characters.
Express
- What is Express? A minimal layer over Node's HTTP server providing routing, middleware and helpers for requests and responses.
- What is middleware? A function
(req, res, next)that runs between the request and the handler. It can inspect, modify, end, or pass along. - Why does middleware order matter? Express runs them top to bottom. Auth after a route protects nothing; the error handler must be last.
- How does error middleware differ? It takes four arguments — that's how Express recognises it.
- How do you reach an error handler? Call
next(err). In Express 4, async handlers need a wrapper; Express 5 does it automatically. - params vs query vs body? URL segment (
/users/:id) vs after the?vs the request payload. All params and query values are strings. - "Cannot set headers after they are sent"? You sent two responses. Usually a missing
returnbeforeres.json(). - What's a Router? A group of routes you can attach (mount) under one URL prefix, so the prefix is defined in one place.
- Why does my request hang? A middleware neither called
next()nor sent a response. - 401 vs 403? Not authenticated vs authenticated but not permitted.
Lifecycles
- What happens when you run
node app.js? Node starts V8 and libuv, loads your modules (each file once, then cached), runs the top-level code, drainsnextTickand promise microtasks, then enters the event loop. It stays in the loop while anything is pending, and exits when nothing is. - Name the event loop phases in order. Timers → pending callbacks → idle/prepare → poll → check → close callbacks. Between every callback, the
nextTickqueue and then the promise queue are emptied. - Why doesn't a web server exit by itself? An open server (a listening socket) counts as pending work, so the loop never becomes empty. A script with only a
console.logexits straight away. 'beforeExit'vs'exit'?beforeExitfires when the loop is empty and you may still schedule async work (which keeps the process alive); it doesn't fire onprocess.exit()or crashes.exitalways fires, but only synchronous code runs inside it.- How do you shut down gracefully? Listen for SIGTERM and SIGINT, call
server.close()so no new requests come in, let in-flight ones finish, close the database and other connections, thenprocess.exit(0)— with a timeout that forcesexit(1)if clean-up hangs. - Walk through an Express request's lifecycle. Node's http server builds
req/res→ Express app → app-level middleware in order (helmet, cors,express.json, logger, auth) → router matches method + path → route middleware (validation) → handler → service/DB →res.json()ends it →resemits'finish'. No match → 404 handler.next(err)→ skips to the 4-argument error handler. Neithernext()nor a response → the request hangs.
Building and running APIs
- Why separate controller, service and repository? Each layer is ignorant of the others' concerns, so business rules are reusable and testable without HTTP, and swapping the database touches one file.
- Why validate on the server if the frontend already does? Anyone can call the API directly. Frontend validation is for user experience, not security.
- How do you prevent SQL injection? Parameterised queries — the value travels separately and is never parsed as SQL.
- What is the N+1 problem? One query for a list, then one per item. Fix with a join, eager loading, or one batched query.
- Why a connection pool? Connections are expensive to create and limited in number. A pool reuses a fixed set.
- How do you store passwords? Hashed with bcrypt or argon2 and a cost factor. Never reversible, never in logs.
- Sessions vs JWT? Server-side state you can revoke instantly vs stateless tokens that scale freely but can't be cancelled early.
- What is graceful shutdown? On SIGTERM: stop accepting new requests, finish current ones, close the database, then exit.
- How do you use multiple CPU cores? Cluster mode or multiple containers — only works if the app keeps no local state.
- Health vs readiness? "The process is alive" vs "it can actually serve traffic".
Learning Path
- JavaScript async first — promises,
async/await, error handling. Every Node concept builds on these. - Node basics — run a script, use
fsandpath, read environment variables, understand blocking vs non-blocking. - Build one HTTP server with no framework (section 11). Once. It makes everything Express does obvious.
- Express fundamentals — routing, middleware, request and response. Build a to-do API with in-memory data.
- Add a database — Postgres with a query builder, or MongoDB with Mongoose. Learn indexes early.
- Proper structure — split into routes, controllers, services, repositories. Add central error handling and validation.
- Authentication — register, log in, hash passwords, protect routes, and check ownership as well as identity.
- Project 1 — a complete REST API: full CRUD, auth, validation, pagination, consistent errors, and tests.
- Going further — file uploads to S3, Redis caching, background jobs with BullMQ, rate limiting.
- Project 2 — something with real pressure: a booking system (concurrency and transactions) or a job board (search and queues).
- Production skills — Docker, environment config, graceful shutdown, structured logging, health checks, and a real deploy.
- Then — TypeScript throughout, and one deeper area: WebSockets, microservices, or a faster framework like Fastify or NestJS.