Testing
- Lesson 35 of 40
- 12 min
- Beginner
- 11 steps
Testing by hand is slow and only covers what someone remembers to check. A good test suite tells you within minutes, on every change, whether you broke something.
- Feedback in
- 20 s → 12 min
- Tests cover
- whole system
- Bugs reaching users
- rare
- request
- response
- slow request
- failed request
Step 1 of 11
Concept
Does this change break anything?
The checkout service totals each purchase and applies coupons. It stores orders in its database, charges through an external payment provider and asks the Inventory service, owned by another team, whether items are in stock.
Someone has just changed how discounts are rounded. To check it, the team makes a couple of purchases by hand on a test server. Everything looks right, and it ships.
Step 2 of 11
Problem
Manual checks leave gaps
Nobody tried a cart of 3 items at $4.99 with a 15% coupon. The discount is 14.97 × 0.15 = 2.2455, which should become $2.25; the new code truncates it to $2.24 and charges $12.73 instead of $12.72. One cent too much on thousands of orders. Every figure in this lesson is an example value.
Because manual checks are slow, they happen rarely and only cover the cases someone thought of. Customers found this bug.
What changed
- The rounding bug reaches production
- Customers find it, not the team
Step 3 of 11
Solution
Unit tests: one piece in isolation
A unit test calls a single function with known inputs and compares the result: here, total(cart, "OFF15") must return $12.72. It touches no database and no network, so it takes about a millisecond.
They are cheap enough to keep thousands and run them on every change: 3,000 tests in 20 s. The rounding case is now written down and fails the moment someone breaks it again.
What changed
- Tests call checkout directly
- No real database, payments or Inventory
Step 4 of 11
Technique
Test doubles for everything else
Checkout cannot run without its dependencies, so tests replace them with doubles: a fake in-memory database, a payment stub that always answers “approved”, and a mock that records whether the card was charged exactly once.
The catch: a double only knows what you told it. If the real provider renames a field, every unit test stays green.
What changed
- Fake db, payment stub and stock stub
Step 5 of 11
Solution
Integration tests: real neighbours
An integration test runs checkout against a real database, created from scratch in a container for each run, and checks what actually ends up in the tables. SQL mistakes, broken migrations and mismatched types show up here instead of in production.
Payments stay simulated (you cannot charge real cards), though a separate, smaller group of tests can call the provider’s sandbox. That is 250 tests in 5 min.
What changed
- Fake db → real db in a container
- Tests take seconds, not milliseconds
Step 6 of 11
Technique
Contract tests between teams
Inventory belongs to another team, who deploy whenever they like. If they rename the field units to quantity, checkout’s stub never notices and the tests keep passing.
A contract test writes down what checkout expects (this request, a response with these fields), and Inventory’s pipeline checks every change against that contract. An incompatible change fails in their build, before it ships.
What changed
- What checkout expects, saved as a contract
- Inventory verifies it on every change
Step 7 of 11
Solution
End-to-end: the whole journey
An end-to-end test uses the real system the way a user would: it opens the shop in an automated browser, adds three items, applies the coupon, pays with a sandbox card and waits for the confirmation.
It is the only test that proves everything works together, and also the slowest and most fragile. About 15 tests for the journeys that bring in revenue are enough: 12 min per run.
What changed
- A test browser walks through the whole system
- Minutes per run
Step 8 of 11
Concept
The test pyramid
Together they form the test pyramid: thousands of unit tests at the base, hundreds of integration and contract tests in the middle and a few end-to-end tests at the top.
Lower layers are fast, precise and cheap to run on every change; upper ones are slow but closer to what a user does. Push every check down to the lowest layer that can catch it.
Step 9 of 11
Problem
Case: the flaky test
One end-to-end test fails once every 20 runs. It waits a fixed 2 seconds for the confirmation email, and sometimes the email takes 3. The team gets used to hitting “retry”, and when a real bug turns it red, they retry that too.
A flaky test is worse than no test at all: it teaches the team to ignore failures.
What changed
- Fails 1 run in 20 for no reason
- Real failures get retried and ignored
Step 10 of 11
Technique
Load tests and failure tests
Working is not enough. A load test sends realistic traffic to a copy of the system (staging) and finds the point where latency starts to climb, before your users do. Here, at 6 times normal traffic, p99 shoots up from about 4.5 times onwards.
Failure tests break things on purpose (stop a replica, make the payment provider take 5 s) and check that timeouts, retries and failover behave as designed.
What changed
- Load test: 6× normal traffic on staging
- Failure test: payments slowed to 5 s
Step 11 of 11
Summary
Recap
Unit tests check one piece with doubles for the rest: fast enough to run by the thousand on every change.
Integration tests use real neighbours, such as a database in a container, to catch what doubles cannot see.
Contract tests catch incompatible API changes between teams, in the provider’s build.
A few end-to-end tests cover the key journeys: keep the pyramid the right way up.
Fix flaky tests, and test load and failure as well, not just whether it works.
← → step · R replay · Space pause
You will learn
- Unit tests and test doubles (covered)
- Integration and contract tests (covered)
- End-to-end tests and the test pyramid (covered)
- Flaky tests, load tests and failure tests (covered)