Encryption and secrets

  • Lesson 34 of 40
  • 13 min
  • Intermediate
  • 12 steps

A working system is not enough: data crosses networks you do not own, sits on disks that get lost and depends on keys someone has to guard. Here is how to protect it at every point.

Encrypted in transit
yes
Encrypted at rest
yes
Secrets live in
a manager
Lucía · café Wi-FiWi-Fiattacker on the Wi-FiLoadbalancerLoad balancerAppAppCustomer DBKMSkeysKMSthiefSecretsmanagerSecrets managerPublicrepoPublic repoPaymentproviderPayment providerLucíaChatserverChat serverTomásreads: lucia / pass123http:// · password in clearrandom bytes onlycertificate: shop.examplehttps:// · encryptedTLSTLSThe TLS handshakethe server sends its certificatethe browser checks the CA signatureboth sides derive fresh session keysfrom here on: encrypted and untamperedcertificate expired 03:00“Your connection is not private”No surprise expiriesrenewed by hand by one personACME: automatic renewalalert 30 days before expirycheck every endpoint, not just wwwmTLS · own certificatemTLSmTLSrandom bytes onlybackup in a public bucketreads every customerbackup copyencrypted · AES-256ciphertext onlybackup copydecrypts the data keyEnvelope encryptiondataencrypted with the data keydata keyencrypted with the master keydisk stolenno usable keyevery useis auditeddata keys re-wrapped · v2master key v2re-wrapRotating keysmaster key v2 in the KMSre-wrap each data key: KBthe data is not re-encryptednew writes use new keysAPI key in the codepay_live_… in historykey found in 3 mingit pushno secrets in the coderotated every 30 dayssecret scan: cleanreads at startupgit pushholds the real numberscard: tok_8d1enever stores card numberssteals tokens · uselesscharges tok_8d1ebackup copyencrypts with Tomás’s public keysees ciphertext onlydecrypts with its private keyTLSmTLSmTLS
  • request
  • response
  • handshake
  • failed request
  • replication

Step 1 of 12

Problem

Unencrypted HTTP on public Wi-Fi

Lucía signs in to the shop from a café’s Wi-Fi. The site uses http://, so her username, her password and the session cookie travel as readable text: first through the café’s router, then through every network between it and the server.

An attacker on the same Wi-Fi captures the traffic and reads it as is. Since nothing protects it, they could also modify the page on its way back to Lucía, for example to slip in a script.

Step 2 of 12

Solution

TLS encrypts the path

With https://, TLS encrypts everything between Lucía’s browser and the load balancer. In the handshake, the server presents a certificate signed by a certificate authority (CA) the browser trusts: that proves it really is shop.example.

Then both sides agree on fresh session keys. The attacker still copies every packet, but sees only random-looking bytes, and any change they try to make is detected.

What changed

  • TLS certificate on the load balancer
  • The attacker only sees ciphertext

Step 3 of 12

Problem

Case: the certificate expired at 3 a.m.

Every certificate has an expiry date. This one was renewed by hand by someone who has left the team. At 03:00 it expires: every browser shows a full-page security warning and almost nobody clicks through.

An outage caused by a date. The fix: automatic renewal with ACME (from a free CA or your cloud’s certificate manager) and an alert 30 days before any certificate expires.

What changed

  • Certificate expired at 03:00
  • Browsers block the site
  • Fix: automatic renewal + alerts

Step 4 of 12

Technique

Encrypt inside the data center too

The load balancer terminates TLS: it decrypts the traffic and, traditionally, forwarded it as plain text to the app and the database, trusting the internal network. One compromised machine in there is enough to read everything.

Modern systems treat no network as safe (zero trust): services talk over mutual TLS (mTLS), where both ends present a certificate. Every hop is encrypted and every caller is identified.

What changed

  • Load balancer → app → DB over mutual TLS
  • Every service has its own certificate

Step 5 of 12

Problem

The stolen backup

Everything on the network is protected now. Then a database backup ends up in a storage bucket someone made public by mistake, and a stranger downloads it.

Encryption in transit does nothing here: the data on disk (every customer’s name, address and order history) reads as is.

What changed

  • Backup in a public bucket
  • Copied and readable

Step 6 of 12

Solution

Encrypt at rest, keep the keys elsewhere

Encrypt the disks, the database files and the backups. The delicate part is the key: if it lives next to the data, the thief takes both. So keys live in a separate service, the KMS (key management service), which never hands out its master key.

With envelope encryption, the data is encrypted with a data key, and that data key is stored encrypted with the KMS master key. To read, you ask the KMS to decrypt it; every use is logged and access can be revoked.

What changed

  • Data and backups encrypted (AES-256)
  • Keys held in a KMS
  • The stolen copy cannot be read

Step 7 of 12

Technique

Rotating keys without re-encrypting everything

Keys should change every so often, and right away if one may have leaked. Re-encrypting terabytes of data would take days.

Envelope encryption makes it cheap: create a new master key in the KMS and re-wrap each data key with it. That is kilobytes, not terabytes: the data is not touched, and new writes use new data keys.

What changed

  • Master key v2 in the KMS
  • Data keys re-wrapped (KB, not TB)
  • The data is not touched

Step 8 of 12

Problem

Case: a secret in the code

To test something quickly, a developer writes the payment provider’s API key straight into the code, and the repository is public. Bots scan public repositories nonstop: the key is found and used within minutes (3 in this example).

Deleting the line fixes nothing: it stays in the git history forever. The only way out is to revoke the key and issue a new one.

What changed

  • API key pushed to a public repo
  • A bot finds it within minutes

Step 9 of 12

Solution

A secrets manager

Take secrets out of the code entirely. The app asks a secrets manager for them at startup and proves its own identity to receive only the secrets it needs.

Secrets are rotated often (every 30 days in this example) or issued short-lived on demand, and a scanner in CI blocks any commit that looks like it holds a key.

What changed

  • Secrets manager
  • The app reads its secrets at startup
  • Automatic rotation · secret scanning in CI

Step 10 of 12

Technique

Case: card numbers, better tokenized

Card numbers are some of the most valuable data you could hold, and every system that stores, processes or transmits them must follow strict rules (PCI DSS). Encrypting them still means holding them.

Better not to hold them. The card goes to the payment provider, which stores it and returns a token like tok_8d1e. Your database, your logs and your analytics only ever see tokens, which are worthless to a thief. With the provider’s hosted form, the number never even passes through your server.

What changed

  • The payment provider holds the real numbers
  • The database stores tokens only
  • Card numbers in your systems

Step 11 of 12

Technique

End-to-end: not even the server can read it

Private messages can go further. With end-to-end encryption (E2E), Lucía’s device encrypts each message with Tomás’s public key, and only Tomás’s device holds the private key to decrypt it.

The chat server stores and forwards ciphertext. If it is breached, or someone demands the messages, there is nothing readable to hand over.

That is the idea in one line. Real protocols use the key pairs to agree on fresh secret keys over and over, so a key stolen later cannot open past conversations (forward secrecy).

What changed

  • Messages encrypted on the device
  • The server only forwards ciphertext

Step 12 of 12

Summary

Recap

TLS on everything coming from outside; mTLS between services inside.

Encrypt at rest and keep the keys apart, in a KMS.

Automate certificate renewal; rotate keys by re-wrapping the data keys.

Keep secrets in a secrets manager, rotate them and scan for leaks.

Tokenize what you would rather not hold, like card numbers.

End-to-end encryption when not even the server should read the data.

You will learn

  • TLS, certificates and how to renew them
  • Encryption inside the data center and at rest
  • Rotating keys
  • Getting secrets out of the code
  • Tokenization and end-to-end encryption