Files and media

  • Lesson 19 of 40
  • 14 min
  • Intermediate
  • 13 steps

Photos and videos don't sit well in your database. Learn where to keep them and how to move them up and down without your servers carrying the weight.

DB growth (GB/day)
1
Uploads via the app
0%
Photo load
35 ms
Storage, month 12 (USD/mo)
5,240
people sharing photosAppserversAppmetadataObjectstorageObjectWorkersWorkersCDNCDNCleanupnightlyCleanupanyonephotos stored as BLOBs95%85%+2 TB a daykey · owner · size · typephotos/9f2c.jpg90%relaying 2 GB videos20%may I upload?pending → uploadedthe bytesupload completeOne presigned URLPUT…/videos/7a1e.mp4expires in900 s (15 min)good for1 key · 1 uploadsignaturemade with the app's keythe bytes never touch the app20%parts 1–200 → 1 objectMultipart uploadstartid + 200 URLsparts 1…20010 MB, 4 at a timepart 37 failsresend part 37completevideos/7a1e.mp4200 × 10 MB = 2 GB20%status: readysizes · no GPS · videojobs9f2c_w640.webp · 1 yearonly on a miss4 s segmentsOne video, five qualities1080p5 Mbit/s720p3 Mbit/s480p1.5 Mbit/s← 2 Mbit/s link360p0.8 Mbit/s240p0.4 Mbit/sexample bitrates312 pending > 1 daydelete orphansoriginals > 90 days → archiveClasses per TB-monthstandard≈ 23 USDcheap to readinfrequent≈ 12.5 USDpay per readarchive≈ 2 USDslow or costly readsmonth 12: 180 + 550 TB≈ 5,240 USDexample prices; originals onlypublic access blockedGET /?list
  • request
  • response
  • slow request
  • waiting in queue
  • cache miss

Step 1 of 13

Concept

Photos inside the database

Back to the photo app from Napkin math. Its first version stores each photo as a BLOB, a binary column, in the same row as its author, caption and date. Everything lives in one place and one backup covers it all.

With a thousand 2 MB uploads a day, the database grows by about 2 GB daily. While the app is small, nobody notices.

Step 2 of 13

Problem

Two terabytes every day

The app takes off: a million 2 MB photos a day is 2 TB a day, about 730 TB a year. Every replica and every backup grows just as fast; a restore takes days and a new replica has to copy all of it.

On top of that, every photo someone opens travels through the servers and the database. Images take the memory the small rows needed, and even loading comments slows down. In the example, a photo goes from 180 to 450 ms.

What changed

  • 1M uploads a day × 2 MB = 2 TB
  • Huge replicas and backups
  • Photos crowd out every other query

Step 3 of 13

Solution

Bytes to an object store, facts to the database

Move the bytes to object storage: a huge key → file store. You save a file with PUT photos/9f2c.jpg and get it back with GET. There are no queries and no joins, and total size is not a practical concern.

The service replicates every object across several data centres, and a terabyte costs far less than on database disks. The database keeps one ~1 KB row per photo with its key, owner, size and type: 1M rows × 1 KB = 1 GB a day.

What changed

  • Object storage for the bytes
  • One ~1 KB row per photo
  • Database growth 2000 → 1 GB a day

Step 4 of 13

Problem

Case: a 2 GB video from a phone

Files now live outside the database, but they still pass through your servers on the way to the store. Now the app accepts videos: uploading 2 GB from a phone takes minutes, and all that time a server holds a connection and memory just to forward bytes.

You end up sizing servers for bandwidth instead of for the logic they run. A wave of video uploads slows every other page down.

What changed

  • Video uploads of up to 2 GB
  • Each upload ties up a server for minutes

Step 5 of 13

Solution

Upload straight to the store

The app stops carrying bytes and only grants permission. The phone asks "may I upload a 2 GB video?"; the app checks who it is, creates a row in pending state and answers with a presigned URL: a link signed with the app's key that allows one upload, to one key, for the next 15 minutes.

The phone sends the bytes straight to object storage. When it finishes, the store sends an event and the app marks the row uploaded.

What changed

  • The app signs an upload URL
  • The phone uploads straight to the store
  • The store reports the finished upload

Step 6 of 13

Technique

Big files: upload them in parts

A single 2 GB request over a mobile network rarely arrives whole. A multipart upload splits the file into parts (200 of 10 MB in the example) that travel in parallel, each with its own presigned URL.

If a part fails, only that part is sent again; if the connection drops, the upload carries on from the last part that arrived. Once all are in, one final call joins them into a single object.

What changed

  • 2 GB → 200 parts of 10 MB
  • Each failed part is retried on its own
  • The upload resumes after a drop

Step 7 of 13

Technique

Build the versions people download

The original is almost never what you show. Each "upload complete" event goes onto a queue (more in Message queues) and workers prepare what the app needs: thumbnails and two or three sizes of each photo, a malware scan and, before anything is shared, removing the GPS location hidden in the photo's metadata.

Videos are transcoded into several resolutions. When a worker finishes, it saves the new files and the row moves to ready; until then the app shows "processing…". With smaller versions, a photo loads in 120 ms in the example.

What changed

  • Upload events → queue → workers
  • Thumbnails, sizes and transcoding
  • Location data removed, malware scanned

Step 8 of 13

Solution

Serve files from the edge

Far more photos are viewed than uploaded, so downloads must not go through the app either. A CDN (seen in CDNs and content delivery) uses object storage as its origin: the first person in a city to ask for a photo pulls it to the nearest edge, and everyone after gets it from there.

Give each version a key that changes when its content does, like photos/9f2c_w640.webp, and you can cache it for a year without it ever going stale. In the example, a photo goes from 120 to 35 ms.

What changed

  • CDN in front of object storage
  • Views served from the nearest edge
  • Photo load: 120 → 35 ms

Step 9 of 13

Technique

Video: let the player choose

A video is too heavy to download whole before watching, and the connection changes from second to second. So it is cut into segments a few seconds long (4 in the example), each encoded at several qualities from 240p to 1080p, with a small playlist file that lists them.

The player asks the CDN for one segment at a time and picks the quality of the next from how fast the last one arrived. On a 2 Mbit/s connection it takes 480p; enter a tunnel and it drops to 360p instead of freezing. This is adaptive bitrate streaming (HLS or DASH).

What changed

  • Each video: 4 s segments × 5 qualities
  • Quality picked segment by segment

Step 10 of 13

Problem

Case: rows with no file, files with no row

Two stores can disagree. A phone gets its URL and loses signal: the row stays pending forever. Someone deletes a photo: the row goes, but the files stay, still costing money and still open to anyone who kept the link.

Treat it like any change across two systems (more in Sagas): every row has a state; store events move it forward; a delete removes the row first and then queues the file deletions; and a nightly cleanup removes pending rows older than a day and files no row points to.

What changed

  • State on every row: pending → uploaded → ready
  • Deletes: row first, files queued
  • Nightly cleanup of orphans

Step 11 of 13

Trade-off

Old files can live somewhere cheaper

Most photos are viewed in their first few weeks and hardly ever after. Object stores offer storage classes: standard for data read often, an infrequent-access class, and archive classes that cost a fraction but charge more per read or take minutes to hours to return a file.

A lifecycle rule moves objects on its own: originals older than 90 days go to archive, while the sizes people still browse stay in standard. With example prices and counting originals only, month 12 holds 730 TB: all in standard costs 730 × 23 ≈ 16,790 USD a month; with the recent 180 TB in standard and 550 TB in archive, 4,140 + 1,100 = 5,240 USD.

What changed

  • Lifecycle: originals to archive after 90 days
  • Storage in month 12: 16,790 → 5,240 USD/month

Step 12 of 13

Problem

Case: the bucket anyone can list

Someone on the team makes the bucket public "so the images show up". Months later a stranger finds they can list it and download every photo ever uploaded, private ones and originals with their location included. Leaks like this are among the most common in the cloud.

Keep buckets private and block public access for the whole account. Serve files through the CDN with signed URLs, give each service a key that reaches only its own paths, and use unguessable names as a last defence, never the only one.

What changed

  • Bucket anyone can list
  • Public access blocked, CDN with signed URLs
  • A least-privilege key per service

Step 13 of 13

Summary

Recap

Bytes go to object storage and metadata to the database; the object key ties them together.

Clients upload directly with presigned URLs, and big files go in parts that can be retried and resumed.

Workers build thumbnails, sizes and transcodes, and strip what should not be shared.

Serve files through a CDN with versioned keys, and video in adaptive segments.

Keep rows and files in agreement, move old files to cheaper classes and keep buckets private.

You will learn

  • Bytes in object storage, metadata in the database
  • Direct uploads with presigned URLs, and big files in parts
  • Processing, CDN delivery and adaptive video
  • Keeping metadata and files in agreement
  • Storage classes, lifecycle rules and private buckets