Design Reliable File Upload Pipelines
Overview
Build upload handling around size limits, generated filenames, content inspection, private storage, and safe download delivery.
Enforce limits before processing
Set request, upload, and application-level size limits. Validate the number of files, total request size, allowed business use, and expected file count. Treat the client filename and MIME type as untrusted metadata. Inspect file signatures where appropriate and use an allowlist of formats the application genuinely supports.
Store files outside executable web paths
Generate server-side identifiers rather than trusting supplied filenames. Store originals in a non-public directory or object storage bucket with private access. Keep metadata such as owner, size, detected type, checksum, and processing state in the database. Do not allow uploaded files to execute as scripts.
Process asynchronously when needed
Large image, document, or archive processing can exceed request limits. Move expensive work to a background job with bounded memory, CPU, and time. Apply archive extraction limits to prevent path traversal and decompression bombs. Quarantine files until scanning or validation completes when the threat model requires it.
Deliver files safely
Authorize every download against the owning user or intended public policy. Set a safe Content-Disposition and an appropriate Content-Type; use X-Content-Type-Options: nosniff. Avoid serving untrusted active content from the main application origin. Log processing outcomes without storing sensitive file contents in logs.
Implementation checklist
- Define the expected behavior and supported environments.
- Keep security, accessibility, and failure states in the design.
- Measure the result with representative user journeys.
- Document operational assumptions and review them after changes.