Why memory usage grows
A common PHP mistake is reading an entire file into a string, decoding it into a large array and then creating another copy while transforming it. Peak memory can be several times larger than the original file. Large CSV imports, archive processing and report exports should be designed around bounded memory.
Prefer streams
PHP stream functions allow data to be processed incrementally. For a CSV import, read one row at a time and persist bounded batches.
$handle = fopen($path, 'rb');
if ($handle === false) {
throw new RuntimeException('Unable to open input file.');
}
try {
$batch = [];
while (($row = fgetcsv($handle)) !== false) {
$batch[] = validateRow($row);
if (count($batch) >= 500) {
$repository->insertBatch($batch);
$batch = [];
}
}
if ($batch !== []) {
$repository->insertBatch($batch);
}
} finally {
fclose($handle);
}
The batch size should be measured against row size, database limits and available memory. Validation must happen before data is written.
Stream exports
For large downloads, write output incrementally rather than building one huge string. Set appropriate headers before output begins, and avoid debug output that corrupts the file. If an export can take a long time, generate it as a background job and provide a controlled download when complete.
Enforce limits
Set maximum upload sizes, execution time policies, row limits and archive expansion limits. Validate file type and content; a filename extension alone is not proof of format. Store uploaded source files outside the public document root.
Recovery and observability
Track job progress by rows or byte offsets where possible. Make imports resumable only when the processing design can safely detect already-processed records. Log counts and failure reasons without recording sensitive file contents.
Checklist
- Use streams for large inputs and outputs.
- Process bounded batches.
- Validate rows before persistence.
- Enforce upload, row and archive limits.
- Keep temporary files private and clean them up.
- Make long-running work resumable where practical.