How would you design a file processing system?

Short explanation:

A file-processing service accepts an upload and transforms it asynchronously, such as extracting data or generating a report. Object storage holds the file, a durable job references it, and workers perform bounded processing. Keeping bytes out of the request server's memory improves stability for large uploads.

Key points:

• Upload to object storage with size and access limits.

• Queue a job referencing the object; workers validate and process it in chunks.

• Save progress/results, clean partial outputs and make retries safe.

• Track upload, queued, running, completed and failed states so the client can poll or receive progress updates.

• Restrict file types and sizes, isolate risky processing, and make output names and cleanup safe across retries.

Interview point:

• Explain what happens if the worker fails after writing half the output and how a retry identifies valid versus incomplete results.