Optimize IMAP Email Fetching

Замовник: AI | Опубліковано: 11.01.2026

I run a service that signs in to multiple Gmail accounts over IMAP, pulls every message, then serves the data through our own REST API. Under load we are now running into two show-stoppers: first, the API stalls or times out; second, the worker that talks to Gmail crashes whenever the IMAP queue spikes, which in turn cascades into more API failures. What I need is a backend engineer who can dig into the existing Node-based IMAP polling module (built on node-imap) and make the whole path—from Gmail connection, through message parsing, to JSON delivery—stable and fast. The biggest pain point is the API response layer: responses are delayed or never arrive when the fetcher crashes, so tightening that hand-off and introducing solid retry / back-pressure logic is top priority. Key facts you should know • All email access must remain IMAP-only; no Gmail API or third-party gateways. • Crashes happen during email fetching; once data is in memory, the rest of the pipeline behaves. • We are not memory-bound—errors are almost always socket timeouts, rate-limit pauses, or unhandled disconnects. Deliverables 1. Refactored IMAP fetching code with reconnection, exponential back-off, and graceful shutdown. 2. Optimised API response handler that can queue or stream results so upstream callers never see 5xx timeouts. 3. A short README outlining configuration flags, environment variables, and how to reproduce the original failure versus the fixed behaviour. To accept the work I will spin up our staging environment, aim 50k messages at it, and expect zero crashes plus median API latency under 300 ms. If that sounds like a puzzle you can solve, let’s talk.