How can I efficiently bulk copy 500-1000 candidate records without user delay

I'm working on a feature where users can perform a bulk copy of 500 to 1000 candidate records. Previously, we supported copying up to 50 candidates, and even that took around 5 minutes to complete. Now, we need to scale this to handle 500 to 1000 candidates, without making the user wait on the screen.

Functionality Overview

The user selects 500 to 1000 candidates from the UI.

The front-end sends a payload containing the selected candidate IDs.

For each candidate ID, we:

• Fetch the full candidate data from another database
• Run some validations
• Trigger SMS notification
• Use an existing method called AddCandidate() to save the candidate to the target database, including some modifications like updating JobOrderId

Constraints: the process must not block the user or require them to stay on the screen.

We are using Angular (front-end) and an ASP.NET Core Web API (back-end).

I need approaches for this problem statement that how to handle the large amount of data.

Continue reading...