What would be the best design to handle batch processing with Messenger ?
workflow : User fill a form in my backoffice to ask information about a bunch of product (let’s say 50 identifiers)
At this point i can dispatch an event with identifiers in envelope and return status pending to user (with jobId) to perform my background operations.
Each task need several third party API calls (up to 10) to get a full response, so i can dispatch an event for each one and perform operations aysnc with scaled workers, messenger is perfect for this and already handle failures.
But how to keep track in master process of each subtask progress needed to update job status to completed and notify user ?
Should i persist each subtasks results (eg in database) and add a cron task to check if everything is finished ? or there is a better way to perform this only with messenger component ?
Source: Symfony Questions