We have a platform running with Symfony where we need to execute a lot of heavy background jobs daily.
We currently use a bundle which handles our jobs and their execution.
With the growing user base, it has started to let us down.
We are getting more and more jobs that are stuck, hung, etc…
To solve this, i have developed a new bundle that offers a batch processing system with the Docker engine.
All jobs are executed separately in docker containers.
This way is more viable as Docker handles the execution of the started containers.
The bundle also offers a Monitoring user interface which you may chose to activate.
With this interface, you can:
— get an overview of your jobs
— inspect jobs
— stop jobs
— get usage info for each running job
If you have similar problems, check out the bundle to see if it’ll solve them.
Github repo: https://github.com/polkovnik-z/docker-jobs-bundle