private void submitTasks(ListtasksList) { final ThreadPoolExecutor executor = new ThreadPoolExecutor(MIN_POOL_SIZE, MAX_POOL_SIZE, 60L, TimeUnit.SECONDS, new LinkedBlockingQueue ()); for (Runnable task : tasksList) { while (executor.getQueue().size() > MAX_Q_SIZE) { // Throttle for WAIT_FOR_SECONDS seconds if queue is full try { log.info("Waiting " + WAIT_FOR_SECONDS + " seconds as Queue size is: " + executor.getQueue().size()); Thread.sleep(WAIT_FOR_SECONDS * 1000); } catch (InterruptedException e) { // Ignore } } executor.execute(task); } // inform the executor there are no more tasks executor.shutdown(); }
Instead of using the factory to get a pre-configured executor, I create my own ThreadPoolExecutor using a LinkedBlockingQueue. This allows us access to the underlying queue for the throttling feature. Before submitting the task, I check if the current queue size is greater than a certain size and if so, then wait for a configured time repeatedly till the queue is smaller.
This implementation is very useful for the scenarios where you have lots of short running tasks that need to be processed by a fixed small number of threads.
1 comment:
I'm wondering whether either base Java (which I doubt, because I checked) or somewhere else reliable (e.g. Apache Commons) provides this, or if I have to write my own. Preferably something lightweight. I don't mind writing it myself, but if there's a "standard" version out there somewhere I'd at least like to look at it first. Thanks ~ Michael from qa and software testing
Post a Comment