Package org.eclipse.collections.impl.parallel

Class ParallelIterate

java.lang.Object
org.eclipse.collections.impl.parallel.ParallelIterate
public final class ParallelIterate
extends Object
The ParallelIterate class contains several parallel algorithms that work with Collections. All of the higher level parallel algorithms depend on the basic parallel algorithm named forEach. The forEach algorithm employs a batching fork and join approach.

All Collections that are not either a RandomAccess or List are first converted to a Java array using Iterate.toArray(Iterable), and then run with one of the ParallelArrayIterate.forEach methods.

See Also:
ParallelArrayIterate

  • Method Details

    • forEachWithIndex

      public static <T> void forEachWithIndex​(Iterable<T> iterable, ObjectIntProcedure<? super T> objectIntProcedure)
      Iterate over the collection specified, in parallel batches using default runtime parameter values. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared.

      e.g. 

       final Map<Integer, Object> chm = new ConcurrentHashMap<Integer, Object>();
 ParallelIterate.forEachWithIndex(collection, new ObjectIntProcedure()
 {
     public void value(Object object, int index)
     {
         chm.put(index, object);
     }
 });

    • forEachWithIndex

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable, BT procedure, Executor executor)
      Iterate over the collection specified in parallel batches using the default runtime parameters. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared. The code is executed against the specified executor. 
      e.g.
 final Map<Integer, Object> chm = new ConcurrentHashMap<Integer, Object>();
 ParallelIterate.forEachWithIndex(collection, new ObjectIntProcedure()
 {
     public void value(Object object, int index)
     {
         chm.put(index, object);
     }
 }, executor);
      Parameters:
      executor - Use this executor for all execution.

    • forEachWithIndex

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable, BT procedure, int minForkSize, int taskCount)
      Iterate over the collection specified in parallel batches. The ObjectIntProcedure used must be stateless, or use concurrent aware objects if they are to be shared. The specified minimum fork size and task count are used instead of the default values.
      Parameters:
      minForkSize - Only run in parallel if input collection is longer than this.
      taskCount - How many parallel tasks to submit to the executor.
      See Also:
      forEachWithIndex(Iterable, ObjectIntProcedure)

    • forEachWithIndex

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable, ObjectIntProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, Executor executor)

    • forEachWithIndex

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable, ObjectIntProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount)

    • forEachWithIndex

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndex​(Iterable<T> iterable, ObjectIntProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount, Executor executor)

    • forEachWithIndexInListOnExecutor

      public static <T,​ BT extends ObjectIntProcedure<? super T>> void forEachWithIndexInListOnExecutor​(List<T> list, ObjectIntProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount, Executor executor)

    • forEach

      public static <T> void forEach​(Iterable<T> iterable, Procedure<? super T> procedure)
      Iterate over the collection specified in parallel batches using default runtime parameter values. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.

      e.g. 

       final Map<Object, Boolean> chm = new ConcurrentHashMap<Object, Boolean>();
 ParallelIterate.forEach(collection, new Procedure()
 {
     public void value(Object object)
     {
         chm.put(object, Boolean.TRUE);
     }
 });

    • forEach

      public static <T> void forEach​(Iterable<T> iterable, Procedure<? super T> procedure, int batchSize)
      Iterate over the collection specified in parallel batches using default runtime parameter values. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.

      e.g. 

       final Map<Object, Boolean> chm = new ConcurrentHashMap<Object, Boolean>();
 ParallelIterate.forEachBatchSize(collection, new Procedure()
 {
     public void value(Object object)
     {
         chm.put(object, Boolean.TRUE);
     }
 }, 100);

    • forEach

      public static <T> void forEach​(Iterable<T> iterable, Procedure<? super T> procedure, int batchSize, Executor executor)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, BT procedure, Executor executor)
      Iterate over the collection specified in parallel batches using default runtime parameter values and the specified executor. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.
      Parameters:
      executor - Use this executor for all execution.
      See Also:
      forEach(Iterable, Procedure)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, BT procedure, int minForkSize, int taskCount)
      Iterate over the collection specified in parallel batches using the specified minimum fork and task count sizes. The Procedure used must be stateless, or use concurrent aware objects if they are to be shared.
      Parameters:
      minForkSize - Only run in parallel if input collection is longer than this.
      taskCount - How many parallel tasks to submit to the executor. TODO: How does the taskCount relate to the number of threads in the executor?
      See Also:
      forEach(Iterable, Procedure)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, BT procedure, int minForkSize, int taskCount, Executor executor)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, Executor executor)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int batchSize)
      Iterate over the collection specified in parallel batches using the default values for the task size. The ProcedureFactory can create stateful closures that will be collected and combined using the specified Combiner. 
      e.g. The ParallelIterate.select() implementation

 CollectionCombiner<T, SelectProcedure<T>> combiner = CollectionCombiner.forSelect(collection);
 ParallelIterate.forEach(collection, new SelectProcedureFactory<T>(predicate, taskSize), combiner, 1000);

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int batchSize, Executor executor)

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount)
      Iterate over the collection specified in parallel batches using the default values for the task size. The ProcedureFactory can create stateful closures that will be collected and combined using the specified Combiner. 
      e.g. The ParallelIterate.select() implementation

 int taskCount = Math.max(DEFAULT_PARALLEL_TASK_COUNT, collection.size() / DEFAULT_MIN_FORK_SIZE);
 final int taskSize = collection.size() / taskCount / 2;
 CollectionCombiner<T, SelectProcedure<T>> combiner = CollectionCombiner.forSelect(collection);
 ParallelIterate.forEach(collection, new SelectProcedureFactory<T>(predicate, taskSize), combiner, DEFAULT_MIN_FORK_SIZE, taskCount);

    • forEach

      public static <T,​ BT extends Procedure<? super T>> void forEach​(Iterable<T> iterable, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount, Executor executor)

    • forEachInListOnExecutor

      public static <T,​ BT extends Procedure<? super T>> void forEachInListOnExecutor​(List<T> list, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount, Executor executor)

    • forEachInBatchWithExecutor

      public static <T,​ BT extends Procedure<? super T>> void forEachInBatchWithExecutor​(BatchIterable<T> set, ProcedureFactory<BT> procedureFactory, Combiner<BT> combiner, int minForkSize, int taskCount, Executor executor)

    • select

      public static <T> Collection<T> select​(Iterable<T> iterable, Predicate<? super T> predicate)
      Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches.
      Returns:
      The selected elements. The Collection will be of the same type as the input (List or Set) and will be in the same order as the input (if it is an ordered collection).
      See Also:
      select(Iterable, Predicate, boolean)

    • select

      public static <T> Collection<T> select​(Iterable<T> iterable, Predicate<? super T> predicate, boolean allowReorderedResult)
      Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and with a potentially reordered result.
      Parameters:
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The selected elements. The Collection will be of the same type (List or Set) as the input.

    • select

      public static <T,​ R extends Collection<T>> R select​(Iterable<T> iterable, Predicate<? super T> predicate, R target, boolean allowReorderedResult)
      Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
      Parameters:
      target - Where to write the output.
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The 'target' collection, with the selected elements added.

    • select

      public static <T,​ R extends Collection<T>> R select​(Iterable<T> iterable, Predicate<? super T> predicate, R target, int batchSize, Executor executor, boolean allowReorderedResult)
      Same effect as Iterate.select(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
      Parameters:
      target - Where to write the output.
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The 'target' collection, with the selected elements added.

    • reject

      public static <T> Collection<T> reject​(Iterable<T> iterable, Predicate<? super T> predicate)
      Same effect as Iterate.reject(Iterable, Predicate), but executed in parallel batches.
      Returns:
      The rejected elements. The Collection will be of the same type as the input (List or Set) and will be in the same order as the input (if it is an ordered collection).
      See Also:
      reject(Iterable, Predicate, boolean)

    • reject

      public static <T> Collection<T> reject​(Iterable<T> iterable, Predicate<? super T> predicate, boolean allowReorderedResult)
      Same effect as Iterate.reject(Iterable, Predicate), but executed in parallel batches, and with a potentially reordered result.
      Parameters:
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The rejected elements. The Collection will be of the same type (List or Set) as the input.

    • reject

      public static <T,​ R extends Collection<T>> R reject​(Iterable<T> iterable, Predicate<? super T> predicate, R target, boolean allowReorderedResult)
      Same effect as Iterate.reject(Iterable, Predicate), but executed in parallel batches, and writing output into the specified collection.
      Parameters:
      target - Where to write the output.
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The 'target' collection, with the rejected elements added.

    • reject

      public static <T,​ R extends Collection<T>> R reject​(Iterable<T> iterable, Predicate<? super T> predicate, R target, int batchSize, Executor executor, boolean allowReorderedResult)

    • count

      public static <T> int count​(Iterable<T> iterable, Predicate<? super T> predicate)
      Same effect as Iterate.count(Iterable, Predicate), but executed in parallel batches.
      Returns:
      The number of elements which satisfy the predicate.

    • count

      public static <T> int count​(Iterable<T> iterable, Predicate<? super T> predicate, int batchSize, Executor executor)
      Same effect as Iterate.count(Iterable, Predicate), but executed in parallel batches.
      Returns:
      The number of elements which satisfy the predicate.

    • collect

      public static <T,​ V> Collection<V> collect​(Iterable<T> iterable, Function<? super T,​V> function)
      Same effect as Iterate.collect(Iterable, Function), but executed in parallel batches.
      Returns:
      The collected elements. The Collection will be of the same type as the input (List or Set) and will be in the same order as the input (if it is an ordered collection).
      See Also:
      collect(Iterable, Function, boolean)

    • collect

      public static <T,​ V> Collection<V> collect​(Iterable<T> iterable, Function<? super T,​V> function, boolean allowReorderedResult)
      Same effect as Iterate.collect(Iterable, Function), but executed in parallel batches, and with potentially reordered result.
      Parameters:
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The collected elements. The Collection will be of the same type (List or Set) as the input.

    • collect

      public static <T,​ V,​ R extends Collection<V>> R collect​(Iterable<T> iterable, Function<? super T,​V> function, R target, boolean allowReorderedResult)
      Same effect as Iterate.collect(Iterable, Function), but executed in parallel batches, and writing output into the specified collection.
      Parameters:
      target - Where to write the output.
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The 'target' collection, with the collected elements added.

    • collect

      public static <T,​ V,​ R extends Collection<V>> R collect​(Iterable<T> iterable, Function<? super T,​V> function, R target, int batchSize, Executor executor, boolean allowReorderedResult)

    • flatCollect

      public static <T,​ V> Collection<V> flatCollect​(Iterable<T> iterable, Function<? super T,​Collection<V>> function)

    • flatCollect

      public static <T,​ V> Collection<V> flatCollect​(Iterable<T> iterable, Function<? super T,​Collection<V>> function, boolean allowReorderedResult)

    • flatCollect

      public static <T,​ V,​ R extends Collection<V>> R flatCollect​(Iterable<T> iterable, Function<? super T,​Collection<V>> function, R target, boolean allowReorderedResult)

    • flatCollect

      public static <T,​ V,​ R extends Collection<V>> R flatCollect​(Iterable<T> iterable, Function<? super T,​Collection<V>> function, R target, int batchSize, Executor executor, boolean allowReorderedResult)

    • collectIf

      public static <T,​ V> Collection<V> collectIf​(Iterable<T> iterable, Predicate<? super T> predicate, Function<? super T,​V> function)
      Same effect as Iterate.collectIf(Iterable, Predicate, Function), but executed in parallel batches.
      Returns:
      The collected elements. The Collection will be of the same type as the input (List or Set) and will be in the same order as the input (if it is an ordered collection).
      See Also:
      collectIf(Iterable, Predicate, Function, boolean)

    • collectIf

      public static <T,​ V> Collection<V> collectIf​(Iterable<T> iterable, Predicate<? super T> predicate, Function<? super T,​V> function, boolean allowReorderedResult)
      Same effect as Iterate.collectIf(Iterable, Predicate, Function), but executed in parallel batches, and with potentially reordered results.
      Parameters:
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The collected elements. The Collection will be of the same type as the input (List or Set)

    • collectIf

      public static <T,​ V,​ R extends Collection<V>> R collectIf​(Iterable<T> iterable, Predicate<? super T> predicate, Function<? super T,​V> function, R target, boolean allowReorderedResult)
      Same effect as Iterate.collectIf(Iterable, Predicate, Function), but executed in parallel batches, and writing output into the specified collection.
      Parameters:
      target - Where to write the output.
      allowReorderedResult - If the result can be in a different order. Allowing reordering may yield faster execution.
      Returns:
      The 'target' collection, with the collected elements added.

    • collectIf

      public static <T,​ V,​ R extends Collection<V>> R collectIf​(Iterable<T> iterable, Predicate<? super T> predicate, Function<? super T,​V> function, R target, int batchSize, Executor executor, boolean allowReorderedResult)

    • groupBy

      public static <K,​ V> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • aggregateBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator)

    • aggregateBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator, R mutableMap)

    • aggregateBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator, int batchSize)

    • aggregateBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator, R mutableMap, int batchSize)

    • aggregateBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator, int batchSize, Executor executor)

    • aggregateBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Function2<? super V,​? super T,​? extends V> nonMutatingAggregator, R mutableMap, int batchSize, Executor executor)

    • aggregateInPlaceBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator)

    • aggregateInPlaceBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator, R mutableMap)

    • aggregateInPlaceBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator, int batchSize)

    • aggregateInPlaceBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator, R mutableMap, int batchSize)

    • aggregateInPlaceBy

      public static <T,​ K,​ V> MutableMap<K,​V> aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator, int batchSize, Executor executor)

    • aggregateInPlaceBy

      public static <T,​ K,​ V,​ R extends MutableMapIterable<K,​ V>> R aggregateInPlaceBy​(Iterable<T> iterable, Function<? super T,​? extends K> groupBy, Function0<? extends V> zeroValueFactory, Procedure2<? super V,​? super T> mutatingAggregator, R mutableMap, int batchSize, Executor executor)

    • groupBy

      public static <K,​ V,​ R extends MutableMultimap<K,​ V>> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function, R concurrentMultimap)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • groupBy

      public static <K,​ V,​ R extends MutableMultimap<K,​ V>> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function, R concurrentMultimap, int batchSize)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • groupBy

      public static <K,​ V> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function, int batchSize)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • groupBy

      public static <K,​ V> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function, int batchSize, Executor executor)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • groupBy

      public static <K,​ V,​ R extends MutableMultimap<K,​ V>> MutableMultimap<K,​V> groupBy​(Iterable<V> iterable, Function<? super V,​? extends K> function, R concurrentMultimap, int batchSize, Executor executor)
      Same effect as Iterate.groupBy(Iterable, Function), but executed in parallel batches, and writing output into a SynchronizedPutFastListMultimap.

    • sumByDouble

      public static <T,​ V> ObjectDoubleMap<V> sumByDouble​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, DoubleFunction<? super T> function)

    • sumByFloat

      public static <T,​ V> ObjectDoubleMap<V> sumByFloat​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, FloatFunction<? super T> function)

    • sumByLong

      public static <T,​ V> ObjectLongMap<V> sumByLong​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, LongFunction<? super T> function)

    • sumByInt

      public static <T,​ V> ObjectLongMap<V> sumByInt​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, IntFunction<? super T> function)

    • sumByBigDecimal

      public static <V,​ T> MutableMap<V,​BigDecimal> sumByBigDecimal​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, Function<? super T,​BigDecimal> function)
      Since:
      6.0

    • sumByBigInteger

      public static <V,​ T> MutableMap<V,​BigInteger> sumByBigInteger​(Iterable<T> iterable, Function<? super T,​? extends V> groupBy, Function<? super T,​BigInteger> function)
      Since:
      6.0

    • newPooledExecutor

      public static ExecutorService newPooledExecutor​(int newPoolSize, String poolName, boolean useDaemonThreads)
      Returns a brand new ExecutorService using the specified poolName with the specified maximum thread pool size. The same poolName may be used more than once resulting in multiple pools with the same name.

      The pool will be initialised with newPoolSize threads. If that number of threads are in use and another thread is requested, the pool will reject execution and the submitting thread will execute the task.

    • newPooledExecutor

      public static ExecutorService newPooledExecutor​(String poolName, boolean useDaemonThreads)
      Returns a brand new ExecutorService using the specified poolName and uses the optional property named to set the maximum thread pool size. The same poolName may be used more than once resulting in multiple pools with the same name.

    • getDefaultTaskCount

      public static int getDefaultTaskCount()

    • getDefaultMaxThreadPoolSize

      public static int getDefaultMaxThreadPoolSize()

    • getTaskRatio

      public static int getTaskRatio()