org.infinispan.distexec.mapreduce

Class MapReduceManagerImpl

java.lang.Object

org.infinispan.distexec.mapreduce.MapReduceManagerImpl

All Implemented Interfaces:

MapReduceManager

public class MapReduceManagerImpl
extends Object
implements MapReduceManager

Default implementation of MapReduceManager.

This is an internal class, not intended to be used by clients.

Since:

5.2

Author:

Vladimir Blagojevic

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MapReduceManagerImpl.IntermediateCompositeKey<V> IntermediateCompositeKey

Method Summary

Methods

Modifier and Type	Method and Description
protected <KIn,VIn,KOut,VOut> Set<KOut>	combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc, org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> collector)
protected <KIn> Set<KIn>	filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys, DistributionManager dm)
ExecutorService	getExecutorService() ExecutorService provided for local task execution
void	init(EmbeddedCacheManager cacheManager, CacheLoaderManager cacheLoaderManager, ExecutorService asyncTransportExecutor)
protected <KIn> Set<KIn>	loadAllKeysFromCacheLoaderUsingFilter(Set<KIn> filterOutSet)
protected <KIn,KOut> KOut	loadValueFromCacheLoader(KIn key)
protected <KIn,VIn,KOut,VOut> org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut>	map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
<KIn,VIn,KOut,VOut> Set<KOut>	mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc) Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.
<KIn,VIn,KOut,VOut> Map<KOut,List<VOut>>	mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc) Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.
<T> Map<Address,List<T>>	mapKeysToNodes(DistributionManager dm, String taskId, Collection<T> keysToMap, boolean useIntermediateCompositeKey) Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster
<KOut,VOut> Map<KOut,VOut>	reduce(ReduceCommand<KOut,VOut> reduceCommand) Invoked when ReduceCommand arrives to a target Infinispan node.
protected CacheLoader	resolveCacheLoader()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

init

public void init(EmbeddedCacheManager cacheManager,
        CacheLoaderManager cacheLoaderManager,
        ExecutorService asyncTransportExecutor)

getExecutorService

public ExecutorService getExecutorService()

Description copied from interface: MapReduceManager

ExecutorService provided for local task execution

Specified by:

getExecutorService in interface MapReduceManager

Returns:

ExecutorService for local tasks

mapAndCombineForLocalReduction

public <KIn,VIn,KOut,VOut> Map<KOut,List<VOut>> mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                                    throws InterruptedException

Description copied from interface: MapReduceManager

Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.

Unlike MapReduceManager.mapAndCombineForDistributedReduction(MapCombineCommand) implementations should return a map Map> which should be ready for reduction at master Infinispan node

Specified by:

mapAndCombineForLocalReduction in interface MapReduceManager

Parameters:

mcc - MapCombineCommand sent from MapReduceTask

Returns:

a map Map> which should be ready for reduction at master Infinispan node

Throws:

InterruptedException

mapAndCombineForDistributedReduction

public <KIn,VIn,KOut,VOut> Set<KOut> mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                               throws InterruptedException

Description copied from interface: MapReduceManager

Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.

Specified by:

mapAndCombineForDistributedReduction in interface MapReduceManager

Parameters:

mcc - MapCombineCommand sent from MapReduceTask

Returns:

a set of intermediate keys to be reduced distributively

Throws:

InterruptedException

reduce

public <KOut,VOut> Map<KOut,VOut> reduce(ReduceCommand<KOut,VOut> reduceCommand)
                      throws InterruptedException

Description copied from interface: MapReduceManager

Invoked when ReduceCommand arrives to a target Infinispan node. Implementations should return a map of reduced output keys and values to be returned to invoker of MapReduceTask

Specified by:

reduce in interface MapReduceManager

Parameters:

reduceCommand - ReduceCommand sent from MapReduceTask

Returns:

map of reduced output keys and values returned to MapReduceTask

Throws:

InterruptedException

map

protected <KIn,VIn,KOut,VOut> org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                                                                              throws InterruptedException

Throws:

InterruptedException

combine

protected <KIn,VIn,KOut,VOut> Set<KOut> combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc,
                                    org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> collector)
                     throws Exception

Throws:

Exception

loadAllKeysFromCacheLoaderUsingFilter

protected <KIn> Set<KIn> loadAllKeysFromCacheLoaderUsingFilter(Set<KIn> filterOutSet)

loadValueFromCacheLoader

protected <KIn,KOut> KOut loadValueFromCacheLoader(KIn key)

resolveCacheLoader

protected CacheLoader resolveCacheLoader()

mapKeysToNodes

public <T> Map<Address,List<T>> mapKeysToNodes(DistributionManager dm,
                                      String taskId,
                                      Collection<T> keysToMap,
                                      boolean useIntermediateCompositeKey)

Description copied from interface: MapReduceManager

Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster

Specified by:

mapKeysToNodes in interface MapReduceManager

Parameters:

dm - distribution manager to use for locating keys on hash wheel

taskId - id of the map/reduce task

keysToMap - list of input keys to locate in the cluster

useIntermediateCompositeKey - if true use composite keys for shared intermediate cache

Returns:

map where each key is an Address in the cluster and value are the keys mapped to that Address

filterLocalPrimaryOwner

protected <KIn> Set<KIn> filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys,
                                     DistributionManager dm)