com.metamatrix.common.buffer.impl
Class BufferManagerImpl

java.lang.Object
  com.metamatrix.common.buffer.impl.BufferManagerImpl

All Implemented Interfaces:

BufferManager

public class BufferManagerImpl
extends java.lang.Object
implements BufferManager

Default implementation of BufferManager. This buffer manager implementation assumes the usage of a StorageManager of type memory and optionally (preferred) an additional StorageManager of type FILE or DISK. If no persistent manager is specified, everything managed by this BufferManager is assumed to fit in memory. This can be useful for testing or for small uses.

Lots of state is cached in memory. The tupleSourceMap contains a map of TupleSourceID --> TupleSourceInfo. Everything about a particular tuple source is stored there. The memoryState contains everything pertaining to memory management. The config contains all config info.

Nested Class Summary

Nested classes/interfaces inherited from interface com.metamatrix.common.buffer.BufferManager

BufferManager.TupleSourceStatus, BufferManager.TupleSourceType

Constructor Summary

BufferManagerImpl()

Method Summary

void	addStorageManager(StorageManager storageManager) Add a storage manager to this buffer manager, order is unimportant
void	addStreamablePart(TupleSourceID tupleSourceID, LobChunk streamChunk, int beginRow) Add a streamable object to the persistent store.
void	addTupleBatch(TupleSourceID tupleSourceID, TupleBatch tupleBatch) Add a batch to the given tuple source.
protected void	clean(long memoryRequired) Clean the memory state, using LRU.
protected void	clean(long memoryRequired, com.metamatrix.common.buffer.impl.TupleGroupInfo targetGroupInfo) Over memory limit for this session.
TupleSourceID	createTupleSource(java.util.List schema, java.lang.String[] types, java.lang.String groupName, BufferManager.TupleSourceType tupleSourceType) Register a new tuple source and return a unique ID for it.
BufferConfig	getConfig() Get the configuration of the buffer manager
int	getConnectorBatchSize() Get connector batch size
int	getFinalRowCount(TupleSourceID tupleSourceID) Gets the final row count if tuple source is FULL, otherwise returns -1.
int	getPinnedCount() for testing purposes
int	getProcessorBatchSize() Get processor batch size
int	getRowCount(TupleSourceID tupleSourceID) Get the row count for a particular tuple source
BufferStats	getStats() Construct a BufferStats object by looking at all the state.
BufferManager.TupleSourceStatus	getStatus(TupleSourceID tupleSourceID) Get the status for a particular tuple source
LobChunk	getStreamablePart(TupleSourceID tupleSourceID, int beginRow) Returns the streamable batch object's part stored with specified identifier
java.util.List	getTupleSchema(TupleSourceID tupleSourceID) Get the tuple schema for a particular tuple source
IndexedTupleSource	getTupleSource(TupleSourceID tupleSourceID) Get a tuple source to walk through the rows for a particular tupleSourceID.
void	initialize(java.lang.String lookup, java.util.Properties properties) See BufferManagerPropertyNames for a description of all the properties.
TupleBatch	pinTupleBatch(TupleSourceID tupleSourceID, int beginRow, int maxEndRow) Pin a tuple source in memory and return it.
void	releasePinnedBatches() Release batches that have been pinned by this thread.
void	removeTupleSource(TupleSourceID tupleSourceID) Remove a tuple source based on ID
void	removeTupleSources(java.lang.String groupName) Remove all the tuple sources with the specified group name.
void	setStatus(TupleSourceID tupleSourceID, BufferManager.TupleSourceStatus status) Set the status for a particular tuple source
void	stop()
void	unpinTupleBatch(TupleSourceID tupleSourceID, int beginRow, int endRow) Unpin a tuple source batch.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferManagerImpl

public BufferManagerImpl()

Method Detail

initialize

public void initialize(java.lang.String lookup,
                       java.util.Properties properties)
                throws MetaMatrixComponentException

See BufferManagerPropertyNames for a description of all the properties.

Specified by:

initialize in interface BufferManager

Parameters:

lookup - An object telling the buffer manager what his location is and how to find other buffer managers of different locations

properties - Properties to configure the buffer manager

Throws:

MetaMatrixComponentException

getConfig

public BufferConfig getConfig()

Get the configuration of the buffer manager

Returns:

Configuration

getStats

public BufferStats getStats()

Construct a BufferStats object by looking at all the state. This should be multi-thread safe but calling this method clears several counters and resets that state.

Returns:

Buffer statistics

getProcessorBatchSize

public int getProcessorBatchSize()

Get processor batch size

Specified by:

getProcessorBatchSize in interface BufferManager

Returns:

Number of rows in a processor batch

getConnectorBatchSize

public int getConnectorBatchSize()

Get connector batch size

Specified by:

getConnectorBatchSize in interface BufferManager

Returns:

Number of rows in a connector batch

addStorageManager

public void addStorageManager(StorageManager storageManager)

Add a storage manager to this buffer manager, order is unimportant

Specified by:

addStorageManager in interface BufferManager

Parameters:

storageManager - Storage manager to add

createTupleSource

public TupleSourceID createTupleSource(java.util.List schema,
                                       java.lang.String[] types,
                                       java.lang.String groupName,
                                       BufferManager.TupleSourceType tupleSourceType)
                                throws MetaMatrixComponentException

Register a new tuple source and return a unique ID for it.

Specified by:

createTupleSource in interface BufferManager

Parameters:

schema - List of ElementSymbol

groupName - Group name

tupleSourceType - Type of tuple source as defined in BufferManager constants

Returns:

New unique ID for this tuple source

Throws:

MetaMatrixComponentException - If internal server error occurs

removeTupleSource

public void removeTupleSource(TupleSourceID tupleSourceID)
                       throws TupleSourceNotFoundException,
                              MetaMatrixComponentException

Remove a tuple source based on ID

Specified by:

removeTupleSource in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If internal server error occurs

removeTupleSources

public void removeTupleSources(java.lang.String groupName)
                        throws MetaMatrixComponentException

Remove all the tuple sources with the specified group name. Typically the group name is really a session identifier and this is called to remove all tuple sources used by a particular session.

Specified by:

removeTupleSources in interface BufferManager

Parameters:

groupName - Name of the group

Throws:

MetaMatrixComponentException - If internal server error occurs

getTupleSource

public IndexedTupleSource getTupleSource(TupleSourceID tupleSourceID)
                                  throws TupleSourceNotFoundException,
                                         MetaMatrixComponentException

Get a tuple source to walk through the rows for a particular tupleSourceID.

Specified by:

getTupleSource in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Returns:

TupleSource

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

getTupleSchema

public java.util.List getTupleSchema(TupleSourceID tupleSourceID)
                              throws TupleSourceNotFoundException,
                                     MetaMatrixComponentException

Get the tuple schema for a particular tuple source

Specified by:

getTupleSchema in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Returns:

List of elements describing tuple source

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

setStatus

public void setStatus(TupleSourceID tupleSourceID,
                      BufferManager.TupleSourceStatus status)
               throws TupleSourceNotFoundException,
                      MetaMatrixComponentException

Set the status for a particular tuple source

Specified by:

setStatus in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

status - New status

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

See Also:

BufferManager.TupleSourceStatus.ACTIVE, BufferManager.TupleSourceStatus.FULL

getStatus

public BufferManager.TupleSourceStatus getStatus(TupleSourceID tupleSourceID)
                                          throws TupleSourceNotFoundException,
                                                 MetaMatrixComponentException

Get the status for a particular tuple source

Specified by:

getStatus in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Returns:

Status

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

See Also:

BufferManager.TupleSourceStatus.ACTIVE, BufferManager.TupleSourceStatus.FULL

getRowCount

public int getRowCount(TupleSourceID tupleSourceID)
                throws TupleSourceNotFoundException,
                       MetaMatrixComponentException

Get the row count for a particular tuple source

Specified by:

getRowCount in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Returns:

Row count

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

addTupleBatch

public void addTupleBatch(TupleSourceID tupleSourceID,
                          TupleBatch tupleBatch)
                   throws TupleSourceNotFoundException,
                          MetaMatrixComponentException

Add a batch to the given tuple source. It is assumed that batches are added in order, so the row count is reset to the end row of this batch.

Specified by:

addTupleBatch in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

tupleBatch - New batch

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

pinTupleBatch

public TupleBatch pinTupleBatch(TupleSourceID tupleSourceID,
                                int beginRow,
                                int maxEndRow)
                         throws TupleSourceNotFoundException,
                                MemoryNotAvailableException,
                                MetaMatrixComponentException

Pin a tuple source in memory and return it. This batch must be unpinned by passed the identical tuple source ID and beginning row.

Specified by:

pinTupleBatch in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

beginRow - Beginning row

maxEndRow - Max end row to return

Returns:

Batch of rows starting from beginRow and not past maxEndRow

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

MemoryNotAvailableException - If memory was not available for the pin

unpinTupleBatch

public void unpinTupleBatch(TupleSourceID tupleSourceID,
                            int beginRow,
                            int endRow)
                     throws TupleSourceNotFoundException,
                            MetaMatrixComponentException

Unpin a tuple source batch.

Specified by:

unpinTupleBatch in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

beginRow - Beginning row

endRow - Last row to unpin (inclusive)

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

getFinalRowCount

public int getFinalRowCount(TupleSourceID tupleSourceID)
                     throws TupleSourceNotFoundException,
                            MetaMatrixComponentException

Gets the final row count if tuple source is FULL, otherwise returns -1.

Specified by:

getFinalRowCount in interface BufferManager

Parameters:

tupleSourceID - Tuple source identifier

Returns:

Final row count if status == FULL, -1 otherwise

Throws:

TupleSourceNotFoundException - If tuple source not found

MetaMatrixComponentException - If an internal server error occurred

clean

protected void clean(long memoryRequired)

Clean the memory state, using LRU. This can be done either via the background cleaning thread or actively if someone wants memory and none is free.

clean

protected void clean(long memoryRequired,
                     com.metamatrix.common.buffer.impl.TupleGroupInfo targetGroupInfo)
              throws TupleSourceNotFoundException

Over memory limit for this session. Clean the memory for this session. Clean the memory state, using LRU. This can be done actively if someone wants memory and none is free.

Throws:

TupleSourceNotFoundException

stop

public void stop()

Specified by:

stop in interface BufferManager

See Also:

BufferManager.stop()

addStreamablePart

public void addStreamablePart(TupleSourceID tupleSourceID,
                              LobChunk streamChunk,
                              int beginRow)
                       throws TupleSourceNotFoundException,
                              MetaMatrixComponentException

Description copied from interface: BufferManager

Add a streamable object to the persistent store. The distinction is made between regular tuple sources and streamable object beacuse, streamable objects are fairly large in size can not be loaded totally into memory, as other tuple sources are done this mechanism allows to stream these objects chunk by chunk.

Specified by:

addStreamablePart in interface BufferManager

streamChunk - part of the stream

Throws:

TupleSourceNotFoundException

MetaMatrixComponentException

See Also:

BufferManager.addStreamablePart(com.metamatrix.common.buffer.TupleSourceID, com.metamatrix.common.lob.LobChunk, int)

getStreamablePart

public LobChunk getStreamablePart(TupleSourceID tupleSourceID,
                                  int beginRow)
                           throws TupleSourceNotFoundException,
                                  MetaMatrixComponentException

Description copied from interface: BufferManager

Returns the streamable batch object's part stored with specified identifier

Specified by:

getStreamablePart in interface BufferManager

Parameters:

tupleSourceID - - identifier

Returns:

LobChunk a part of the Streamable object stored.

Throws:

TupleSourceNotFoundException

MetaMatrixComponentException

See Also:

com.metamatrix.common.buffer.BufferManager#getStreamable(com.metamatrix.common.buffer.TupleSourceID)

releasePinnedBatches

public void releasePinnedBatches()
                          throws MetaMatrixComponentException

Description copied from interface: BufferManager

Release batches that have been pinned by this thread. This method should be called when processing is terminated to ensure that the memory can be freed.

Specified by:

releasePinnedBatches in interface BufferManager

Throws:

MetaMatrixComponentException

See Also:

BufferManager.releasePinnedBatches()

getPinnedCount

public int getPinnedCount()

for testing purposes