org.teiid.common.buffer.impl
Class BufferManagerImpl

java.lang.Object
  org.teiid.common.buffer.impl.BufferManagerImpl

All Implemented Interfaces:

BufferManager, StorageManager

public class BufferManagerImpl
extends java.lang.Object
implements BufferManager, StorageManager

Default implementation of BufferManager.

Responsible for creating/tracking TupleBuffers and providing access to the StorageManager.

The buffering strategy attempts to purge batches from the least recently used TupleBuffer from before (which wraps around circularly) the last used batch. This attempts to compensate for our tendency to read buffers in a forward manner. If our processing algorithms are changed to use alternating ascending/descending access, then the buffering approach could be replaced with a simple LRU. TODO: allow for cached stores to use lru - (result set/mat view) TODO: account for row/content based sizing (difficult given value sharing) TODO: account for memory based lobs (it would be nice if the approximate buffer size matched at 100kB) TODO: add detection of pinned batches to prevent unnecessary purging of non-persistent batches - this is not necessary for already persistent batches, since we hold a weak reference

Nested Class Summary

Nested classes/interfaces inherited from interface org.teiid.common.buffer.BufferManager

BufferManager.BufferReserveMode, BufferManager.TupleSourceType

Field Summary

Fields inherited from interface org.teiid.common.buffer.BufferManager

DEFAULT_CONNECTOR_BATCH_SIZE, DEFAULT_MAX_PROCESSING_KB, DEFAULT_PROCESSOR_BATCH_SIZE, DEFAULT_RESERVE_BUFFER_KB

Constructor Summary

BufferManagerImpl()

Method Summary

void	addTupleBuffer(TupleBuffer tb)
FileStore	createFileStore(java.lang.String name) Creates a new FileStore.
STree	createSTree(java.util.List elements, java.lang.String groupName, int keyLength)
TupleBuffer	createTupleBuffer(java.util.List elements, java.lang.String groupName, BufferManager.TupleSourceType tupleSourceType)
long	getBatchesAdded()
int	getConnectorBatchSize() Get connector batch size
int	getMaxProcessingKB() Return the maximum KB that can be temporarily held potentially across even a blocked exception.
int	getProcessorBatchSize() Get processor batch size
long	getReadAttempts()
long	getReadCount()
int	getSchemaSize(java.util.List<? extends Expression> elements) Get the size estimate in KB for the given schema.
StorageManager	getStorageManager()
TupleBuffer	getTupleBuffer(java.lang.String id)
long	getWriteCount()
void	initialize()
void	releaseBuffers(int count) Releases the buffers reserved by a call to BufferManager#reserveBuffers(int, boolean)
int	reserveBuffers(int count, BufferManager.BufferReserveMode mode) Reserve up to count buffers for use.
void	setConnectorBatchSize(int connectorBatchSize)
void	setMaxActivePlans(int maxActivePlans) Set the maxActivePlans as a hint at determining the maxProcessingKB
void	setMaxProcessingKB(int maxProcessingKB)
void	setMaxReserveKB(int maxReserveBatchKB)
void	setProcessorBatchSize(int processorBatchSize)
void	setStorageManager(StorageManager storageManager) Add a storage manager to this buffer manager, order is unimportant
void	setUseWeakReferences(boolean useWeakReferences)
void	shutdown()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferManagerImpl

public BufferManagerImpl()

Method Detail

getBatchesAdded

public long getBatchesAdded()

getReadCount

public long getReadCount()

getWriteCount

public long getWriteCount()

getReadAttempts

public long getReadAttempts()

getMaxProcessingKB

public int getMaxProcessingKB()

Description copied from interface: BufferManager

Return the maximum KB that can be temporarily held potentially across even a blocked exception.

Specified by:

getMaxProcessingKB in interface BufferManager

Returns:

getProcessorBatchSize

public int getProcessorBatchSize()

Get processor batch size

Specified by:

getProcessorBatchSize in interface BufferManager

Returns:

Number of rows in a processor batch

getConnectorBatchSize

public int getConnectorBatchSize()

Get connector batch size

Specified by:

getConnectorBatchSize in interface BufferManager

Returns:

Number of rows in a connector batch

setConnectorBatchSize

public void setConnectorBatchSize(int connectorBatchSize)

setProcessorBatchSize

public void setProcessorBatchSize(int processorBatchSize)

setStorageManager

public void setStorageManager(StorageManager storageManager)

Add a storage manager to this buffer manager, order is unimportant

Parameters:

storageManager - Storage manager to add

getStorageManager

public StorageManager getStorageManager()

createTupleBuffer

public TupleBuffer createTupleBuffer(java.util.List elements,
                                     java.lang.String groupName,
                                     BufferManager.TupleSourceType tupleSourceType)

Specified by:

createTupleBuffer in interface BufferManager

createSTree

public STree createSTree(java.util.List elements,
                         java.lang.String groupName,
                         int keyLength)

Specified by:

createSTree in interface BufferManager

createFileStore

public FileStore createFileStore(java.lang.String name)

Description copied from interface: BufferManager

Creates a new FileStore. See FileStore.setCleanupReference(Object) to automatically cleanup the underlying resources.

Specified by:

createFileStore in interface BufferManager

Specified by:

createFileStore in interface StorageManager

setMaxActivePlans

public void setMaxActivePlans(int maxActivePlans)

Description copied from interface: BufferManager

Set the maxActivePlans as a hint at determining the maxProcessingKB

Specified by:

setMaxActivePlans in interface BufferManager

setMaxProcessingKB

public void setMaxProcessingKB(int maxProcessingKB)

setMaxReserveKB

public void setMaxReserveKB(int maxReserveBatchKB)

initialize

public void initialize()
                throws TeiidComponentException

Specified by:

initialize in interface StorageManager

Throws:

TeiidComponentException

releaseBuffers

public void releaseBuffers(int count)

Description copied from interface: BufferManager

Releases the buffers reserved by a call to BufferManager#reserveBuffers(int, boolean)

Specified by:

releaseBuffers in interface BufferManager

reserveBuffers

public int reserveBuffers(int count,
                          BufferManager.BufferReserveMode mode)

Description copied from interface: BufferManager

Reserve up to count buffers for use.

Specified by:

reserveBuffers in interface BufferManager

Returns:

getSchemaSize

public int getSchemaSize(java.util.List<? extends Expression> elements)

Description copied from interface: BufferManager

Get the size estimate in KB for the given schema.

Specified by:

getSchemaSize in interface BufferManager

shutdown

public void shutdown()

addTupleBuffer

public void addTupleBuffer(TupleBuffer tb)

Specified by:

addTupleBuffer in interface BufferManager

getTupleBuffer

public TupleBuffer getTupleBuffer(java.lang.String id)

Specified by:

getTupleBuffer in interface BufferManager

setUseWeakReferences

public void setUseWeakReferences(boolean useWeakReferences)