public class BufferFrontedFileStoreCache extends Object implements Cache<org.teiid.common.buffer.impl.PhysicalInfo>
FileStore
abstraction using a fronting
memory buffer with a filesystem paradigm. All objects must go through the
memory (typically off-heap) buffer so that they can be put into their appropriately
sized storage bucket.
The memory uses a 31bit address space on top of 2^13 byte blocks.
Therefore there is 2^31*2^13 = 2^44 or 16 terabytes max of addressable space.
This is well beyond any current needs.
The 64 byte inode format is:
14 32 bit direct block pointers
1 32 bit block indirect pointer
1 32 bit block doubly indirect pointer (should be rarely used)
This means that the maximum number of blocks available to an object is
14 + (2^13)/4 + ((2^13)/4)^2 ~= 2^22
Thus the max serialized object size is: 2^22*(2^13) ~= 32GB.
Typically the max object size will be much smaller, such as 8MB.
Inodes are held separately from the data/index blocks, and introduce an overhead
that is ~ 1/128th the size of memory buffer.
The filesystem stores are broken up into block specific sizes starting with 8KB.
The root directory "physicalMapping" is held in memory for performance. It will grow in
proportion to the number of tables/tuplebuffers in use.
The locking is as fine grained as possible to prevent contention. See PhysicalInfo
for
flags that are used when it is used as a lock. It is important to not access the
group maps when a PhysicalInfo
lock is held.Modifier and Type | Field and Description |
---|---|
static int |
DEFAuLT_MAX_OBJECT_SIZE |
static long |
MAX_ADDRESSABLE_MEMORY |
Constructor and Description |
---|
BufferFrontedFileStoreCache() |
Modifier and Type | Method and Description |
---|---|
boolean |
add(CacheEntry entry,
Serializer s)
Adds an entry to the cache.
|
boolean |
addToCacheGroup(Long gid,
Long oid)
Must be called prior to adding an entry
|
void |
createCacheGroup(Long gid)
Must be called prior to adding any group entries
|
FileStore |
createFileStore(String name) |
CacheEntry |
get(org.teiid.common.buffer.impl.PhysicalInfo info,
Long oid,
WeakReference<? extends Serializer<?>> ref)
Get method, must be called using the object obtained in the
lockForLoad method
|
int |
getDataBlocksInUse() |
long |
getDiskUsage() |
int |
getInodesInUse() |
int |
getMaxMemoryBlocks() |
long |
getMaxStorageSpace() |
long |
getMemoryBufferSpace() |
long |
getMemoryInUseBytes() |
StorageManager |
getStorageManager() |
long |
getStorageReads() |
long |
getStorageWrites() |
void |
initialize() |
org.teiid.common.buffer.impl.PhysicalInfo |
lockForLoad(Long oid,
Serializer<?> serializer)
Lock the object for load and return an identifier/lock
that can be used to retrieve the object.
|
Integer |
remove(Long gid,
Long id)
Remove an entry from the cache, return the sizeEstimate if the entry existed
|
Collection<Long> |
removeCacheGroup(Long gid)
Remove an entire cache group
TODO: this should use a callback on the buffermangaer to remove memory entries
without materializing all group keys
|
void |
setBufferManager(BufferManagerImpl bufferManager) |
void |
setCompactBufferFiles(boolean compactBufferFiles) |
void |
setDirect(boolean direct) |
void |
setMaxStorageObjectSize(int maxStorageBlockSize) |
void |
setMemoryBufferSpace(long maxBufferSpace) |
void |
setMinDefrag(long minDefrag) |
void |
setStorageManager(StorageManager storageManager) |
void |
setTruncateInterval(int truncateInterval) |
void |
shutdown() |
void |
unlockForLoad(org.teiid.common.buffer.impl.PhysicalInfo info)
Must be called after lockForLoad
|
public static final int DEFAuLT_MAX_OBJECT_SIZE
public static final long MAX_ADDRESSABLE_MEMORY
public void initialize() throws TeiidComponentException
initialize
in interface StorageManager
TeiidComponentException
public boolean add(CacheEntry entry, Serializer s)
Cache
public org.teiid.common.buffer.impl.PhysicalInfo lockForLoad(Long oid, Serializer<?> serializer)
Cache
lockForLoad
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public void unlockForLoad(org.teiid.common.buffer.impl.PhysicalInfo info)
Cache
unlockForLoad
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public CacheEntry get(org.teiid.common.buffer.impl.PhysicalInfo info, Long oid, WeakReference<? extends Serializer<?>> ref) throws TeiidComponentException
Cache
get
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
TeiidComponentException
public FileStore createFileStore(String name)
createFileStore
in interface StorageManager
public void setDirect(boolean direct)
public boolean addToCacheGroup(Long gid, Long oid)
Cache
addToCacheGroup
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public void createCacheGroup(Long gid)
Cache
createCacheGroup
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public Integer remove(Long gid, Long id)
Cache
public Collection<Long> removeCacheGroup(Long gid)
Cache
removeCacheGroup
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public void setStorageManager(StorageManager storageManager)
public StorageManager getStorageManager()
public void setMemoryBufferSpace(long maxBufferSpace)
public int getInodesInUse()
public int getDataBlocksInUse()
public void setMaxStorageObjectSize(int maxStorageBlockSize)
public long getStorageReads()
public long getStorageWrites()
public long getMemoryBufferSpace()
getMemoryBufferSpace
in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
public void setMinDefrag(long minDefrag)
public int getMaxMemoryBlocks()
public long getMemoryInUseBytes()
public void setBufferManager(BufferManagerImpl bufferManager)
public void setTruncateInterval(int truncateInterval)
public long getDiskUsage()
public void shutdown()
public void setCompactBufferFiles(boolean compactBufferFiles)
public long getMaxStorageSpace()
getMaxStorageSpace
in interface StorageManager
Copyright © 2018 JBoss by Red Hat. All rights reserved.