Package org.teiid.common.buffer.impl

Class BufferFrontedFileStoreCache

  • All Implemented Interfaces:
    Cache<org.teiid.common.buffer.impl.PhysicalInfo>, StorageManager
    public class BufferFrontedFileStoreCache
extends Object
implements Cache<org.teiid.common.buffer.impl.PhysicalInfo>
    Implements storage against a FileStore abstraction using a fronting memory buffer with a filesystem paradigm. All objects must go through the memory (typically off-heap) buffer so that they can be put into their appropriately sized storage bucket. The memory uses a 31bit address space on top of 2^13 byte blocks. Therefore there is 2^31*2^13 = 2^44 or 16 terabytes max of addressable space. This is well beyond any current needs. The 64 byte inode format is: 14 32 bit direct block pointers 1 32 bit block indirect pointer 1 32 bit block doubly indirect pointer (should be rarely used) This means that the maximum number of blocks available to an object is 14 + (2^13)/4 + ((2^13)/4)^2 ~= 2^22 Thus the max serialized object size is: 2^22*(2^13) ~= 32GB. Typically the max object size will be much smaller, such as 8MB. Inodes are held separately from the data/index blocks, and introduce an overhead that is ~ 1/128th the size of memory buffer. The filesystem stores are broken up into block specific sizes starting with 8KB. The root directory "physicalMapping" is held in memory for performance. It will grow in proportion to the number of tables/tuplebuffers in use. The locking is as fine grained as possible to prevent contention. See PhysicalInfo for flags that are used when it is used as a lock. It is important to not access the group maps when a PhysicalInfo lock is held.

    • Constructor Detail

      • BufferFrontedFileStoreCache

        public BufferFrontedFileStoreCache()

    • Method Detail

      • add

        public boolean add​(CacheEntry entry,
                   Serializer s)
        Description copied from interface: Cache
        Adds an entry to the cache.
        Specified by:
        add in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • lockForLoad

        public org.teiid.common.buffer.impl.PhysicalInfo lockForLoad​(Long oid,
                                                             Serializer<?> serializer)
        Description copied from interface: Cache
        Lock the object for load and return an identifier/lock that can be used to retrieve the object.
        Specified by:
        lockForLoad in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
        Returns:
        the identifier, may be null

      • unlockForLoad

        public void unlockForLoad​(org.teiid.common.buffer.impl.PhysicalInfo info)
        Description copied from interface: Cache
        Must be called after lockForLoad
        Specified by:
        unlockForLoad in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • getCacheGroupCount

        public int getCacheGroupCount()
        Specified by:
        getCacheGroupCount in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • setDirect

        public void setDirect​(boolean direct)

      • addToCacheGroup

        public boolean addToCacheGroup​(Long gid,
                               Long oid)
        Description copied from interface: Cache
        Must be called prior to adding an entry
        Specified by:
        addToCacheGroup in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
        Returns:
        if the add was successful

      • createCacheGroup

        public void createCacheGroup​(Long gid)
        Description copied from interface: Cache
        Must be called prior to adding any group entries
        Specified by:
        createCacheGroup in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • remove

        public Integer remove​(Long gid,
                      Long id)
        Description copied from interface: Cache
        Remove an entry from the cache, return the sizeEstimate if the entry existed
        Specified by:
        remove in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • removeCacheGroup

        public Collection<Long> removeCacheGroup​(Long gid)
        Description copied from interface: Cache
        Remove an entire cache group TODO: this should use a callback on the buffermangaer to remove memory entries without materializing all group keys
        Specified by:
        removeCacheGroup in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>
        Returns:

      • setStorageManager

        public void setStorageManager​(StorageManager storageManager)

      • setMemoryBufferSpace

        public void setMemoryBufferSpace​(long maxBufferSpace)

      • getInodesInUse

        public int getInodesInUse()

      • getDataBlocksInUse

        public int getDataBlocksInUse()

      • setMaxStorageObjectSize

        public void setMaxStorageObjectSize​(int maxStorageBlockSize)

      • getStorageReads

        public long getStorageReads()

      • getStorageWrites

        public long getStorageWrites()

      • getMemoryBufferSpace

        public long getMemoryBufferSpace()
        Specified by:
        getMemoryBufferSpace in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • setMinDefrag

        public void setMinDefrag​(long minDefrag)

      • getMaxMemoryBlocks

        public int getMaxMemoryBlocks()

      • getMemoryInUseBytes

        public long getMemoryInUseBytes()

      • setBufferManager

        public void setBufferManager​(BufferManagerImpl bufferManager)

      • setTruncateInterval

        public void setTruncateInterval​(int truncateInterval)

      • getDiskUsage

        public long getDiskUsage()

      • shutdown

        public void shutdown()
        Specified by:
        shutdown in interface Cache<org.teiid.common.buffer.impl.PhysicalInfo>

      • setCompactBufferFiles

        public void setCompactBufferFiles​(boolean compactBufferFiles)