CernVM-FS  2.12.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
RamCacheManager Class Reference

#include <cache_ram.h>

Inheritance diagram for RamCacheManager:
Collaboration diagram for RamCacheManager:

Classes

struct  Counters
 
struct  ReadOnlyHandle
 
struct  Transaction
 

Public Member Functions

virtual CacheManagerIds id ()
 
virtual std::string Describe ()
 
 RamCacheManager (uint64_t max_size, unsigned max_entries, MemoryKvStore::MemoryAllocator alloc, perf::StatisticsTemplate statistics)
 
virtual ~RamCacheManager ()
 
virtual bool AcquireQuotaManager (QuotaManager *quota_mgr)
 
virtual int Open (const LabeledObject &object)
 
virtual int64_t GetSize (int fd)
 
virtual int Close (int fd)
 
virtual int64_t Pread (int fd, void *buf, uint64_t size, uint64_t offset)
 
virtual int Dup (int fd)
 
virtual int Readahead (int fd)
 
virtual uint32_t SizeOfTxn ()
 
virtual int StartTxn (const shash::Any &id, uint64_t size, void *txn)
 
virtual void CtrlTxn (const Label &label, const int flags, void *txn)
 
virtual int64_t Write (const void *buf, uint64_t size, void *txn)
 
virtual int Reset (void *txn)
 
virtual int OpenFromTxn (void *txn)
 
virtual int AbortTxn (void *txn)
 
virtual int CommitTxn (void *txn)
 
virtual void Spawn ()
 
- Public Member Functions inherited from CacheManager
virtual ~CacheManager ()
 
int ChecksumFd (int fd, shash::Any *id)
 
int OpenPinned (const LabeledObject &object)
 
bool Open2Mem (const LabeledObject &object, unsigned char **buffer, uint64_t *size)
 
bool CommitFromMem (const LabeledObject &object, const unsigned char *buffer, const uint64_t size)
 
QuotaManagerquota_mgr ()
 
void * SaveState (const int fd_progress)
 
int RestoreState (const int fd_progress, void *state)
 
void FreeState (const int fd_progress, void *state)
 
CacheManagerIds PeekState (void *state)
 
virtual manifest::Breadcrumb LoadBreadcrumb (const std::string &)
 
virtual bool StoreBreadcrumb (const manifest::Manifest &)
 

Private Member Functions

MemoryKvStoreGetStore (const ReadOnlyHandle &fd)
 
MemoryKvStoreGetTransactionStore (Transaction *txn)
 
int AddFd (const ReadOnlyHandle &handle)
 
int64_t CommitToKvStore (Transaction *transaction)
 
virtual int DoOpen (const shash::Any &id)
 

Private Attributes

uint64_t max_size_
 
FdTable< ReadOnlyHandlefd_table_
 
pthread_rwlock_t rwlock_
 
MemoryKvStore regular_entries_
 
MemoryKvStore volatile_entries_
 
Counters counters_
 

Static Private Attributes

static const shash::Any kInvalidHandle
 

Additional Inherited Members

- Static Public Attributes inherited from CacheManager
static const uint64_t kSizeUnknown = uint64_t(-1)
 
static const int kLabelCatalog = 0x01
 
static const int kLabelPinned = 0x02
 
static const int kLabelVolatile = 0x04
 
static const int kLabelExternal = 0x08
 
static const int kLabelChunked = 0x10
 
static const int kLabelCertificate = 0x20
 
static const int kLabelMetainfo = 0x40
 
static const int kLabelHistory = 0x80
 
- Protected Member Functions inherited from CacheManager
 CacheManager ()
 
virtual void * DoSaveState ()
 
virtual int DoRestoreState (void *data)
 
virtual bool DoFreeState (void *data)
 
- Protected Attributes inherited from CacheManager
QuotaManagerquota_mgr_
 

Detailed Description

This file is part of the CernVM File System. The RamCacheManager class provides a cache backend that operates entirely from the host's RAM. This backend does not require any additional privileges on the host such as filesystem access. This cache resides in a single process' memory, and does not support sharing or persistence. This cache backend is a good choice in highly restrictive environments, e.g. HPC resources, where it is not possible to make use of a shared/local filesystem.

To use this cache backend, set CVMFS_CACHE_PRIMARY=ram. There are not many knobs to configure; at present only the size to be used for cached objects. CVMFS_CACHE_RAM_SIZE sets the amount of memory to use (in MB) for caching objects. If the size ends with a percent sign, the cache size will be set to that percentage of the system memory. If no size is specified, the RAM cache size defaults to ~3% of the system memory. The minimum cache size is 200 MB.

RamCacheManager uses a custom heap allocator rather than the system's libc malloc(). To switch to libc malloc, set CVMFS_CACHE_RAM_MALLOC=libc

Definition at line 44 of file cache_ram.h.

Constructor & Destructor Documentation

RamCacheManager::RamCacheManager ( uint64_t  max_size,
unsigned  max_entries,
MemoryKvStore::MemoryAllocator  alloc,
perf::StatisticsTemplate  statistics 
)

Definition at line 29 of file cache_ram.cc.

Here is the call graph for this function:

RamCacheManager::~RamCacheManager ( )
virtual

Definition at line 58 of file cache_ram.cc.

Member Function Documentation

int RamCacheManager::AbortTxn ( void *  txn)
virtual

Free the resources allocated for the pending transaction

Parameters
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 308 of file cache_ram.cc.

Here is the call graph for this function:

bool RamCacheManager::AcquireQuotaManager ( QuotaManager quota_mgr)
virtual

Implements CacheManager.

Definition at line 73 of file cache_ram.cc.

Referenced by FileSystem::SetupRamCacheMgr().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::AddFd ( const ReadOnlyHandle handle)
private

Definition at line 63 of file cache_ram.cc.

Referenced by DoOpen(), and Dup().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::Close ( int  fd)
virtual

Close the descriptor in the cache. Entries not marked as pinned will become subject to eviction once closed

Parameters
idThe hash key
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 136 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::CommitToKvStore ( Transaction transaction)
private

Definition at line 329 of file cache_ram.cc.

Referenced by CommitTxn(), and OpenFromTxn().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::CommitTxn ( void *  txn)
virtual

Commit a transaction to the cache. If there is not enough free space in the cache, first try to make room by evicting volatile entries. If there is still not enough room, try evicting regular entries. If there is still not enough space, give up an return failure.

Parameters
txnA pointer to space allocated for storing the transaction details
Return values
-ENOSPCThe transaction would exceed the size of the cache
-ENFILENo handles are available
-EEXISTAn entry with nonzero reference count already exists

Implements CacheManager.

Definition at line 318 of file cache_ram.cc.

Here is the call graph for this function:

void RamCacheManager::CtrlTxn ( const Label label,
const int  flags,
void *  txn 
)
virtual

Set the transaction parameters. At present, only type is used for pinned, volatile, etc.

Parameters
descriptionUnused
typeThe type of the entry, e.g. pinned
flagsUnused
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 229 of file cache_ram.cc.

Here is the call graph for this function:

string RamCacheManager::Describe ( )
virtual

Return a human readable description of the cache instance. Used in cvmfs_talk.

Implements CacheManager.

Definition at line 23 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::DoOpen ( const shash::Any id)
privatevirtual

Definition at line 87 of file cache_ram.cc.

Referenced by Open(), and OpenFromTxn().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::Dup ( int  fd)
virtual

Duplicates the open file descriptor, allowing the original and the new one to be used independently

Parameters
idThe hash key
Returns
A new, nonnegative integer fd
Return values
-EBADFfd is not valid
-ENFILENo handles are available

Implements CacheManager.

Definition at line 174 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::GetSize ( int  fd)
virtual

Look up the size in bytes of the open cache entry

Parameters
idThe hash key
Returns
The size of the entry
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 124 of file cache_ram.cc.

Here is the call graph for this function:

MemoryKvStore* RamCacheManager::GetStore ( const ReadOnlyHandle fd)
inlineprivate

Definition at line 276 of file cache_ram.h.

Referenced by Close(), DoOpen(), Dup(), GetSize(), and Pread().

Here is the caller graph for this function:

MemoryKvStore* RamCacheManager::GetTransactionStore ( Transaction txn)
inlineprivate

Definition at line 284 of file cache_ram.h.

virtual CacheManagerIds RamCacheManager::id ( )
inlinevirtual

Implements CacheManager.

Definition at line 103 of file cache_ram.h.

Referenced by StartTxn().

Here is the caller graph for this function:

int RamCacheManager::Open ( const LabeledObject object)
virtual

Open a new file descriptor into the cache. Note that opening entries effectively pins them in the cache, so it may be necessary to close unneeded file descriptors to allow eviction to make room in the cache

Parameters
objectThe tagged hash key
Returns
A nonnegative integer file descriptor
Return values
-ENOENTThe entry is not in the cache
-ENFILENo handles are available

Implements CacheManager.

Definition at line 81 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::OpenFromTxn ( void *  txn)
virtual

Commit a transaction and open the resulting cache entry. This is necessary to avoid a race condition in which the cache entry is evicted between the calls to CommitTxn and Open.

Parameters
txnA pointer to space allocated for storing the transaction details
Returns
A file descriptor to the new cache entry, or a negative error from CommitTxn

Implements CacheManager.

Definition at line 291 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::Pread ( int  fd,
void *  buf,
uint64_t  size,
uint64_t  offset 
)
virtual

Read a section from the cache entry. See pread(3) for a discussion of the arguments

Parameters
idThe hash key
Returns
The number of bytes copied
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 156 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::Readahead ( int  fd)
virtual

No effect for in-memory caches

Parameters
idThe hash key
Return values
-EBADFfd is not valid

For a RAM cache, read-ahead is a no-op.

Implements CacheManager.

Definition at line 196 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::Reset ( void *  txn)
virtual

Seek to the beginning of the transaction buffer

Parameters
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 281 of file cache_ram.cc.

Here is the call graph for this function:

virtual uint32_t RamCacheManager::SizeOfTxn ( )
inlinevirtual

Get the amount of space to be allocated for a call to StartTxn

Implements CacheManager.

Definition at line 173 of file cache_ram.h.

Here is the call graph for this function:

virtual void RamCacheManager::Spawn ( )
inlinevirtual

Implements CacheManager.

Definition at line 238 of file cache_ram.h.

int RamCacheManager::StartTxn ( const shash::Any id,
uint64_t  size,
void *  txn 
)
virtual

Start a new transaction, allocating the required memory and initializing the transaction parameters

Parameters
idThe hash key
sizeThe total size of the new cache entry
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 209 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::Write ( const void *  buf,
uint64_t  size,
void *  txn 
)
virtual

Copy the given memory region into the transaction buffer. Copying starts at the transaction's current offset

Parameters
bufThe source address
sizeThe number of bytes to copy
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 240 of file cache_ram.cc.

Here is the call graph for this function:

Member Data Documentation

Counters RamCacheManager::counters_
private
FdTable<ReadOnlyHandle> RamCacheManager::fd_table_
private

Definition at line 297 of file cache_ram.h.

Referenced by AddFd(), Close(), Dup(), GetSize(), Pread(), and Readahead().

const shash::Any RamCacheManager::kInvalidHandle
staticprivate

Definition at line 243 of file cache_ram.h.

Referenced by Close(), Dup(), GetSize(), Pread(), and Readahead().

uint64_t RamCacheManager::max_size_
private

Definition at line 296 of file cache_ram.h.

Referenced by CommitToKvStore().

MemoryKvStore RamCacheManager::regular_entries_
private

Definition at line 299 of file cache_ram.h.

Referenced by CommitToKvStore(), DoOpen(), GetStore(), and GetTransactionStore().

pthread_rwlock_t RamCacheManager::rwlock_
private
MemoryKvStore RamCacheManager::volatile_entries_
private

Definition at line 300 of file cache_ram.h.

Referenced by CommitToKvStore(), DoOpen(), GetStore(), and GetTransactionStore().


The documentation for this class was generated from the following files: