CernVM-FS  2.9.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
RamCacheManager Class Reference

#include <cache_ram.h>

Inheritance diagram for RamCacheManager:
Collaboration diagram for RamCacheManager:

Classes

struct  Counters
 
struct  ReadOnlyHandle
 
struct  Transaction
 

Public Member Functions

virtual CacheManagerIds id ()
 
virtual std::string Describe ()
 
 RamCacheManager (uint64_t max_size, unsigned max_entries, MemoryKvStore::MemoryAllocator alloc, perf::StatisticsTemplate statistics)
 
virtual ~RamCacheManager ()
 
virtual bool AcquireQuotaManager (QuotaManager *quota_mgr)
 
virtual int Open (const BlessedObject &object)
 
virtual int64_t GetSize (int fd)
 
virtual int Close (int fd)
 
virtual int64_t Pread (int fd, void *buf, uint64_t size, uint64_t offset)
 
virtual int Dup (int fd)
 
virtual int Readahead (int fd)
 
virtual uint32_t SizeOfTxn ()
 
virtual int StartTxn (const shash::Any &id, uint64_t size, void *txn)
 
virtual void CtrlTxn (const ObjectInfo &object_info, const int flags, void *txn)
 
virtual int64_t Write (const void *buf, uint64_t size, void *txn)
 
virtual int Reset (void *txn)
 
virtual int OpenFromTxn (void *txn)
 
virtual int AbortTxn (void *txn)
 
virtual int CommitTxn (void *txn)
 
virtual void Spawn ()
 
- Public Member Functions inherited from CacheManager
virtual ~CacheManager ()
 
int OpenPinned (const shash::Any &id, const std::string &description, bool is_catalog)
 
int ChecksumFd (int fd, shash::Any *id)
 
bool Open2Mem (const shash::Any &id, const std::string &description, unsigned char **buffer, uint64_t *size)
 
bool CommitFromMem (const shash::Any &id, const unsigned char *buffer, const uint64_t size, const std::string &description)
 
QuotaManagerquota_mgr ()
 
void * SaveState (const int fd_progress)
 
int RestoreState (const int fd_progress, void *state)
 
void FreeState (const int fd_progress, void *state)
 
virtual manifest::Breadcrumb LoadBreadcrumb (const std::string &)
 
virtual bool StoreBreadcrumb (const manifest::Manifest &)
 

Private Member Functions

MemoryKvStoreGetStore (const ReadOnlyHandle &fd)
 
MemoryKvStoreGetTransactionStore (Transaction *txn)
 
int AddFd (const ReadOnlyHandle &handle)
 
int64_t CommitToKvStore (Transaction *transaction)
 
virtual int DoOpen (const shash::Any &id)
 

Private Attributes

uint64_t max_size_
 
FdTable< ReadOnlyHandlefd_table_
 
pthread_rwlock_t rwlock_
 
MemoryKvStore regular_entries_
 
MemoryKvStore volatile_entries_
 
Counters counters_
 

Static Private Attributes

static const shash::Any kInvalidHandle
 

Additional Inherited Members

- Public Types inherited from CacheManager
enum  ObjectType { kTypeRegular = 0, kTypeCatalog, kTypePinned, kTypeVolatile }
 
- Static Public Member Functions inherited from CacheManager
static BlessedObject Bless (const shash::Any &id)
 
static BlessedObject Bless (const shash::Any &id, const ObjectInfo &info)
 
static BlessedObject Bless (const shash::Any &id, ObjectType type)
 
static BlessedObject Bless (const shash::Any &id, const std::string &description)
 
static BlessedObject Bless (const shash::Any &id, ObjectType type, const std::string &description)
 
- Static Public Attributes inherited from CacheManager
static const uint64_t kSizeUnknown = uint64_t(-1)
 
- Protected Member Functions inherited from CacheManager
 CacheManager ()
 
virtual void * DoSaveState ()
 
virtual int DoRestoreState (void *data)
 
virtual bool DoFreeState (void *data)
 
- Protected Attributes inherited from CacheManager
QuotaManagerquota_mgr_
 

Detailed Description

This file is part of the CernVM File System. The RamCacheManager class provides a cache backend that operates entirely from the host's RAM. This backend does not require any additional privileges on the host such as filesystem access. This cache resides in a single process' memory, and does not support sharing or persistence. This cache backend is a good choice in highly restrictive environments, e.g. HPC resources, where it is not possible to make use of a shared/local filesystem.

To use this cache backend, set CVMFS_CACHE_PRIMARY=ram. There are not many knobs to configure; at present only the size to be used for cached objects. CVMFS_CACHE_RAM_SIZE sets the amount of memory to use (in MB) for caching objects. If the size ends with a percent sign, the cache size will be set to that percentage of the system memory. If no size is specified, the RAM cache size defaults to ~3% of the system memory. The minimum cache size is 200 MB.

RamCacheManager uses a custom heap allocator rather than the system's libc malloc(). To switch to libc malloc, set CVMFS_CACHE_RAM_MALLOC=libc

Definition at line 44 of file cache_ram.h.

Constructor & Destructor Documentation

RamCacheManager::RamCacheManager ( uint64_t  max_size,
unsigned  max_entries,
MemoryKvStore::MemoryAllocator  alloc,
perf::StatisticsTemplate  statistics 
)

Definition at line 29 of file cache_ram.cc.

Here is the call graph for this function:

RamCacheManager::~RamCacheManager ( )
virtual

Definition at line 55 of file cache_ram.cc.

Member Function Documentation

int RamCacheManager::AbortTxn ( void *  txn)
virtual

Free the resources allocated for the pending transaction

Parameters
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 307 of file cache_ram.cc.

Here is the call graph for this function:

bool RamCacheManager::AcquireQuotaManager ( QuotaManager quota_mgr)
virtual

Implements CacheManager.

Definition at line 70 of file cache_ram.cc.

Referenced by FileSystem::SetupRamCacheMgr().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::AddFd ( const ReadOnlyHandle handle)
private

Definition at line 60 of file cache_ram.cc.

Referenced by DoOpen(), and Dup().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::Close ( int  fd)
virtual

Close the descriptor in the cache. Entries not marked as pinned will become subject to eviction once closed

Parameters
idThe hash key
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 133 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::CommitToKvStore ( Transaction transaction)
private

Definition at line 328 of file cache_ram.cc.

Referenced by CommitTxn(), and OpenFromTxn().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::CommitTxn ( void *  txn)
virtual

Commit a transaction to the cache. If there is not enough free space in the cache, first try to make room by evicting volatile entries. If there is still not enough room, try evicting regular entries. If there is still not enough space, give up an return failure.

Parameters
txnA pointer to space allocated for storing the transaction details
Return values
-ENOSPCThe transaction would exceed the size of the cache
-ENFILENo handles are available
-EEXISTAn entry with nonzero reference count already exists

Implements CacheManager.

Definition at line 317 of file cache_ram.cc.

Here is the call graph for this function:

void RamCacheManager::CtrlTxn ( const ObjectInfo object_info,
const int  flags,
void *  txn 
)
virtual

Set the transaction parameters. At present, only type is used for pinned, volatile, etc.

Parameters
descriptionUnused
typeThe type of the entry, e.g. pinned
flagsUnused
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 226 of file cache_ram.cc.

Here is the call graph for this function:

string RamCacheManager::Describe ( )
virtual

Return a human readable description of the cache instance. Used in cvmfs_talk.

Implements CacheManager.

Definition at line 23 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::DoOpen ( const shash::Any id)
privatevirtual

Definition at line 84 of file cache_ram.cc.

Referenced by Open(), and OpenFromTxn().

Here is the call graph for this function:

Here is the caller graph for this function:

int RamCacheManager::Dup ( int  fd)
virtual

Duplicates the open file descriptor, allowing the original and the new one to be used independently

Parameters
idThe hash key
Returns
A new, nonnegative integer fd
Return values
-EBADFfd is not valid
-ENFILENo handles are available

Implements CacheManager.

Definition at line 171 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::GetSize ( int  fd)
virtual

Look up the size in bytes of the open cache entry

Parameters
idThe hash key
Returns
The size of the entry
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 121 of file cache_ram.cc.

Here is the call graph for this function:

MemoryKvStore* RamCacheManager::GetStore ( const ReadOnlyHandle fd)
inlineprivate

Definition at line 278 of file cache_ram.h.

Referenced by Close(), DoOpen(), Dup(), GetSize(), and Pread().

Here is the caller graph for this function:

MemoryKvStore* RamCacheManager::GetTransactionStore ( Transaction txn)
inlineprivate

Definition at line 286 of file cache_ram.h.

virtual CacheManagerIds RamCacheManager::id ( )
inlinevirtual

Implements CacheManager.

Definition at line 103 of file cache_ram.h.

Referenced by StartTxn().

Here is the caller graph for this function:

int RamCacheManager::Open ( const BlessedObject object)
virtual

Open a new file descriptor into the cache. Note that opening entries effectively pins them in the cache, so it may be necessary to close unneeded file descriptors to allow eviction to make room in the cache

Parameters
objectThe blessed hash key
Returns
A nonnegative integer file descriptor
Return values
-ENOENTThe entry is not in the cache
-ENFILENo handles are available

Implements CacheManager.

Definition at line 78 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::OpenFromTxn ( void *  txn)
virtual

Commit a transaction and open the resulting cache entry. This is necessary to avoid a race condition in which the cache entry is evicted between the calls to CommitTxn and Open.

Parameters
txnA pointer to space allocated for storing the transaction details
Returns
A file descriptor to the new cache entry, or a negative error from CommitTxn

Implements CacheManager.

Definition at line 290 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::Pread ( int  fd,
void *  buf,
uint64_t  size,
uint64_t  offset 
)
virtual

Read a section from the cache entry. See pread(3) for a discussion of the arguments

Parameters
idThe hash key
Returns
The number of bytes copied
Return values
-EBADFfd is not valid

Implements CacheManager.

Definition at line 153 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::Readahead ( int  fd)
virtual

No effect for in-memory caches

Parameters
idThe hash key
Return values
-EBADFfd is not valid

For a RAM cache, read-ahead is a no-op.

Implements CacheManager.

Definition at line 193 of file cache_ram.cc.

Here is the call graph for this function:

int RamCacheManager::Reset ( void *  txn)
virtual

Seek to the beginning of the transaction buffer

Parameters
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 280 of file cache_ram.cc.

Here is the call graph for this function:

virtual uint32_t RamCacheManager::SizeOfTxn ( )
inlinevirtual

Get the amount of space to be allocated for a call to StartTxn

Implements CacheManager.

Definition at line 173 of file cache_ram.h.

Here is the call graph for this function:

virtual void RamCacheManager::Spawn ( )
inlinevirtual

Implements CacheManager.

Definition at line 240 of file cache_ram.h.

int RamCacheManager::StartTxn ( const shash::Any id,
uint64_t  size,
void *  txn 
)
virtual

Start a new transaction, allocating the required memory and initializing the transaction parameters

Parameters
idThe hash key
sizeThe total size of the new cache entry
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 206 of file cache_ram.cc.

Here is the call graph for this function:

int64_t RamCacheManager::Write ( const void *  buf,
uint64_t  size,
void *  txn 
)
virtual

Copy the given memory region into the transaction buffer. Copying starts at the transaction's current offset

Parameters
bufThe source address
sizeThe number of bytes to copy
txnA pointer to space allocated for storing the transaction details

Implements CacheManager.

Definition at line 239 of file cache_ram.cc.

Here is the call graph for this function:

Member Data Documentation

Counters RamCacheManager::counters_
private
FdTable<ReadOnlyHandle> RamCacheManager::fd_table_
private

Definition at line 299 of file cache_ram.h.

Referenced by AddFd(), Close(), Dup(), GetSize(), Pread(), and Readahead().

const shash::Any RamCacheManager::kInvalidHandle
staticprivate

Definition at line 245 of file cache_ram.h.

Referenced by Close(), Dup(), GetSize(), Pread(), and Readahead().

uint64_t RamCacheManager::max_size_
private

Definition at line 298 of file cache_ram.h.

Referenced by CommitToKvStore().

MemoryKvStore RamCacheManager::regular_entries_
private

Definition at line 301 of file cache_ram.h.

Referenced by CommitToKvStore(), DoOpen(), GetStore(), and GetTransactionStore().

pthread_rwlock_t RamCacheManager::rwlock_
private
MemoryKvStore RamCacheManager::volatile_entries_
private

Definition at line 302 of file cache_ram.h.

Referenced by CommitToKvStore(), DoOpen(), GetStore(), and GetTransactionStore().


The documentation for this class was generated from the following files: