CernVM-FS  2.9.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
CacheManager Class Referenceabstract

#include <cache.h>

Inheritance diagram for CacheManager:
Collaboration diagram for CacheManager:

Classes

struct  BlessedObject
 
struct  ObjectInfo
 
struct  State
 

Public Types

enum  ObjectType { kTypeRegular = 0, kTypeCatalog, kTypePinned, kTypeVolatile }
 

Public Member Functions

virtual CacheManagerIds id ()=0
 
virtual std::string Describe ()=0
 
virtual bool AcquireQuotaManager (QuotaManager *quota_mgr)=0
 
virtual ~CacheManager ()
 
virtual int Open (const BlessedObject &object)=0
 
virtual int64_t GetSize (int fd)=0
 
virtual int Close (int fd)=0
 
virtual int64_t Pread (int fd, void *buf, uint64_t size, uint64_t offset)=0
 
virtual int Dup (int fd)=0
 
virtual int Readahead (int fd)=0
 
virtual uint32_t SizeOfTxn ()=0
 
virtual int StartTxn (const shash::Any &id, uint64_t size, void *txn)=0
 
virtual void CtrlTxn (const ObjectInfo &object_info, const int flags, void *txn)=0
 
virtual int64_t Write (const void *buf, uint64_t sz, void *txn)=0
 
virtual int Reset (void *txn)=0
 
virtual int AbortTxn (void *txn)=0
 
virtual int OpenFromTxn (void *txn)=0
 
virtual int CommitTxn (void *txn)=0
 
virtual void Spawn ()=0
 
int OpenPinned (const shash::Any &id, const std::string &description, bool is_catalog)
 
int ChecksumFd (int fd, shash::Any *id)
 
bool Open2Mem (const shash::Any &id, const std::string &description, unsigned char **buffer, uint64_t *size)
 
bool CommitFromMem (const shash::Any &id, const unsigned char *buffer, const uint64_t size, const std::string &description)
 
QuotaManagerquota_mgr ()
 
void * SaveState (const int fd_progress)
 
int RestoreState (const int fd_progress, void *state)
 
void FreeState (const int fd_progress, void *state)
 
virtual manifest::Breadcrumb LoadBreadcrumb (const std::string &)
 
virtual bool StoreBreadcrumb (const manifest::Manifest &)
 

Static Public Member Functions

static BlessedObject Bless (const shash::Any &id)
 
static BlessedObject Bless (const shash::Any &id, const ObjectInfo &info)
 
static BlessedObject Bless (const shash::Any &id, ObjectType type)
 
static BlessedObject Bless (const shash::Any &id, const std::string &description)
 
static BlessedObject Bless (const shash::Any &id, ObjectType type, const std::string &description)
 

Static Public Attributes

static const uint64_t kSizeUnknown = uint64_t(-1)
 

Protected Member Functions

 CacheManager ()
 
virtual void * DoSaveState ()
 
virtual int DoRestoreState (void *data)
 
virtual bool DoFreeState (void *data)
 

Protected Attributes

QuotaManagerquota_mgr_
 

Static Private Attributes

static const unsigned kStateVersion = 0
 

Additional Inherited Members

- Private Member Functions inherited from SingleCopy
 SingleCopy ()
 

Detailed Description

The Cache Manager provides (virtual) file descriptors to content-addressable objects in the cache. The implementation can use a POSIX file system or other means such as a key-value store. A file descriptor must remain readable until closed, no matter if it is removed from the backend storage or not (POSIX semantics).

Writing into the cache is streamed and transactional: a file descriptor must be acquired from StartTxn and the object is only visible in the cache after CommitTxn. The state of the transaction is carried in an opque transaction object, which needs to be provided by the caller. The size of the object is returned by SizeOfTxn. This way, users of derived classes can take care of the storage allocation (e.g. on the stack), while the derived class determines the contents of the transaction object. For race-free read access to objects that are just being written to the cache, the OpenFromTxn routine is used just before the transaction is committed.

Writing to the cache can be coupled to a quota manager. The quota manager maintains some extra information for data chunks: whether they are volatile, whether they are pinned, and a description (usually the path that corresponds to a data chunk). By default the NoopQuotaManager is used, which ignores all this extra information. The CtrlTxn() function is used to specify this extra information sometime between StartTxn() and CommitTxn().

The integer return values can be negative and, in this case, represent a -errno failure code. Otherwise the return value 0 indicates a success, or >= 0 for a file descriptor.

Definition at line 65 of file cache.h.

Member Enumeration Documentation

Relevant for the quota management

Enumerator
kTypeRegular 
kTypeCatalog 
kTypePinned 
kTypeVolatile 

Definition at line 77 of file cache.h.

Constructor & Destructor Documentation

CacheManager::~CacheManager ( )
virtual

Definition at line 30 of file cache.cc.

CacheManager::CacheManager ( )
protected

Definition at line 27 of file cache.cc.

Member Function Documentation

virtual int CacheManager::AbortTxn ( void *  txn)
pure virtual
virtual bool CacheManager::AcquireQuotaManager ( QuotaManager quota_mgr)
pure virtual
static BlessedObject CacheManager::Bless ( const shash::Any id)
inlinestatic

Definition at line 125 of file cache.h.

Referenced by LHashMagicXattr::GetValue(), Open2Mem(), OpenPinned(), and cvmfs::Fetcher::OpenSelect().

Here is the caller graph for this function:

static BlessedObject CacheManager::Bless ( const shash::Any id,
const ObjectInfo info 
)
inlinestatic

Definition at line 128 of file cache.h.

static BlessedObject CacheManager::Bless ( const shash::Any id,
ObjectType  type 
)
inlinestatic

Definition at line 134 of file cache.h.

static BlessedObject CacheManager::Bless ( const shash::Any id,
const std::string &  description 
)
inlinestatic

Definition at line 137 of file cache.h.

static BlessedObject CacheManager::Bless ( const shash::Any id,
ObjectType  type,
const std::string &  description 
)
inlinestatic

Definition at line 143 of file cache.h.

int CacheManager::ChecksumFd ( int  fd,
shash::Any id 
)

Compresses and checksums the file pointed to by fd. The hash algorithm needs to be set in id.

Definition at line 39 of file cache.cc.

Referenced by LHashMagicXattr::GetValue().

Here is the call graph for this function:

Here is the caller graph for this function:

bool CacheManager::CommitFromMem ( const shash::Any id,
const unsigned char *  buffer,
const uint64_t  size,
const std::string &  description 
)

Commits the memory blob buffer to the given chunk id. No checking! The hash and the memory blob need to match.

Definition at line 79 of file cache.cc.

Referenced by catalog::ClientCatalogManager::LoadCatalog().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual int CacheManager::CommitTxn ( void *  txn)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by CommitFromMem(), TieredCacheManager::CommitTxn(), and TieredCacheManager::Open().

Here is the caller graph for this function:

virtual void CacheManager::CtrlTxn ( const ObjectInfo object_info,
const int  flags,
void *  txn 
)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by CommitFromMem(), TieredCacheManager::CtrlTxn(), and TieredCacheManager::Open().

Here is the caller graph for this function:

virtual std::string CacheManager::Describe ( )
pure virtual

Return a human readable description of the cache instance. Used in cvmfs_talk.

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by TieredCacheManager::Describe(), and TalkManager::MainResponder().

Here is the caller graph for this function:

virtual bool CacheManager::DoFreeState ( void *  data)
inlineprotectedvirtual

Reimplemented in PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Definition at line 232 of file cache.h.

Referenced by FreeState().

Here is the caller graph for this function:

virtual int CacheManager::DoRestoreState ( void *  data)
inlineprotectedvirtual

Reimplemented in PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Definition at line 231 of file cache.h.

Referenced by RestoreState().

Here is the caller graph for this function:

virtual void* CacheManager::DoSaveState ( )
inlineprotectedvirtual

Reimplemented in PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Definition at line 230 of file cache.h.

Referenced by SaveState().

Here is the caller graph for this function:

virtual int CacheManager::Dup ( int  fd)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by TieredCacheManager::Dup(), and cvmfs::Fetcher::SignalWaitingThreads().

Here is the caller graph for this function:

void CacheManager::FreeState ( const int  fd_progress,
void *  state 
)

Definition at line 100 of file cache.cc.

Referenced by TieredCacheManager::DoFreeState(), and FreeSavedState().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual int64_t CacheManager::GetSize ( int  fd)
pure virtual
virtual CacheManagerIds CacheManager::id ( )
pure virtual
virtual manifest::Breadcrumb CacheManager::LoadBreadcrumb ( const std::string &  )
inlinevirtual

While not strictly necessary, cache managers often have a directory associated with them. This directory is currently used to find the cached manifest copy, the cvmfschecksum.$reponame file. This is important to make pre-loaded alien caches work, even in a tiered setup.

Reimplemented in PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Definition at line 219 of file cache.h.

Referenced by TieredCacheManager::LoadBreadcrumb(), and catalog::ClientCatalogManager::LoadCatalog().

Here is the caller graph for this function:

virtual int CacheManager::Open ( const BlessedObject object)
pure virtual

Opening an object might get it from a third-party source, e.g. when the tiered cache manager issues a copy-up operation. In this case it is beneficial to register the object with the accurate meta-data, in the same way it is done during transactions.

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by LHashMagicXattr::GetValue(), TieredCacheManager::Open(), Open2Mem(), OpenPinned(), and cvmfs::Fetcher::OpenSelect().

Here is the caller graph for this function:

bool CacheManager::Open2Mem ( const shash::Any id,
const std::string &  description,
unsigned char **  buffer,
uint64_t *  size 
)

Tries to open a file and copies its contents into a newly malloc'd memory area. User of the function has to free buffer (if successful).

Parameters
[in]idcontent hash of the catalog entry.
[out]bufferContents of the file
[out]sizeSize of the file
Returns
True if successful, false otherwise.

Definition at line 127 of file cache.cc.

Referenced by catalog::CachedManifestEnsemble::FetchCertificate().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual int CacheManager::OpenFromTxn ( void *  txn)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by TieredCacheManager::Open(), and TieredCacheManager::OpenFromTxn().

Here is the caller graph for this function:

int CacheManager::OpenPinned ( const shash::Any id,
const std::string &  description,
bool  is_catalog 
)

Uses the regular open and, if the file exists in the cache, pins it. There is a race condition: the file can be removed between the open and the Pin. This is fixed by the quota manager's unpin method that removes files which do not exist anymore in the cache. (The quota manager also translates double pins into a no-op, so that the accounting does not get out of sync.)

Definition at line 170 of file cache.cc.

Referenced by cvmfs::Fetcher::OpenSelect().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual int64_t CacheManager::Pread ( int  fd,
void *  buf,
uint64_t  size,
uint64_t  offset 
)
pure virtual
virtual int CacheManager::Readahead ( int  fd)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by TieredCacheManager::Readahead(), and sqlite::VfsRdOnlyOpen().

Here is the caller graph for this function:

virtual int CacheManager::Reset ( void *  txn)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by cvmfs::TransactionSink::Reset(), and TieredCacheManager::Reset().

Here is the caller graph for this function:

int CacheManager::RestoreState ( const int  fd_progress,
void *  state 
)

When RestoreState is called, the cache has already exactly one file descriptor open: the root file catalog. This file descriptor might be remapped to another number. A return value of -1 means no action needs to take place. A smaller value inidicates an error.

Definition at line 194 of file cache.cc.

Referenced by TieredCacheManager::DoRestoreState(), and RestoreState().

Here is the call graph for this function:

Here is the caller graph for this function:

void * CacheManager::SaveState ( const int  fd_progress)

The actual work is done in the concrete cache managers.

Definition at line 221 of file cache.cc.

Referenced by TieredCacheManager::DoSaveState(), and SaveState().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void CacheManager::Spawn ( )
pure virtual

Implemented in RamCacheManager, ExternalCacheManager, PosixCacheManager, and TieredCacheManager.

Referenced by TieredCacheManager::Spawn(), and Spawn().

Here is the caller graph for this function:

virtual int CacheManager::StartTxn ( const shash::Any id,
uint64_t  size,
void *  txn 
)
pure virtual

Implemented in RamCacheManager, PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Referenced by CommitFromMem(), TieredCacheManager::Open(), and TieredCacheManager::StartTxn().

Here is the caller graph for this function:

virtual bool CacheManager::StoreBreadcrumb ( const manifest::Manifest )
inlinevirtual

Reimplemented in PosixCacheManager, ExternalCacheManager, and TieredCacheManager.

Definition at line 222 of file cache.h.

Referenced by catalog::ClientCatalogManager::LoadCatalog(), and TieredCacheManager::StoreBreadcrumb().

Here is the caller graph for this function:

virtual int64_t CacheManager::Write ( const void *  buf,
uint64_t  sz,
void *  txn 
)
pure virtual

Member Data Documentation

const uint64_t CacheManager::kSizeUnknown = uint64_t(-1)
static

Sizes of objects should be known for StartTxn(). For file catalogs we cannot ensure that, however, because the size field for nested catalogs was only recently added.

Definition at line 72 of file cache.h.

Referenced by MountPoint::FetchHistory(), RepoMetainfoMagicXattr::GetValue(), catalog::ClientCatalogManager::LoadCatalogCas(), RamCacheManager::StartTxn(), ExternalCacheManager::Write(), and RamCacheManager::Write().

const unsigned CacheManager::kStateVersion = 0
staticprivate

Definition at line 240 of file cache.h.

Referenced by FreeState(), and RestoreState().


The documentation for this class was generated from the following files: