#include <pack.h>

Inheritance diagram for ObjectPack:

Collaboration diagram for ObjectPack:

Classes
struct	Bucket

Public Types
enum	BucketContentType { kEmpty, kNamed, kCas }

typedef Bucket *	BucketHandle

Public Member Functions
	ObjectPack (const uint64_t limit=kDefaultLimit)

	~ObjectPack ()

BucketHandle	NewBucket ()

bool	CommitBucket (const BucketContentType type, const shash::Any &id, const BucketHandle handle, const std::string &name="")

void	DiscardBucket (const BucketHandle handle)

void	TransferBucket (const BucketHandle handle, ObjectPack *other)

unsigned char *	BucketContent (size_t idx) const

uint64_t	BucketSize (size_t idx) const

const shash::Any &	BucketId (size_t idx) const

uint64_t	size () const

size_t	GetNoObjects () const

Static Public Member Functions
static void	AddToBucket (const void *buf, const uint64_t size, const BucketHandle handle)

Static Public Attributes
static const uint64_t	kDefaultLimit = 200 * 1024 * 1024

static const uint64_t	kMaxObjects = 100000

Private Member Functions
	FRIEND_TEST (T_Pack, Bucket)

	FRIEND_TEST (T_Pack, ObjectPack)

	FRIEND_TEST (T_Pack, ObjectPackTransfer)

void	InitLock ()

Private Member Functions inherited from SingleCopy
	SingleCopy ()

Private Attributes
pthread_mutex_t *	lock_

uint64_t	limit_

uint64_t	size_

std::set< BucketHandle >	open_buckets_

std::vector< BucketHandle >	buckets_

Detailed Description

This file is part of the CernVM File System. Multiple content-addressable objects in a single BLOB. A (serialized) ObjectPack has a header, an index containing all the objects and their offsets followed by the concatenated objects. The secure hash of the index is in the header.

This allows to verify the hash of the index and the hash of all objects individually. Thus, objects can be read and written in parallel to and from the ObjectPack.

Objects are used by "tentacles" to send change sets to the "octopus server" as well as by the stratum 0 to transfer object bulks to stratum 1s during replication.

Definition at line 35 of file pack.h.

Member Typedef Documentation

typedef Bucket* ObjectPack::BucketHandle

Definition at line 41 of file pack.h.

Member Enumeration Documentation

enum ObjectPack::BucketContentType

This is used to identify the content type of different buckets. Initially, the contents of a bucket are identified as kEmpty. When committing a bucket, this is set to either kNamed - if the bucket holds the contents of a named file - or kCas - if the bucket holds the contents of a content addressable buffer.

Enumerator
kEmpty
kNamed
kCas

Definition at line 53 of file pack.h.

Constructor & Destructor Documentation

ObjectPack::ObjectPack ( const uint64_t limit = kDefaultLimit )

explicit

Definition at line 81 of file pack.cc.

Here is the call graph for this function:

ObjectPack::~ObjectPack ( )

Definition at line 85 of file pack.cc.

Member Function Documentation

void ObjectPack::AddToBucket	(	const void *	buf,
		const uint64_t	size,
		const BucketHandle	handle
	)

static

Definition at line 97 of file pack.cc.

Referenced by upload::GatewayUploader::DoUpload(), and upload::GatewayUploader::StreamedUpload().

Here is the call graph for this function:

Here is the caller graph for this function:

unsigned char * ObjectPack::BucketContent ( size_t idx ) const

Definition at line 156 of file pack.cc.

Referenced by ObjectPackProducer::ProduceNext().

Here is the call graph for this function:

Here is the caller graph for this function:

const shash::Any & ObjectPack::BucketId ( size_t idx ) const

Definition at line 166 of file pack.cc.

Referenced by ObjectPackProducer::ObjectPackProducer().

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t ObjectPack::BucketSize ( size_t idx ) const

Definition at line 161 of file pack.cc.

Referenced by ObjectPackProducer::ObjectPackProducer(), and ObjectPackProducer::ProduceNext().

Here is the call graph for this function:

Here is the caller graph for this function:

bool ObjectPack::CommitBucket	(	const BucketContentType	type,
		const shash::Any &	id,
		const BucketHandle	handle,
		const std::string &	name = `""`
	)

Can only fail due to insufficient remaining space in the ObjectPack.

Definition at line 113 of file pack.cc.

Referenced by upload::SessionContextBase::CommitBucket().

Here is the caller graph for this function:

void ObjectPack::DiscardBucket ( const BucketHandle handle )

Definition at line 133 of file pack.cc.

ObjectPack::FRIEND_TEST	(	T_Pack	,
		Bucket
	)

private

ObjectPack::FRIEND_TEST	(	T_Pack	,
		ObjectPack
	)

private

ObjectPack::FRIEND_TEST	(	T_Pack	,
		ObjectPackTransfer
	)

private

size_t ObjectPack::GetNoObjects ( ) const

inline

Definition at line 86 of file pack.h.

Referenced by upload::SessionContextBase::CommitBucket(), upload::SessionContextBase::Finalize(), and ObjectPackProducer::ObjectPackProducer().

Here is the caller graph for this function:

void ObjectPack::InitLock ( )

private

Definition at line 139 of file pack.cc.

Referenced by ObjectPack().

Here is the call graph for this function:

Here is the caller graph for this function:

ObjectPack::BucketHandle ObjectPack::NewBucket ( )

Definition at line 102 of file pack.cc.

Referenced by upload::SessionContextBase::NewBucket().

Here is the caller graph for this function:

uint64_t ObjectPack::size ( ) const

inline

Definition at line 82 of file pack.h.

Referenced by ObjectPack::Bucket::Add(), upload::SessionContextBase::CommitBucket(), upload::SessionContextBase::Dispatch(), upload::SessionContext::DoUpload(), and ObjectPackProducer::ObjectPackProducer().

Here is the caller graph for this function:

void ObjectPack::TransferBucket	(	const BucketHandle	handle,
		ObjectPack *	other
	)

If a commit failed, an open Bucket can be transferred to another ObjectPack with more space.

Definition at line 149 of file pack.cc.

Referenced by upload::SessionContextBase::CommitBucket().

Here is the caller graph for this function:

Member Data Documentation

std::vector<BucketHandle> ObjectPack::buckets_

private

Buckets that are committed to the object pack.

Definition at line 130 of file pack.h.

Referenced by BucketContent(), BucketId(), BucketSize(), CommitBucket(), GetNoObjects(), and ~ObjectPack().

const uint64_t ObjectPack::kDefaultLimit = 200 * 1024 * 1024

static

Definition at line 55 of file pack.h.

const uint64_t ObjectPack::kMaxObjects = 100000

static

Limit the maximum number of objects to avoid very large headers. Assuming Sha256 (71 bytes hex) + 9 bytes for the file sizes, a header with 100,000 files should fit in 10M.

Definition at line 62 of file pack.h.

Referenced by CommitBucket(), and ObjectPackConsumer::ObjectPackConsumer().

uint64_t ObjectPack::limit_

private

Maximum size of this object pack.

Definition at line 118 of file pack.h.

Referenced by CommitBucket().

pthread_mutex_t* ObjectPack::lock_

private

Protects open_buckets_ and buckets_ collections.

Definition at line 113 of file pack.h.

Referenced by CommitBucket(), DiscardBucket(), InitLock(), NewBucket(), TransferBucket(), and ~ObjectPack().

std::set<BucketHandle> ObjectPack::open_buckets_

private

Buckets that were requested but that are not yet committed

Definition at line 126 of file pack.h.

Referenced by CommitBucket(), DiscardBucket(), NewBucket(), TransferBucket(), and ~ObjectPack().

uint64_t ObjectPack::size_

private

Accumulated size of all committed buckets.

Definition at line 122 of file pack.h.

Referenced by CommitBucket(), and size().

The documentation for this class was generated from the following files:

/home/sftnight/jenkins/workspace/CvmfsDoxygen/label/docker-x86_64/cvmfs/cvmfs/pack.h
/home/sftnight/jenkins/workspace/CvmfsDoxygen/label/docker-x86_64/cvmfs/cvmfs/pack.cc

Classes

Public Types

Public Member Functions

Static Public Member Functions

Static Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

Member Enumeration Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation