#include <fetch.h>

Inheritance diagram for cvmfs::Fetcher:

Collaboration diagram for cvmfs::Fetcher:

Classes
struct	ThreadLocalStorage

Public Member Functions
	Fetcher (CacheManager cache_mgr, download::DownloadManager download_mgr, BackoffThrottle *backoff_throttle, perf::StatisticsTemplate statistics)

	~Fetcher ()

int	Fetch (const CacheManager::LabeledObject &object, const std::string &alt_url="")

void	ReplaceCacheManager (CacheManager *new_cache_mgr)

CacheManager *	cache_mgr ()

download::DownloadManager *	download_mgr ()

Private Types
typedef std::map< shash::Any, std::vector< int > * >	ThreadQueues

Private Member Functions
	FRIEND_TEST (T_Fetcher, GetTls)

	FRIEND_TEST (T_Fetcher, SignalWaitingThreads)

ThreadLocalStorage *	GetTls ()

void	CleanupTls (ThreadLocalStorage *tls)

void	SignalWaitingThreads (const int fd, const shash::Any &id, ThreadLocalStorage *tls)

int	OpenSelect (const CacheManager::LabeledObject &object)

Private Member Functions inherited from SingleCopy
	SingleCopy ()

Private Attributes
pthread_key_t	thread_local_storage_

ThreadQueues	queues_download_

pthread_mutex_t *	lock_queues_download_

std::vector< ThreadLocalStorage * >	tls_blocks_

pthread_mutex_t *	lock_tls_blocks_

CacheManager *	cache_mgr_

download::DownloadManager *	download_mgr_

BackoffThrottle *	backoff_throttle_

perf::Counter *	n_downloads

perf::Counter *	n_invocations

Friends
void *	TestGetTls (void *data)

void *	TestFetchCollapse (void *data)

void *	TestFetchCollapse2 (void *data)

void	TLSDestructor (void *data)

Detailed Description

The Fetcher uses a cache manager and a download manager in order to provide a (virtual) file descriptor to a requested object, which is valid in the context of the cache manager. If the object is not in the cache, it is downloaded and stored in the cache.

Concurrent download requests for the same id are collapsed.

Definition at line 106 of file fetch.h.

Member Typedef Documentation

typedef std::map< shash::Any, std::vector<int> * > cvmfs::Fetcher::ThreadQueues

private

Maps currently downloaded chunks to the other_pipes_waiting member of the thread local storage of the downloading thread. This way, a thread can enqueue itself to such an other_pipes_waiting list and gets informed when the download is completed.

Definition at line 170 of file fetch.h.

Constructor & Destructor Documentation

cvmfs::Fetcher::Fetcher	(	CacheManager *	cache_mgr,
		download::DownloadManager *	download_mgr,
		BackoffThrottle *	backoff_throttle,
		perf::StatisticsTemplate	statistics
	)

Definition at line 208 of file fetch.cc.

Here is the call graph for this function:

cvmfs::Fetcher::~Fetcher ( )

Definition at line 237 of file fetch.cc.

Here is the call graph for this function:

Member Function Documentation

CacheManager* cvmfs::Fetcher::cache_mgr ( )

inline

Definition at line 127 of file fetch.h.

Referenced by catalog::ClientCatalogManager::GetNewRootCatalogContext(), catalog::ClientCatalogManager::LoadCatalogByHash(), catalog::ClientCatalogManager::StageNestedCatalogByHash(), catalog::ClientCatalogManager::UnloadCatalog(), and catalog::ClientCatalogManager::~ClientCatalogManager().

Here is the caller graph for this function:

void cvmfs::Fetcher::CleanupTls ( ThreadLocalStorage * tls )

private

Called when a thread exists, releases a ThreadLocalStorage object and removes the pointer to it from tls_blocks_.

Definition at line 51 of file fetch.cc.

Referenced by cvmfs::TLSDestructor(), and ~Fetcher().

Here is the call graph for this function:

Here is the caller graph for this function:

download::DownloadManager* cvmfs::Fetcher::download_mgr ( )

inline

Definition at line 128 of file fetch.h.

Referenced by catalog::ClientCatalogManager::GetNewRootCatalogContext().

Here is the caller graph for this function:

int cvmfs::Fetcher::Fetch	(	const CacheManager::LabeledObject &	object,
		const std::string &	alt_url = `""`
	)

Definition at line 81 of file fetch.cc.

Referenced by cvmfs::cvmfs_open(), cvmfs::cvmfs_read(), catalog::ClientCatalogManager::FetchCatalogByHash(), MountPoint::FetchHistory(), RepoMetainfoMagicXattr::FinalizeValue(), LibContext::Open(), cvmfs::Pin(), LibContext::Pread(), and catalog::ClientCatalogManager::StageNestedCatalogByHash().

Here is the call graph for this function:

Here is the caller graph for this function:

cvmfs::Fetcher::FRIEND_TEST	(	T_Fetcher	,
		GetTls
	)

private

cvmfs::Fetcher::FRIEND_TEST	(	T_Fetcher	,
		SignalWaitingThreads
	)

private

Fetcher::ThreadLocalStorage * cvmfs::Fetcher::GetTls ( )

private

Initialized thread-local storage if called the first time in a new thread.

Definition at line 60 of file fetch.cc.

Here is the call graph for this function:

int cvmfs::Fetcher::OpenSelect ( const CacheManager::LabeledObject & object )

private

Depending on the object type, uses either Open() or OpenPinned() from the cache manager

Definition at line 263 of file fetch.cc.

Here is the call graph for this function:

void cvmfs::Fetcher::ReplaceCacheManager ( CacheManager * new_cache_mgr )

inline

Definition at line 124 of file fetch.h.

Referenced by RestoreState().

Here is the caller graph for this function:

void cvmfs::Fetcher::SignalWaitingThreads	(	const int	fd,
		const shash::Any &	id,
		ThreadLocalStorage *	tls
	)

private

Definition at line 272 of file fetch.cc.

Here is the call graph for this function:

Friends And Related Function Documentation

void* TestFetchCollapse ( void * data )

friend

void* TestFetchCollapse2 ( void * data )

friend

void* TestGetTls ( void * data )

friend

void TLSDestructor ( void * data )

friend

Definition at line 25 of file fetch.cc.

Referenced by Fetcher().

Member Data Documentation

BackoffThrottle* cvmfs::Fetcher::backoff_throttle_

private

Definition at line 195 of file fetch.h.

CacheManager* cvmfs::Fetcher::cache_mgr_

private

Definition at line 193 of file fetch.h.

Referenced by cache_mgr(), OpenSelect(), ReplaceCacheManager(), and SignalWaitingThreads().

download::DownloadManager* cvmfs::Fetcher::download_mgr_

private

Definition at line 194 of file fetch.h.

Referenced by download_mgr().

pthread_mutex_t* cvmfs::Fetcher::lock_queues_download_

private

Definition at line 184 of file fetch.h.

Referenced by Fetcher(), SignalWaitingThreads(), and ~Fetcher().

pthread_mutex_t* cvmfs::Fetcher::lock_tls_blocks_

private

Definition at line 191 of file fetch.h.

Referenced by Fetcher(), cvmfs::TLSDestructor(), and ~Fetcher().

perf::Counter* cvmfs::Fetcher::n_downloads

private

Definition at line 196 of file fetch.h.

Referenced by Fetcher().

perf::Counter* cvmfs::Fetcher::n_invocations

private

Definition at line 197 of file fetch.h.

Referenced by Fetcher().

ThreadQueues cvmfs::Fetcher::queues_download_

private

Definition at line 183 of file fetch.h.

Referenced by SignalWaitingThreads().

pthread_key_t cvmfs::Fetcher::thread_local_storage_

private

Key to the thread's ThreadLocalStorage memory

Definition at line 181 of file fetch.h.

Referenced by Fetcher(), and ~Fetcher().

std::vector<ThreadLocalStorage *> cvmfs::Fetcher::tls_blocks_

private

All the threads register their thread local storage here, so that it can be cleaned up properly in the destructor of Fetcher.

Definition at line 190 of file fetch.h.

Referenced by cvmfs::TLSDestructor(), and ~Fetcher().

The documentation for this class was generated from the following files:

/home/sftnight/jenkins/workspace/CvmfsDoxygen/label/docker-x86_64/cvmfs/cvmfs/fetch.h
/home/sftnight/jenkins/workspace/CvmfsDoxygen/label/docker-x86_64/cvmfs/cvmfs/fetch.cc

Classes

Public Member Functions

Private Types

Private Member Functions

Private Attributes

Friends

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation