CernVM-FS  2.12.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Classes | Public Member Functions | Protected Types | Protected Member Functions | Private Attributes | List of all members
GarbageCollector< CatalogTraversalT, HashFilterT > Class Template Reference

#include <garbage_collector.h>

Collaboration diagram for GarbageCollector< CatalogTraversalT, HashFilterT >:
[legend]

Classes

struct  Configuration
 
class  ReflogBasedInfoShim
 

Public Member Functions

 GarbageCollector (const Configuration &configuration)
 
void UseReflogTimestamps ()
 
bool Collect ()
 
uint64_t preserved_catalog_count () const
 
uint64_t condemned_catalog_count () const
 
uint64_t condemned_objects_count () const
 
uint64_t duplicate_delete_requests () const
 
uint64_t condemned_bytes_count () const
 
uint64_t oldest_trunk_catalog () const
 

Protected Types

typedef
CatalogTraversalT::ObjectFetcherTN 		ObjectFetcherTN
 
typedef ObjectFetcherTN::HistoryTN HistoryTN
 
typedef ObjectFetcherTN::ReflogTN ReflogTN
 
typedef
CatalogTraversalT::CatalogTN 		CatalogTN
 
typedef
CatalogTraversalT::CallbackDataTN 		TraversalCallbackDataTN
 
typedef
CatalogTraversalT::Parameters 		TraversalParameters
 
typedef std::vector< shash::Any > HashVector
 

Protected Member Functions

TraversalParameters GetTraversalParams (const Configuration &configuration)
 
void PreserveDataObjects (const TraversalCallbackDataTN &data)
 
void SweepDataObjects (const TraversalCallbackDataTN &data)
 
bool AnalyzePreservedCatalogTree ()
 
bool CheckPreservedRevisions ()
 
bool SweepReflog ()
 
void CheckAndSweep (const shash::Any &hash)
 
void Sweep (const shash::Any &hash)
 
bool RemoveCatalogFromReflog (const shash::Any &catalog)
 
void PrintCatalogTreeEntry (const unsigned int tree_level, const CatalogTN *catalog) const
 
void LogDeletion (const shash::Any &hash) const
 

Private Attributes

const Configuration configuration_
 
ReflogBasedInfoShim catalog_info_shim_
 
CatalogTraversalT traversal_
 
HashFilterT hash_filter_
 
HashFilterT hash_map_delete_requests_
 
bool use_reflog_timestamps_
 
uint64_t oldest_trunk_catalog_
 
bool oldest_trunk_catalog_found_
 
uint64_t preserved_catalogs_
 
uint64_t unreferenced_trees_
 
uint64_t condemned_trees_
 
uint64_t condemned_catalogs_
 
float last_reported_status_
 
uint64_t condemned_objects_
 
uint64_t condemned_bytes_
 
uint64_t duplicate_delete_requests_
 

Detailed Description

template<class CatalogTraversalT, class HashFilterT>
class GarbageCollector< CatalogTraversalT, HashFilterT >

This file is part of the CernVM File System.

The GarbageCollector class is figuring out which data objects (represented by their content hashes) can be deleted as outdated garbage. Garbage collection is performed on the granularity of catalog revisions, thus a complete repository revision is either considered to be outdated or active. This way, a mountable repository revision stays completely usable (no nested catalogs or data objects become unavailable). A revision is defined by it's root catalog; the revision numbers of nested catalogs are irrelevant, since they might be referenced by newer (preserved) repository revisions. Thus, garbage objects are those that are not referenced by any of the pre- served root catalogs or their direct subordinate nested catalog try.

We use a two-stage approach:

1st Stage - Initialization The GarbageCollector is reading all the catalogs that are meant to be preserved. It builds up a filter (HashFilterT) containing all content hashes that are not to be deleted

2nd Stage - Sweeping The initialized HashFilterT is presented with all content hashes found in condemned catalogs and decides if they are referenced by the preserved catalog revisions or not.

The GarbageCollector is templated with CatalogTraversalT mainly for testability and with HashFilterT as an instance of the Strategy Pattern to abstract from the actual hash filtering method to be used.

Definition at line 45 of file garbage_collector.h.

Member Typedef Documentation

template<class CatalogTraversalT, class HashFilterT>
typedef CatalogTraversalT::CatalogTN GarbageCollector< CatalogTraversalT, HashFilterT >::CatalogTN
protected

Definition at line 50 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef std::vector<shash::Any> GarbageCollector< CatalogTraversalT, HashFilterT >::HashVector
protected

Definition at line 53 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef ObjectFetcherTN::HistoryTN GarbageCollector< CatalogTraversalT, HashFilterT >::HistoryTN
protected

Definition at line 48 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef CatalogTraversalT::ObjectFetcherTN GarbageCollector< CatalogTraversalT, HashFilterT >::ObjectFetcherTN
protected

Definition at line 47 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef ObjectFetcherTN::ReflogTN GarbageCollector< CatalogTraversalT, HashFilterT >::ReflogTN
protected

Definition at line 49 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef CatalogTraversalT::CallbackDataTN GarbageCollector< CatalogTraversalT, HashFilterT >::TraversalCallbackDataTN
protected

Definition at line 51 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
typedef CatalogTraversalT::Parameters GarbageCollector< CatalogTraversalT, HashFilterT >::TraversalParameters
protected

Definition at line 52 of file garbage_collector.h.

Constructor & Destructor Documentation

template<class CatalogTraversalT , class HashFilterT >
GarbageCollector< CatalogTraversalT, HashFilterT >::GarbageCollector ( const Configuration &  configuration)
explicit

Definition at line 33 of file garbage_collector_impl.h.

Here is the call graph for this function:

Member Function Documentation

template<class CatalogTraversalT , class HashFilterT >
bool GarbageCollector< CatalogTraversalT, HashFilterT >::AnalyzePreservedCatalogTree ( )
protected

Definition at line 231 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
void GarbageCollector< CatalogTraversalT, HashFilterT >::CheckAndSweep ( const shash::Any &  hash)
protected

Definition at line 172 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
bool GarbageCollector< CatalogTraversalT, HashFilterT >::CheckPreservedRevisions ( )
protected

Definition at line 255 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
bool GarbageCollector< CatalogTraversalT, HashFilterT >::Collect ( )

Definition at line 222 of file garbage_collector_impl.h.

Referenced by swissknife::CommandGc::Main().

Here is the caller graph for this function:

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_bytes_count ( ) const
inline

Definition at line 101 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_catalog_count ( ) const
inline

Definition at line 97 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_objects_count ( ) const
inline

Definition at line 98 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::duplicate_delete_requests ( ) const
inline

Definition at line 99 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
GarbageCollector< CatalogTraversalT, HashFilterT >::TraversalParameters GarbageCollector< CatalogTraversalT, HashFilterT >::GetTraversalParams ( const Configuration &  configuration)
protected

Definition at line 67 of file garbage_collector_impl.h.

template<class CatalogTraversalT , class HashFilterT >
void GarbageCollector< CatalogTraversalT, HashFilterT >::LogDeletion ( const shash::Any &  hash) const
protected

Definition at line 359 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::oldest_trunk_catalog ( ) const
inline

Definition at line 102 of file garbage_collector.h.

Referenced by swissknife::CommandGc::Main().

Here is the caller graph for this function:

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::preserved_catalog_count ( ) const
inline

Definition at line 96 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
void GarbageCollector< CatalogTraversalT, HashFilterT >::PreserveDataObjects ( const TraversalCallbackDataTN &  data)
protected

Definition at line 83 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
void GarbageCollector< CatalogTraversalT, HashFilterT >::PrintCatalogTreeEntry ( const unsigned int  tree_level,
const CatalogTN *  catalog 
) const
protected

Definition at line 335 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
bool GarbageCollector< CatalogTraversalT, HashFilterT >::RemoveCatalogFromReflog ( const shash::Any &  catalog)
protected

Definition at line 212 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
void GarbageCollector< CatalogTraversalT, HashFilterT >::Sweep ( const shash::Any &  hash)
protected

Definition at line 189 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT, class HashFilterT>
void GarbageCollector< CatalogTraversalT, HashFilterT >::SweepDataObjects ( const TraversalCallbackDataTN &  data)
protected

Definition at line 126 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
bool GarbageCollector< CatalogTraversalT, HashFilterT >::SweepReflog ( )
protected

Definition at line 268 of file garbage_collector_impl.h.

Here is the call graph for this function:

template<class CatalogTraversalT , class HashFilterT >
void GarbageCollector< CatalogTraversalT, HashFilterT >::UseReflogTimestamps ( )

Definition at line 59 of file garbage_collector_impl.h.

Referenced by swissknife::CommandGc::Main().

Here is the caller graph for this function:

Member Data Documentation

template<class CatalogTraversalT, class HashFilterT>
ReflogBasedInfoShim GarbageCollector< CatalogTraversalT, HashFilterT >::catalog_info_shim_
private

Definition at line 146 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_bytes_
private

Definition at line 181 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_bytes_count().

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_catalogs_
private

Number of catalogs garbage collected, count grows as GC progresses

Definition at line 174 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_catalog_count().

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_objects_
private

Definition at line 180 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_objects_count().

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::condemned_trees_
private

Number of root catalogs garbage collected, count grows as GC progresses

Definition at line 170 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
const Configuration GarbageCollector< CatalogTraversalT, HashFilterT >::configuration_
private

Definition at line 145 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::GarbageCollector().

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::duplicate_delete_requests_
private

Definition at line 182 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::duplicate_delete_requests().

template<class CatalogTraversalT, class HashFilterT>
HashFilterT GarbageCollector< CatalogTraversalT, HashFilterT >::hash_filter_
private

Definition at line 148 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
HashFilterT GarbageCollector< CatalogTraversalT, HashFilterT >::hash_map_delete_requests_
private

Definition at line 149 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
float GarbageCollector< CatalogTraversalT, HashFilterT >::last_reported_status_
private

Keeps track if the last status report issued, between 0 and 1

Definition at line 178 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::oldest_trunk_catalog_
private

A marker for the garbage collection grace period, the time span that is walked back from the current head catalog. There can be named snapshots older than this snapshot. The oldest_trunk_catalog_ is used as a marker for when to remove auxiliary files (meta info, history, ...).

Definition at line 159 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::oldest_trunk_catalog().

template<class CatalogTraversalT, class HashFilterT>
bool GarbageCollector< CatalogTraversalT, HashFilterT >::oldest_trunk_catalog_found_
private

Definition at line 160 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::preserved_catalogs_
private

Definition at line 161 of file garbage_collector.h.

Referenced by GarbageCollector< CatalogTraversalT, HashFilterT >::preserved_catalog_count().

template<class CatalogTraversalT, class HashFilterT>
CatalogTraversalT GarbageCollector< CatalogTraversalT, HashFilterT >::traversal_
private

Definition at line 147 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
uint64_t GarbageCollector< CatalogTraversalT, HashFilterT >::unreferenced_trees_
private

Number of catalogs in the reflog that are to be deleted (in fact, some of them might not exist anymore).

Definition at line 166 of file garbage_collector.h.

template<class CatalogTraversalT, class HashFilterT>
bool GarbageCollector< CatalogTraversalT, HashFilterT >::use_reflog_timestamps_
private

Definition at line 152 of file garbage_collector.h.

The documentation for this class was generated from the following files: