CernVM-FS  2.12.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
publish::SyncItem Class Referenceabstract

#include <sync_item.h>

Inheritance diagram for publish::SyncItem:
Collaboration diagram for publish::SyncItem:

Classes

struct  EntryStat
 

Public Member Functions

 SyncItem ()
 
virtual ~SyncItem ()
 
bool IsDirectory () const
 
bool WasDirectory () const
 
bool IsRegularFile () const
 
bool WasRegularFile () const
 
bool IsSymlink () const
 
bool WasSymlink () const
 
bool IsNew () const
 
bool IsTouched () const
 
bool IsCharacterDevice () const
 
bool IsBlockDevice () const
 
bool IsFifo () const
 
bool IsSocket () const
 
bool IsGraftMarker () const
 
bool IsExternalData () const
 
bool IsDirectIo () const
 
bool IsWhiteout () const
 
bool IsCatalogMarker () const
 
bool IsOpaqueDirectory () const
 
bool IsSpecialFile () const
 
bool WasSpecialFile () const
 
bool IsBundleSpec () const
 
bool WasBundleSpec () const
 
unsigned int GetRdevMajor () const
 
unsigned int GetRdevMinor () const
 
bool HasCatalogMarker () const
 
bool HasGraftMarker () const
 
bool HasCompressionAlgorithm () const
 
bool IsValidGraft () const
 
bool IsChunkedGraft () const
 
const FileChunkListGetGraftChunks () const
 
shash::Any GetContentHash () const
 
void SetContentHash (const shash::Any &hash)
 
bool HasContentHash () const
 
void SetExternalData (bool val)
 
void SetDirectIo (bool val)
 
zlib::Algorithms GetCompressionAlgorithm () const
 
void SetCompressionAlgorithm (const zlib::Algorithms &alg)
 
virtual catalog::DirectoryEntryBase CreateBasicCatalogDirent () const =0
 
std::string GetRelativePath () const
 
std::string GetRdOnlyPath () const
 
std::string GetUnionPath () const
 
std::string GetScratchPath () const
 
void MarkAsWhiteout (const std::string &actual_filename)
 
void MarkAsOpaqueDirectory ()
 
void MaskHardlink ()
 
bool HasHardlinks () const
 
unsigned int GetRdOnlyLinkcount () const
 
uint64_t GetRdOnlyInode () const
 
unsigned int GetUnionLinkcount () const
 
uint64_t GetUnionInode () const
 
uint64_t GetScratchSize () const
 
uint64_t GetRdOnlySize () const
 
std::string filename () const
 
std::string relative_parent_path () const
 
virtual IngestionSourceCreateIngestionSource () const =0
 
virtual void MakePlaceholderDirectory () const =0
 
void SetCatalogMarker ()
 
bool operator== (const SyncItem &other) const
 

Protected Member Functions

 SyncItem (const std::string &relative_parent_path, const std::string &filename, const SyncUnion *union_engine, const SyncItemType entry_type)
 
platform_stat64 GetUnionStat () const
 
SyncItemType GetRdOnlyFiletype () const
 
SyncItemType GetUnionFiletype () const
 
virtual SyncItemType GetScratchFiletype () const =0
 
virtual bool IsType (const SyncItemType expected_type) const =0
 
bool WasType (const SyncItemType expected_type) const
 
SyncItemType GetGenericFiletype (const EntryStat &stat) const
 
void CheckMarkerFiles ()
 

Static Protected Member Functions

static void StatGeneric (const std::string &path, EntryStat *info, const bool refresh)
 

Protected Attributes

SyncItemType rdonly_type_
 
EntryStat scratch_stat_
 
ssize_t graft_size_
 
shash::Any content_hash_
 
SyncItemType scratch_type_
 

Private Member Functions

void CheckCatalogMarker ()
 
std::string GetGraftMarkerPath () const
 
void CheckGraft ()
 
void StatRdOnly (const bool refresh=false) const
 
void StatUnion (const bool refresh=false) const
 
virtual void StatScratch (const bool refresh) const =0
 

Private Attributes

std::string filename_
 
const SyncUnionunion_engine_
 
EntryStat rdonly_stat_
 
EntryStat union_stat_
 
bool whiteout_
 
bool opaque_
 
bool masked_hardlink_
 
bool has_catalog_marker_
 
bool valid_graft_
 
bool graft_marker_present_
 
bool external_data_
 
bool direct_io_
 
std::string relative_parent_path_
 
FileChunkListgraft_chunklist_
 
zlib::Algorithms compression_algorithm_
 
bool has_compression_algorithm_
 

Detailed Description

Every directory entry emitted by the FileSystemTraversal is wrapped in a SyncItem structure by the factory method SyncUnion::CreateSyncItem().

Since we are dealing with a union file system setup, this class represents potentially three concrete files:

  • <read-only path>/<filename> | cf. rdonly_stat_
  • <scratch (read-write) branch>/<filename> | cf. scratch_stat_
  • <union volume="" path>="">/<filename> | cf. union_stat_

This class caches stat calls to the underlying files in different branches of the union file system and hides some interpretation details.

Definition at line 56 of file sync_item.h.

Constructor & Destructor Documentation

publish::SyncItem::SyncItem ( )

Definition at line 25 of file sync_item.cc.

publish::SyncItem::~SyncItem ( )
virtual

Definition at line 66 of file sync_item.cc.

publish::SyncItem::SyncItem ( const std::string &  relative_parent_path,
const std::string &  filename,
const SyncUnion union_engine,
const SyncItemType  entry_type 
)
protected

create a new SyncItem Note: SyncItems cannot be created by any using code. SyncUnion will take care of their creating through a factory method to make sure they are initialised correctly (whiteout, hardlink handling, ...)

Parameters
dirPaththe RELATIVE path to the file
filenamethe name of the file ;-)
entryTypewell...

Definition at line 42 of file sync_item.cc.

Member Function Documentation

void publish::SyncItem::CheckCatalogMarker ( )
private

Definition at line 270 of file sync_item.cc.

Referenced by CheckMarkerFiles().

Here is the call graph for this function:

Here is the caller graph for this function:

void publish::SyncItem::CheckGraft ( )
private

Definition at line 294 of file sync_item.cc.

Referenced by CheckMarkerFiles().

Here is the call graph for this function:

Here is the caller graph for this function:

void publish::SyncItem::CheckMarkerFiles ( )
protected

Definition at line 262 of file sync_item.cc.

Referenced by publish::SyncItemNative::SyncItemNative().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual catalog::DirectoryEntryBase publish::SyncItem::CreateBasicCatalogDirent ( ) const
pure virtual

Generates a DirectoryEntry that can be directly stored into a catalog db. Note: this sets the inode fields to kInvalidInode as well as the link count to 1 if MaskHardlink() has been called before (cf. OverlayFS)

Returns
a DirectoryEntry structure to be written into a catalog

Implemented in publish::SyncItemNative, publish::SyncItemDummyDir, publish::SyncItemDummyCatalog, and publish::SyncItemTar.

virtual IngestionSource* publish::SyncItem::CreateIngestionSource ( ) const
pure virtual
std::string publish::SyncItem::filename ( ) const
inline
zlib::Algorithms publish::SyncItem::GetCompressionAlgorithm ( ) const
inline
SyncItemType publish::SyncItem::GetGenericFiletype ( const EntryStat stat) const
protected

Definition at line 71 of file sync_item.cc.

Referenced by GetRdOnlyFiletype(), publish::SyncItemNative::GetScratchFiletype(), and GetUnionFiletype().

Here is the call graph for this function:

Here is the caller graph for this function:

const FileChunkList* publish::SyncItem::GetGraftChunks ( ) const
inline

Definition at line 120 of file sync_item.h.

std::string publish::SyncItem::GetGraftMarkerPath ( ) const
private

Definition at line 286 of file sync_item.cc.

Referenced by CheckGraft().

Here is the call graph for this function:

Here is the caller graph for this function:

unsigned int publish::SyncItem::GetRdevMajor ( ) const
inline

Definition at line 104 of file sync_item.h.

Referenced by publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the call graph for this function:

Here is the caller graph for this function:

unsigned int publish::SyncItem::GetRdevMinor ( ) const
inline

Definition at line 109 of file sync_item.h.

Referenced by publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the call graph for this function:

Here is the caller graph for this function:

SyncItemType publish::SyncItem::GetRdOnlyFiletype ( ) const
protected

Definition at line 83 of file sync_item.cc.

Referenced by IsTouched(), MarkAsWhiteout(), and WasType().

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t publish::SyncItem::GetRdOnlyInode ( ) const

Definition at line 163 of file sync_item.cc.

Here is the call graph for this function:

unsigned int publish::SyncItem::GetRdOnlyLinkcount ( ) const

Definition at line 157 of file sync_item.cc.

Here is the call graph for this function:

std::string publish::SyncItem::GetRdOnlyPath ( ) const

Definition at line 243 of file sync_item.cc.

Referenced by StatRdOnly().

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t publish::SyncItem::GetRdOnlySize ( ) const

Definition at line 185 of file sync_item.cc.

Here is the call graph for this function:

std::string publish::SyncItem::GetRelativePath ( ) const
inline

Definition at line 143 of file sync_item.h.

Referenced by GetGenericFiletype(), GetRdOnlyPath(), publish::SyncItemNative::GetScratchFiletype(), GetScratchPath(), GetUnionPath(), and MarkAsWhiteout().

Here is the caller graph for this function:

virtual SyncItemType publish::SyncItem::GetScratchFiletype ( ) const
protectedpure virtual

Implemented in publish::SyncItemNative, publish::SyncItemDummyDir, publish::SyncItemDummyCatalog, and publish::SyncItemTar.

Referenced by IsTouched().

Here is the caller graph for this function:

std::string publish::SyncItem::GetScratchPath ( ) const

Definition at line 255 of file sync_item.cc.

Referenced by publish::SyncItemNative::StatScratch().

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t publish::SyncItem::GetScratchSize ( ) const

Definition at line 180 of file sync_item.cc.

Here is the call graph for this function:

SyncItemType publish::SyncItem::GetUnionFiletype ( ) const
protected

Definition at line 108 of file sync_item.cc.

Referenced by IsTouched().

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t publish::SyncItem::GetUnionInode ( ) const

Definition at line 175 of file sync_item.cc.

Here is the call graph for this function:

unsigned int publish::SyncItem::GetUnionLinkcount ( ) const

Definition at line 169 of file sync_item.cc.

Referenced by HasHardlinks().

Here is the call graph for this function:

Here is the caller graph for this function:

std::string publish::SyncItem::GetUnionPath ( ) const
platform_stat64 publish::SyncItem::GetUnionStat ( ) const
inlineprotected

Definition at line 204 of file sync_item.h.

Referenced by publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::HasCatalogMarker ( ) const
inline

Definition at line 114 of file sync_item.h.

bool publish::SyncItem::HasCompressionAlgorithm ( ) const
inline

Definition at line 116 of file sync_item.h.

bool publish::SyncItem::HasContentHash ( ) const
inline

Definition at line 123 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::HasGraftMarker ( ) const
inline

Definition at line 115 of file sync_item.h.

bool publish::SyncItem::HasHardlinks ( ) const
inline

Definition at line 163 of file sync_item.h.

Referenced by publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsBlockDevice ( ) const
inline

Definition at line 77 of file sync_item.h.

Referenced by publish::SyncItemTar::CreateBasicCatalogDirent(), publish::SyncItemNative::CreateBasicCatalogDirent(), and IsSpecialFile().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsBundleSpec ( ) const
inline

Definition at line 97 of file sync_item.h.

bool publish::SyncItem::IsCatalogMarker ( ) const
inline

Definition at line 85 of file sync_item.h.

bool publish::SyncItem::IsCharacterDevice ( ) const
inline

Definition at line 76 of file sync_item.h.

Referenced by publish::SyncItemTar::CreateBasicCatalogDirent(), publish::SyncItemNative::CreateBasicCatalogDirent(), and IsSpecialFile().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsChunkedGraft ( ) const
inline

Definition at line 118 of file sync_item.h.

bool publish::SyncItem::IsDirectIo ( ) const
inline

Definition at line 82 of file sync_item.h.

Referenced by publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the caller graph for this function:

bool publish::SyncItem::IsDirectory ( ) const
inline

Definition at line 64 of file sync_item.h.

Referenced by CheckMarkerFiles(), publish::SyncItemTar::GetStatFromTar(), IsOpaqueDirectory(), and MarkAsOpaqueDirectory().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsExternalData ( ) const
inline
bool publish::SyncItem::IsFifo ( ) const
inline

Definition at line 78 of file sync_item.h.

Referenced by IsSpecialFile().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsGraftMarker ( ) const
inline

Definition at line 80 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::IsNew ( ) const
inline

Definition at line 70 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::IsOpaqueDirectory ( ) const
inline

Definition at line 86 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::IsRegularFile ( ) const
inline

Definition at line 66 of file sync_item.h.

Referenced by CheckMarkerFiles().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsSocket ( ) const
inline

Definition at line 79 of file sync_item.h.

Referenced by IsSpecialFile().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsSpecialFile ( ) const
inline

Definition at line 88 of file sync_item.h.

Referenced by GetRdevMajor(), and GetRdevMinor().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsSymlink ( ) const
inline

Definition at line 68 of file sync_item.h.

Referenced by publish::SyncItemTar::CreateBasicCatalogDirent(), and publish::SyncItemNative::CreateBasicCatalogDirent().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::IsTouched ( ) const
inline

Definition at line 71 of file sync_item.h.

Here is the call graph for this function:

virtual bool publish::SyncItem::IsType ( const SyncItemType  expected_type) const
protectedpure virtual

Checks if the SyncItem is the given file type (file, dir, symlink, ...) in the union file system volume. Hence: After the publish operation, the file will be this type in CVMFS.

Parameters
expected_typethe file type to be checked against
Returns
true if file type matches the expected type

Implemented in publish::SyncItemNative, publish::SyncItemTar, and publish::SyncItemDummyCatalog.

Referenced by IsBlockDevice(), IsCharacterDevice(), IsDirectory(), IsFifo(), IsGraftMarker(), IsRegularFile(), IsSocket(), and IsSymlink().

Here is the caller graph for this function:

bool publish::SyncItem::IsValidGraft ( ) const
inline

Definition at line 117 of file sync_item.h.

bool publish::SyncItem::IsWhiteout ( ) const
inline

Definition at line 84 of file sync_item.h.

virtual void publish::SyncItem::MakePlaceholderDirectory ( ) const
pure virtual
void publish::SyncItem::MarkAsOpaqueDirectory ( )

Definition at line 151 of file sync_item.cc.

Here is the call graph for this function:

void publish::SyncItem::MarkAsWhiteout ( const std::string &  actual_filename)

Definition at line 124 of file sync_item.cc.

Here is the call graph for this function:

void publish::SyncItem::MaskHardlink ( )
inline

Union file systems (i.e. OverlayFS) might not properly support hardlinks, forcing us to ignore them during publishing. A 'masked hardlink' will be treated as a normal file (linkcount == 1). Hence, any created hardlinks will be broken up into individual files with differing inodes.

Definition at line 162 of file sync_item.h.

bool publish::SyncItem::operator== ( const SyncItem other) const
inline

Definition at line 183 of file sync_item.h.

std::string publish::SyncItem::relative_parent_path ( ) const
inline

Definition at line 175 of file sync_item.h.

void publish::SyncItem::SetCatalogMarker ( )
inline

Definition at line 181 of file sync_item.h.

void publish::SyncItem::SetCompressionAlgorithm ( const zlib::Algorithms alg)
inline

Definition at line 130 of file sync_item.h.

Referenced by CheckGraft().

Here is the caller graph for this function:

void publish::SyncItem::SetContentHash ( const shash::Any hash)
inline

Definition at line 122 of file sync_item.h.

Referenced by publish::SyncMediator::PublishFilesCallback().

Here is the caller graph for this function:

void publish::SyncItem::SetDirectIo ( bool  val)
inline

Definition at line 125 of file sync_item.h.

void publish::SyncItem::SetExternalData ( bool  val)
inline

Definition at line 124 of file sync_item.h.

void publish::SyncItem::StatGeneric ( const std::string &  path,
EntryStat info,
const bool  refresh 
)
staticprotected

Definition at line 194 of file sync_item.cc.

Referenced by CheckCatalogMarker(), StatRdOnly(), publish::SyncItemNative::StatScratch(), and StatUnion().

Here is the call graph for this function:

Here is the caller graph for this function:

void publish::SyncItem::StatRdOnly ( const bool  refresh = false) const
inlineprivate

Definition at line 313 of file sync_item.h.

Referenced by GetRdOnlyFiletype(), GetRdOnlyInode(), GetRdOnlyLinkcount(), GetRdOnlySize(), and MarkAsWhiteout().

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void publish::SyncItem::StatScratch ( const bool  refresh) const
privatepure virtual

Implemented in publish::SyncItemNative, publish::SyncItemDummyCatalog, and publish::SyncItemTar.

Referenced by GetScratchSize(), and MarkAsWhiteout().

Here is the caller graph for this function:

void publish::SyncItem::StatUnion ( const bool  refresh = false) const
inlineprivate

Definition at line 316 of file sync_item.h.

Referenced by GetRdevMajor(), GetRdevMinor(), GetUnionFiletype(), GetUnionInode(), GetUnionLinkcount(), and GetUnionStat().

Here is the call graph for this function:

Here is the caller graph for this function:

bool publish::SyncItem::WasBundleSpec ( ) const
inline

Definition at line 100 of file sync_item.h.

bool publish::SyncItem::WasDirectory ( ) const
inline

Definition at line 65 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::WasRegularFile ( ) const
inline

Definition at line 67 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::WasSpecialFile ( ) const
inline

Definition at line 91 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::WasSymlink ( ) const
inline

Definition at line 69 of file sync_item.h.

Here is the call graph for this function:

bool publish::SyncItem::WasType ( const SyncItemType  expected_type) const
inlineprotected

Checks if the SyncItem was the given file type (file, dir, symlink, ...) in CVMFS (or the lower layer of the union file system). Hence: Before the current transaction the file was this type in CVMFS.

Parameters
expected_typethe file type to be checked against
Returns
true if file type was the expected type in CVMFS

Definition at line 230 of file sync_item.h.

Referenced by IsNew(), WasDirectory(), WasRegularFile(), WasSpecialFile(), and WasSymlink().

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

zlib::Algorithms publish::SyncItem::compression_algorithm_
private

Definition at line 308 of file sync_item.h.

Referenced by GetCompressionAlgorithm(), and SetCompressionAlgorithm().

shash::Any publish::SyncItem::content_hash_
protected

Definition at line 274 of file sync_item.h.

Referenced by CheckGraft(), GetContentHash(), HasContentHash(), SetContentHash(), and SyncItem().

bool publish::SyncItem::direct_io_
private

Definition at line 299 of file sync_item.h.

Referenced by IsDirectIo(), and SetDirectIo().

bool publish::SyncItem::external_data_
private

Definition at line 298 of file sync_item.h.

Referenced by IsExternalData(), and SetExternalData().

std::string publish::SyncItem::filename_
private
FileChunkList* publish::SyncItem::graft_chunklist_
private

Chunklist from graft. Not initialized by default to save memory.

Definition at line 305 of file sync_item.h.

Referenced by CheckGraft(), GetGraftChunks(), IsChunkedGraft(), and ~SyncItem().

bool publish::SyncItem::graft_marker_present_
private

.cvmfsgraft-$filename exists

Definition at line 296 of file sync_item.h.

Referenced by CheckGraft(), and HasGraftMarker().

ssize_t publish::SyncItem::graft_size_
protected

Definition at line 271 of file sync_item.h.

Referenced by CheckGraft(), and publish::SyncItemNative::CreateBasicCatalogDirent().

bool publish::SyncItem::has_catalog_marker_
private

directory containing .cvmfscatalog

Definition at line 294 of file sync_item.h.

Referenced by CheckCatalogMarker(), HasCatalogMarker(), and SetCatalogMarker().

bool publish::SyncItem::has_compression_algorithm_
private

Definition at line 310 of file sync_item.h.

Referenced by HasCompressionAlgorithm(), and SetCompressionAlgorithm().

bool publish::SyncItem::masked_hardlink_
private

SyncUnion masked out the linkcount

Definition at line 293 of file sync_item.h.

Referenced by HasHardlinks(), and MaskHardlink().

bool publish::SyncItem::opaque_
private

SyncUnion marked this as opaque dir

Definition at line 292 of file sync_item.h.

Referenced by IsOpaqueDirectory(), and MarkAsOpaqueDirectory().

EntryStat publish::SyncItem::rdonly_stat_
mutableprivate
SyncItemType publish::SyncItem::rdonly_type_
mutableprotected
std::string publish::SyncItem::relative_parent_path_
private

Definition at line 300 of file sync_item.h.

Referenced by GetGraftMarkerPath(), GetRelativePath(), operator==(), and relative_parent_path().

SyncItemType publish::SyncItem::scratch_type_
mutableprotected
const SyncUnion* publish::SyncItem::union_engine_
private

this SyncUnion created this object

Definition at line 286 of file sync_item.h.

Referenced by GetGraftMarkerPath(), GetRdOnlyPath(), GetScratchPath(), and GetUnionPath().

EntryStat publish::SyncItem::union_stat_
mutableprivate
bool publish::SyncItem::valid_graft_
private

checksum and size in graft marker

Definition at line 295 of file sync_item.h.

Referenced by CheckGraft(), and IsValidGraft().

bool publish::SyncItem::whiteout_
private

SyncUnion marked this as whiteout

Definition at line 291 of file sync_item.h.

Referenced by IsWhiteout(), and MarkAsWhiteout().


The documentation for this class was generated from the following files: