CernVM-FS
2.12.0
|
#include <upload_spooler_definition.h>
Public Types | |
enum | DriverType { S3, Local, Gateway, Mock, Unknown } |
Public Member Functions | |
SpoolerDefinition (const std::string &definition_string, const shash::Algorithms hash_algorithm, const zlib::Algorithms compression_algorithm=zlib::kZlibDefault, const bool generate_legacy_bulk_chunks=false, const bool use_file_chunking=false, const size_t min_file_chunk_size=0, const size_t avg_file_chunk_size=0, const size_t max_file_chunk_size=0, const std::string &session_token_file="", const std::string &key_file="") | |
bool | IsValid () const |
SpoolerDefinition | Dup2DefaultCompression () const |
Public Attributes | |
DriverType | driver_type |
the type of the spooler driver More... | |
std::string | temporary_path |
scratch space for the IngestionPipeline More... | |
std::string | spooler_configuration |
shash::Algorithms | hash_algorithm |
zlib::Algorithms | compression_alg |
bool | generate_legacy_bulk_chunks |
bool | use_file_chunking |
size_t | min_file_chunk_size |
size_t | avg_file_chunk_size |
size_t | max_file_chunk_size |
unsigned int | number_of_concurrent_uploads |
unsigned int | num_upload_tasks |
std::string | session_token_file |
std::string | key_file |
bool | valid_ |
Static Public Attributes | |
static const unsigned | kDefaultMaxConcurrentUploads = 512 |
static const unsigned | kDefaultNumUploadTasks = 1 |
static const char * | kDriverNames [] |
corresponds to DriverType More... | |
SpoolerDefinition is given by a string of the form: <spooler type>="">:<spooler description>="">
F.e: local:/srv/cvmfs/dev.cern.ch to define a local spooler with upstream path /srv/cvmfs/dev.cern.ch
Definition at line 22 of file upload_spooler_definition.h.
Enumerator | |
---|---|
S3 | |
Local | |
Gateway | |
Mock | |
Unknown |
Definition at line 26 of file upload_spooler_definition.h.
upload::SpoolerDefinition::SpoolerDefinition | ( | const std::string & | definition_string, |
const shash::Algorithms | hash_algorithm, | ||
const zlib::Algorithms | compression_algorithm = zlib::kZlibDefault , |
||
const bool | generate_legacy_bulk_chunks = false , |
||
const bool | use_file_chunking = false , |
||
const size_t | min_file_chunk_size = 0 , |
||
const size_t | avg_file_chunk_size = 0 , |
||
const size_t | max_file_chunk_size = 0 , |
||
const std::string & | session_token_file = "" , |
||
const std::string & | key_file = "" |
||
) |
Reads a given definition_string as described above and interprets it. If the provided string turns out to be malformed the created SpoolerDefinition object will not be valid. A user should check this after creation using IsValid().
definition_string | the spooler definition string to be inter- preted by the constructor |
Definition at line 17 of file upload_spooler_definition.cc.
SpoolerDefinition upload::SpoolerDefinition::Dup2DefaultCompression | ( | ) | const |
Creates a new SpoolerDefinition based on an existing one. The new spooler has compression set to zlib, which is required for catalogs and other meta- objects.
Definition at line 77 of file upload_spooler_definition.cc.
Referenced by swissknife::Ingest::Main(), and swissknife::CommandSync::Main().
|
inline |
Definition at line 49 of file upload_spooler_definition.h.
Referenced by upload::GatewayUploader::GatewayUploader(), upload::LocalUploader::LocalUploader(), and upload::S3Uploader::S3Uploader().
size_t upload::SpoolerDefinition::avg_file_chunk_size |
Definition at line 76 of file upload_spooler_definition.h.
zlib::Algorithms upload::SpoolerDefinition::compression_alg |
Definition at line 68 of file upload_spooler_definition.h.
Referenced by Dup2DefaultCompression().
DriverType upload::SpoolerDefinition::driver_type |
the type of the spooler driver
Definition at line 58 of file upload_spooler_definition.h.
Referenced by upload::GatewayUploader::GatewayUploader(), upload::LocalUploader::LocalUploader(), swissknife::Ingest::Main(), upload::S3Uploader::S3Uploader(), SpoolerDefinition(), upload::GatewayUploader::WillHandle(), upload::LocalUploader::WillHandle(), and upload::S3Uploader::WillHandle().
bool upload::SpoolerDefinition::generate_legacy_bulk_chunks |
If a file is chunked, clients >= 2.1.7 do not need the bulk chunk. We can force creating the bulk chunks for backwards compatibility.
Definition at line 73 of file upload_spooler_definition.h.
shash::Algorithms upload::SpoolerDefinition::hash_algorithm |
Definition at line 67 of file upload_spooler_definition.h.
|
static |
Definition at line 23 of file upload_spooler_definition.h.
|
static |
Definition at line 24 of file upload_spooler_definition.h.
|
static |
corresponds to DriverType
Definition at line 25 of file upload_spooler_definition.h.
Referenced by publish::SettingsStorage::GetLocator().
std::string upload::SpoolerDefinition::key_file |
Definition at line 93 of file upload_spooler_definition.h.
Referenced by upload::GatewayUploader::ParseSpoolerDefinition().
size_t upload::SpoolerDefinition::max_file_chunk_size |
Definition at line 77 of file upload_spooler_definition.h.
size_t upload::SpoolerDefinition::min_file_chunk_size |
Definition at line 75 of file upload_spooler_definition.h.
unsigned int upload::SpoolerDefinition::num_upload_tasks |
Number of threads used for I/O write calls. Effectively this parameter sets the I/O depth.
Definition at line 89 of file upload_spooler_definition.h.
Referenced by swissknife::CommandSync::Main().
unsigned int upload::SpoolerDefinition::number_of_concurrent_uploads |
This is the number of concurrently open files to be uploaded. It does not, however, specify the degree of parallelism of the I/O write operations.
Definition at line 83 of file upload_spooler_definition.h.
Referenced by swissknife::Ingest::Main(), and swissknife::CommandSync::Main().
std::string upload::SpoolerDefinition::session_token_file |
Definition at line 92 of file upload_spooler_definition.h.
Referenced by publish::ConstructSpoolers(), and upload::GatewayUploader::ParseSpoolerDefinition().
std::string upload::SpoolerDefinition::spooler_configuration |
A driver specific spooler configuration string (interpreted by the concrete spooler)
Definition at line 65 of file upload_spooler_definition.h.
Referenced by upload::GatewayUploader::ParseSpoolerDefinition(), upload::S3Uploader::ParseSpoolerDefinition(), and SpoolerDefinition().
std::string upload::SpoolerDefinition::temporary_path |
scratch space for the IngestionPipeline
Definition at line 59 of file upload_spooler_definition.h.
Referenced by upload::AbstractUploader::CreateAndOpenTemporaryChunkFile(), and SpoolerDefinition().
bool upload::SpoolerDefinition::use_file_chunking |
Definition at line 74 of file upload_spooler_definition.h.
bool upload::SpoolerDefinition::valid_ |
Definition at line 95 of file upload_spooler_definition.h.
Referenced by IsValid(), and SpoolerDefinition().