Archived copy.
This page was recovered from the Internet Archive snapshot of
/display/SP/EVAL ARC2WARC-TOMAR with Tika taken on 2016-09-11.
The original wiki at wiki.opf-labs.org is being decommissioned.
EVAL ARC2WARC-TOMAR with Tika
Evaluator(s)
Sven Schlarb <[email protected]>
Evaluation points
Assessment of measurable points
| Metric | Description | Metric baseline | Metric goal | March 04, 2014 (1000) |
March 04, 2014 (4924) | March 04, 2014 (9856) |
|---|---|---|---|---|---|---|
| NumberOfObjectsPerHour | Number of objects processed in one hour |
545,17246 | 3317,97235 | 2813,267735 | 7006,635071 | |
| MinObjectSizeHandledInGbytes | Smallest ARC file in sample |
0,001638618 | 0,001638618 | 0,0001516 | 0,000151632 | |
| MaxObjectSizeHandledInGbytes | Biggest ARC file in sample |
0,295765739 | 0,295765739 | 0,295765739 | 0,295765739 | |
| ThroughputGbytesPerMinute | The throughput of data measured in Gybtes per minute | 0,832135864 | 5,064459399 | 4,241862946 | 10,54745844 | |
| ThroughputGbytesPerHour | The throughput of data measured in Gbytes per hour | 49,92815185 | 303,8675639 | 254,5117767 | 632,8475062 | |
| ReliableAndStableAssessment | Manual assessment on if the experiment performed reliable and stable | true | true | true | true | |
| NumberOfFailedFiles | Number of files that failed in the workflow | 0 | 0 | 0 | 0 | |
| AverageRuntimePerItemInSeconds | The average processing time in seconds per item | 6,60 | 1,09 | 1,27965069 | 0,513798701 |
Technical details
The different evaluation points in the table above refer to data sets of different size, the Hadoop Job-ID links to details about the job execution:
March 04, 2014 (1000): waa-full-arcs-1 (subset 1000), job_201401221447_0057
March 04, 2014 (4924): waa-full-arcs-1 (4924 arc files)
These data samples are subsets of the ONB web archive crawl ONB Web Archive Dataset.