Archived copy.
This page was recovered from the Internet Archive snapshot of
/pages/viewpage.action?pageId=59834443 taken on 2021-01-23.
The original wiki at wiki.opf-labs.org is being decommissioned.
Evaluation of the number of abnormal results in laboratory examinations for a given disease codes in a given period
Evaluator(s)
Tomasz Hofmann (PSNC)
Evaluation points
In this task the efficiency of hadoop job for gathering information about abnormal laboratory results for specified ICD10 code has been evaluated. As the metric the number of HL7 files parsed per second is used. It is worth to note, that test 2 has been performed on local file system and the result is not comparable with results of other tests.
Assessment of measurable points
| Metric | Description | Metric baseline | Metric goal | July 21, 2014 [Test 1] | July 28, 2014 [Test 2] |
|---|---|---|---|---|---|
number of objects per second |
the number of HL7 files parsed in one second | - | - | 69.60 [obj/s] |
66,17 [obj/s] |
Note: *as an object we proposed to use one HL7 file
Metrics must be registered in the metrics catalogue
Technical details
Test 1
Test 3
Evaluation notes
Workflow
The experiment is composed of the following steps (accordingly to the MapReduce schema):
- the map task,
- for each HL7 file saved on HDFS do:
- parse document in order to find out abnormal laboratory results - count them and next add into the context the following pair: Key=icd10 code, Value=the count of the abnormal results
- for each HL7 file saved on HDFS do:
- the reduce task:
- for each icd10 code accumulate the count of the abnormal results
- produce the result pair Key=icd10 code, Value=the number of abnormal results
Scripts