Archived copy. This page was recovered from the Internet Archive snapshot of /display/SP/Local instance at STFC taken on 2012-11-28. The original wiki at wiki.opf-labs.org is being decommissioned.

Local instance at STFC

Created by Erica Yang on Aug 15, 2012 · last edited by Erica Yang · on Aug 15, 2012 (view history) · 4 versions

Basic cluster software

Scientific Linux 2.6.18-238.9.1.el5 #1 SMP (Symmetric Multi Processors) Tue Apr 12 18:53:46 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

OpenSSH as part of the OS installation

Hadoop 0.20.2
Subversion -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14

Cloudera CDH3u0

java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

Cluster hardware

NETWORK infrastructure

The head nodes and the slaves are connected to the Tier1's core network at 1Gbps, they are connected to several switch stacks with 80Gbps backplanes.

The switch stacks are in turn connected by 20Gbps backbone links to a core non-blocking switch.

NameNode and JobTracker

Total disk capacity: 70+TBs

CPU: 2 x Xeon [email protected] Dualcore CPU ( 4 cores )
RAM: 8GB
NIC: 2 x GbE (1 used)
DISK: 1 x 250GB DISK

Slaves 1 - 4

Supermicro/Boston custom storage solution
CPU: 2 x Dual-Core AMD Opteron(tm) Processor 2220
RAM: 8GB
NIC: 2 x GbE (1 used)
DISK: 16 x 750GB disks as JBOD

Slaves 5 - 6

Supermicro/Boston custom storage solution

CPU: 2 x Intel(R) Xeon(R) CPU E5405 @ 2.00GHz ( 8 cores )
RAM: 8GB
NIC: 2 x GbE (1 used)
DISK: 24 x 1TB disks as JBOD