View source for Al-salam

Al-Salam is the working name for the Earlham Computer Science Department's upcoming cluster computer.

At the moment Al-Salam exists only as a $40,000 grant and a growing list of tentative specifications:

== Latest Overarching Questions ==
*Should we build this machine ourselves?
*#Are we wasting our money and learning opportunity letting them do the building for us?
*#If it is cheaper, Would it be a useful experience for the students this coming semester to take a large collection of hardware and make it into a cluster?
*# Yes, this was pretty clear from the email thread in early December.
*How much if any GPGPU hardware do we want?  0, 1 or 2 nodes worth?
*Do we want a high bandwidth/low latency network?
**We do not. More expensive than it is worth.
*What software stack do we want to run?  Vendor supplied or the BCCD?
**Both. Vendor-supplied base with a BCCD virtual machine
***Will the virtual machine support CUDA?
* Do compute nodes have spinning disk? 
** Compute nodes have a spinning disk. Solid state is still expensive
* What's on the local persistent store?  /tmp? An entire OS?
* Support
** Consider getting the cheapest hardware support, loosing a node isn't critical as long as they send a replacement quickly.

== Parts List ==
# Nodes - case, motherboard(s), power supply, CPU, RAM, GPGPU cards
# Switch - managed, cut-through
# Power distribution - rack-mount PDUs

== Tentative Specifications ==

=== Budget ===
* $35,000 (leaving $5,000 for discretionary spending)

=== Nodes ===
* Intel Nehalem processors
* 4 core processors minimum
** Six cores still expensive
* 1.5GB RAM per core

=== Specialty Nodes ===
* Two nodes should support CUDA GPGPU

Educationally, we could expect to get significant use out of GPGPUs, but the production use is limited.
Increasing the variance of the architecture landscape would be a bonus to education.

=== Network ===
* Gigabit Ethernet fabric with switch

=== Disk ===
* Spinning Disk

=== OS ===
* Virtual BCCD on top of built-in OS.

== Quick breakdown ==

=== Nodes ===

{| class="wikitable" border="1"
!
! [[Al-salam#ION_Computer_Systems_Quotation_.2361116|ION #61116]]
! [[Al-salam#ION_Computer_Systems_Quotation_.2361164|ION #61164]]
! [[Al-salam#Silicon_Mechanics_Quote_.23174536|SM #174536]]
! [[Al-salam#Newegg_Quote_.231|Newegg #1]]
! [[Al-salam#Newegg_Quote_.232|Newegg #2]]
|-
| '''CPU'''
| 72 2.4GHz Intel E5530
| 80 2.4GHz Intel E5530
| 80 2.4GHz Intel E5530
| 128 2.4GHz Intel E5530
| 112 2.4GHz Intel E5530
|-
| '''RAM'''
| 108GB PC3-10600
| 120GB PC3-10600
| 120GB DDR3-1333
| 192GB DDR3-1333
| 168GB DDR3-1333
|-
| '''GPU'''
| 2 Tesla C1060
| 2 Tesla C1060
| 2 Tesla C1060
| None
| 4 Tesla C1060
|-
| '''Local disk'''
| Yes
| Yes
| Yes
| Yes
| Yes
|-
| '''Shared chassis'''
| No
| Yes
| Yes
| No
| No
|-
| '''Remote mgmt'''
| No
| No
| IPMI
| No
| IPMI on GPU nodes
|-
| '''Size (just nodes)'''
| 9U
| 6U
| 6U
| 16U
| 12U
|-
| '''Price'''
| $33,173.20
| $33,054.30
| $30,078.00
| $32,910.56
| $34,696.78
|}

=== Power distribution ===

{| class="wikitable" border="1"
!
! [http://www.tripplite.com/en/products/model.cfm?txtSeriesID=446&EID=14295&txtModelID=2005 PDU1220]
! [http://www.tripplite.com/en/products/model.cfm?txtSeriesID=513&EID=77300&txtModelID=3867 PDUMH20]
! [http://accessories.us.dell.com/sna/products/Server_Network/productdetail.aspx?c=us&l=en&cs=04&sku=A0151958 AP9563]
! [http://accessories.us.dell.com/sna/products/Server_Network/productdetail.aspx?c=us&l=en&s=bsd&cs=04&sku=A0748689 AP7801]
|-
| '''Vendor'''
| TrippLite
| TrippLite
| APC
| APC
|-
| '''Size'''
| 1U
| 1U
| 1U
| 1U
|-
| '''Capabilities'''
| Dumb
| Metered
| Dumb
| Metered
|-
| '''Input power'''
| 20A, 1x NEMA 5-20P
| 20A, 1x NEMA L5-20P w/ NEMA 5-20P adapter
| 20A, 1x NEMA 5-20P
| 20A, 1x NEMA 5-20P
|-
| '''Output power'''
| 13x NEMA 5-20R
| 12x NEMA 5-20R
| 10x NEMA 5-20R
| 8x NEMA 5-20R
|-
| '''Price'''
| $195
| $230
| $120
| $380
|}

== ION Computer Systems Quotation #61116  ==
*2  ION G10 Server with GPU: 4,972.00 each
** (2) Intel� Quad-Core Xeon� processor E5530 (2.40GHz, 8MB Cache, 5.86GT/s, 80W)
** 12GB RAM [Bank 1 of 2: (6) 2GB ECC PC3-10600 1333MHz 2rank DDR3 RDIMM Modules][Smart]
** Total memory: 12GB DDR3_1333
** Configure 1 RAID sets / arrays.
** Seagate SV35.3 250GB, 7200RPM, SATA 3Gb for SDVR 3.5� Disk
** (1) NVidia Tesla C1060 w. 4GB DDR3
** Dual Intel Gigabit Server NICs with IOAT2 Integrated

*7 ION G10 Server without GPU: $3,697.00 each
** (2) Intel� Quad-Core Xeon� processor E5530 (2.40GHz, 8MB Cache, 5.86GT/s, 80W)
** 12GB RAM [Bank 1 of 2: (6) 2GB ECC PC3-10600 1333MHz 2rank DDR3 RDIMM Modules][Smart]
** Total memory: 12GB DDR3_1333
** Configure 1 RAID sets / arrays.
** Seagate SV35.3 250GB, 7200RPM, SATA 3Gb for SDVR 3.5� Disk
** Dual Intel Gigabit Server NICs with IOAT2 Integrated

* Networking Fabric
**Network not included

* Other stuff
** scorpion: ION bootable USB Flash device for trouble shooting.
** 3 year Next Business Day response Onsite Repair Service by Source Support
** Default load for testing (Service Partition + CentOS 5.3 for Intel64)

* Price tag: $33,173.20

== ION Computer Systems Quotation #61164  ==

* 2 ION G10 Server with GPU: $4,972.00 each
** (2) Intel� Quad-Core Xeon� processor E5530 (2.40GHz, 8MB Cache, 5.86GT/s, 80W)
** 12GB RAM [Bank 1 of 2: (6) 2GB ECC PC3-10600 1333MHz 2rank DDR3 RDIMM Modules][Smart]
** Total memory: 12GB DDR3_1333
** Configure 1 RAID sets / arrays.
** Seagate SV35.3 250GB, 7200RPM, SATA 3Gb for SDVR 3.5� Disk
** (1) NVidia Tesla C1060 w. 4GB DDR3
** Dual Intel Gigabit Server NICs with IOAT2 Integrated

* 4 ION T11 DualNode: $6,477.0 each
** (2x2) Intel� Quad-Core Xeon� processor E5530 (2.40GHz, 8MB Cache, 5.86GT/s, 80W)
** Total memory: 12GB DDR3_1333 per node
** No RAID, Separate disks (NO redundancy)
** Configure 1 RAID sets / arrays.
** Seagate Constellation 160GB, 7200RPM, SATA 3Gb NCQ 2.5� Disk
** Dual Intel Gigabit Server NICs with IOAT2 Integrated
** These nodes are modular. One can be unplugged and worked on while the others remain running.

* Network
** Network Not Included

* Other stuff
** scorpion: ION bootable USB Flash device for trouble shooting.
** 3 year Next Business Day response Onsite Repair Service by Source Support
** Default load for testing (Service Partition + CentOS 5.3 for Intel64)

*Price Tag: $33,054.30

==Silicon Mechanics Quote #174536==
* 2x Rackform iServ R4410:  $11043.00 each ($10601.00 each with education) [http://www.siliconmechanics.com/quotes/174536?confirmation=879366549 link]
**Shared Chassis: The following chassis resources are shared by all 4 compute nodes
**External Optical Drive: No Item Selected
**Power Supply: Shared, Redundant 1400W Power Supply with PMBus - 80 PLUS Gold Certified
**Rail Kit: Quick-Release Rail Kit for Square Holes, 26.5 - 36.4 inches
**Compute Nodes x4
***CPU: 2 x Intel Xeon E5530 Quad-Core 2.40GHz, 8MB Cache, 5.86GT/s QPI
***RAM: 12GB (6 x 2GB) Operating at 1333MHz Max (DDR3-1333 ECC Unbuffered DIMMs)
***NIC: Intel 82576 Dual-Port Gigabit Ethernet Controller - Integrated
***Management: Integrated IPMI with KVM over LAN
***Hot-Swap Drive - 1: 250GB Western Digital RE3 (3.0Gb/s, 7.2Krpm, 16MB Cache) SATA

* 2x Rackform iServ R350-GPU: $5196.00 each ($4433.00 each with education) [http://www.siliconmechanics.com/quotes/174542?confirmation=712641984 link]
**CPU: 2 x Intel Xeon E5530 Quad-Core 2.40GHz, 8MB Cache, 5.86GT/s QPI
**RAM: 12GB (6 x 2GB) Operating at 1333MHz Max (DDR3-1333 ECC Unbuffered DIMMs)
**NIC: Intel 82576 Dual-Port Gigabit Ethernet Controller - Integrated
**Management: Integrated IPMI 2.0 & KVM with Dedicated LAN
**GPU: 1U System with 1 x Tesla C1060 GPU, Actively Cooled
**LP PCIe x4 2.0 (x16 Slot): No Item Selected
**Hot-Swap Drive - 1: 250GB Seagate Barracuda ES.2 (3Gb/s, 7.2Krpm, 32MB Cache, NCQ) SATA
**Power Supply: 1400W Power Supply with PMBus - 80 PLUS Gold Certified
**Rail Kit: 1U Rail Kit

*Price Tag: $32,478 ($30,078 with education)

*Questions
**Can we lose the hot-swappability to save money?
**Do we need to get a Gig-Switch?
***Would Cairo do?

==Newegg Quote #1==
* 16x [http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=16958187 Newegg list]
** 1U [http://www.newegg.com/Product/Product.aspx?item=N82E16811116011 link]
** 2x Intel Xeon (Nehalem) E5530, Quad-Core, 2.4GHz, 80 Watt [http://www.newegg.com/Product/Product.aspx?item=N82E16819117184 link]
** Slim CD/DVD Drive
** 4x Gigabit ethernet [http://www.newegg.com/Product/Product.aspx?item=N82E16813151195R motherboard]
** 500W non-redundant power supply
** 160GB 7200RPM Seagate [http://www.newegg.com/Product/Product.aspx?item=N82E16822148511 link]
** 12GB RAM (240-pin DDR3 1333 ECC, unbuffered)

* Price Tag: $32,910.56

==Newegg Quote #2==
* 2x [http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=8560589 Newegg list]
** 1U
** 2x Intel Xeon (Nehalem) E5530, Quad-Core, 2.4GHz, 80 Watt
** 2x Gigabit ethernet
** IPMI
** 1400W non-redundant power supply
** 2x C1060 Tesla
** 160GB 7200RPM Seagate
** 12GB RAM (240-pin DDR3 1333 ECC, unbuffered)

* 12x [http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=16958187 Newegg list]
** 1U [http://www.newegg.com/Product/Product.aspx?item=N82E16811116011 link]
** 2x Intel Xeon (Nehalem) E5530, Quad-Core, 2.4GHz, 80 Watt
** Slim CD/DVD Drive
** 4x Gigabit ethernet
** 500W non-redundant power supply
** 160GB 7200RPM Seagate
** 12GB RAM (240-pin DDR3 1333 ECC, unbuffered)

* Price tag: $34,696.78