Difference between revisions of "Sysadmin"

From Earlham CS Department
Jump to navigation Jump to search
(→‎Machines and Brief Descriptions of Services)
(→‎Compute (servers and clusters))
(31 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
 
 
This is the hub for the CS sysadmins on the wiki.
 
This is the hub for the CS sysadmins on the wiki.
  
−
== Common Tasks ==
+
= Overview =
  
−
* [[Sysadmin:New Sysadmins | Welcoming a new sysadmin ]] <- log in here with wiki credentials to begin learning to be an admin
+
[https://docs.google.com/drawings/d/1XaULz5IxXV_BZQjrko3QJ8wV5aXsSTYcSWxxT49OyZk/edit If you're visually inclined, we have a colorful and easy-to-edit map of our servers here!]
−
* [[Sysadmin:Troubleshooting|General troubleshooting tips for admins]]
 
−
* [[Sysadmin:SSH|Useful ssh information for admins]]
 
−
* [[Sysadmin:Recurring Tasks | Recurring tasks - e.g. software updates, hardware replacements]]
 
−
* [[Sysadmin:SlackAndGitLab | Slack and GitLab integration]]
 
−
* [https://code.cs.earlham.edu/sysadmin/ticket-tracker Ticket tracking for current projects]
 
−
* [[Sysadmin:User Management | User Management]]
 
−
** [[Sysadmin:Contacting all users|Contacting all users]]
 
−
** [[Reset password]]
 
−
* [[Sysadmin:Software installation | Software installation]]
 
−
** [[Modules | Installing software under modules ]]
 
−
* [[Sysadmin:Monitoring | Monitoring ]]
 
−
* [[Sysadmin:Backup|Backup]]
 
−
* [[Sysadmin:AddComputer|Add a computer]]
 
−
** [[Sysadmin:Setting up Lovelace Lab Machines | Setting up Lovelace Lab Machines]]
 
−
* [[Senior projects]]
 
−
* [[ShutdownProcedure| Shutdown and Boot up]]
 
−
* [[Password managers]]
 
−
* [[Server safety]]
 
−
* [[Sysadmin:Upgrading SSL Certificate | Upgrading SSL Certificates ]]
 
−
** [[Sysadmin:ImportantInfo:SSLcerts| Generating SSL Certificates]]
 
−
* [[Sysadmin:Launch at startup|Launch a process at startup]]
 
−
* [[Sysadmin:CS-ITS Interoperability|Working with ITS]]
 
−
* [[Sysadmin:Recurring spending | Recurring spending ]]
 
  
−
== Services ==
+
== Server room ==
−
* [[Sysadmin:Services:ClusterOverview|Cluster Overview]] and [[Sysadmin:Ccg-admin|additional details]]
 
−
* [[Sysadmin:Jupyterhub Notebook Server|Jupyterhub]] and [[Nbgrader notes|NBGrader]]
 
−
* [[Sysadmin:Services:Apache2|Apache2]]
 
−
* [[Sysadmin:Services:Databases|Databases]]
 
−
* [[Sysadmin:DNS & DHCP|DNS and DHCP]]
 
−
* [[Sysadmin:VirtualBox | VirtualBox]]
 
−
* [[Sysadmin:Services:XenServerSetup | Xen Server]]
 
−
* [[X Applications]]
 
−
* [[Bash_start_up_script|Bash startup scripts]]
 
−
For old documentation, see: [[Sysadmin:Old | Old Wiki Information]]
 
  
−
= Machines and Brief Descriptions of Services =
+
Our servers are in Noyes, the science building that predates the CST. For general information about the server room and how to use it, check out [[Sysadmin:Server Room|this page]].
  
−
[https://docs.google.com/drawings/d/1XaULz5IxXV_BZQjrko3QJ8wV5aXsSTYcSWxxT49OyZk/edit If you're visually inclined, we have a colorful and easy-to-edit map of our servers here!]
+
Columns: machine name, IPs, type (virtual, metal), purpose, dies, cores, RAM
  
−
=== Compute (servers and clusters) ===
+
== Compute (servers and clusters) ==
  
−
We have CS and cluster machines.
 
  
−
CS machines:
+
{| class="wikitable"
−
* bowie: hosts and exports user files; Jupyterhub; landing server
+
|+ CS machines and cluster machines
−
* smiley: VM host, not accessible to regular users
+
|-
−
* web: website host
+
! Machine name !! 159 Ip Address !! 10Gb Ip address !! Operating System !! Metal or Virtual !! Description
−
* net: network administration host for CS
+
|-
−
* code: GitLab host
+
| Bowie || 159.28.22.5 || 10.10.10.15 || Debian 9 || Metal || hosts and exports user files; Jupyterhub; landing server
−
* auth: host of the LDAP user database
+
|-
 +
| Smiley || 159.28.22.251 || 10.10.10.252 || Ubuntu 18.04 || Metal || VM host, not accessible to regular users
 +
|-
 +
| Web || 159.28.22.2 || 10.10.10.200 || Ubuntu 18.04 || Virtual || Website host
 +
|-
 +
| Auth || 159.28.22.39 || No 10Gb internet|| CentOS 7 || Virtual || host of LDAP user database
 +
|-
 +
| Code || 159.28.22.42 || 10.10.10.42 || Ubuntu 18.04 || Virtual || Gitlab host
 +
|-
 +
| Net || 159.28.22.1 || 10.10.10.100 || Ubuntu 18.04 || Virtual || network administration host for CS
 +
|-
 +
| Lovelace || 159.28.23.35 || 10.10.10.35 || CentOS 7 || Metal || Example
 +
|-
 +
| Hopper || 159.28.23.1 || 10.10.10.1 || Debian 10 || Metal || landing server, NFS host for cluster
 +
|-
 +
| Sakurai || 159.23.23.3 || 10.10.10.3 || Debian 10 || Metal || Runs Backup
 +
|-
 +
| Miyamoto || 159.28.23.45 || Fill in || Debian 10 || Metal || Runs Backup
 +
|-
 +
|HopperPrime || 159.28.23.142 || 10.10.10.142 || Debian 10 || Metal || Runs Backup
 +
|-
 +
| Monitor || 159.28.23.250 || No 10Gb internet || CentOS 7 || Metal || Server Monitoring
 +
|-
 +
| Bronte || 159.28.23.140 || No 10Gb internet || CentOS 7 || Metal || Example
 +
|-
 +
| Layout 0 || 159.28.23.2 || 10.10.10.2 || CentOS 7 || Metal || Example
 +
|-
 +
| Layout 1 || fill in || fill in || CentOS 7 || Metal || Example
 +
|-
 +
| Layout 2 || fill in || fill in || CentOS 7 || Metal || Example
 +
|-
 +
| Layout 3 || fill in || fill in || CentOS 7 || Metal || Example
 +
|-
 +
| Whedon || 159.28.23.4 || No 10Gb internet|| CentOS 7 || Metal || Example
 +
|-
 +
| Pollock || 159.28.23.8 || 10.10.10.8 || CentOS 7 || Metal || Example
 +
|}
 +
CS machines:  
 +
bowie.cs.earlham.edu web.cs.earlham.edu auth.cs.earlham.edu code.cs.earlham.edu net.cs.earlham.edu
  
−
Cluster machines:
+
Cluster Machines: lovelace.cluster.earlham.edu hopper.cluster.earlham.edu hopperprime.cluster.earlham.edu sakurai.cluster.earlham.edu bronte.cluster.earlham.edu whedon.cluster.earlham.edu pollock.cluster.earlham.edu layout.cluster.earlham.edu monitor.cluster.earlham.edu miyamoto.cluster.earlham.edu
−
* hopper: landing server
 
−
* bronte, pollock, lovelace: large compute servers
 
−
* layout, whedon: clusters of multiple nodes linked together through a switch and managed through a headnode
 
−
* sakurai: big data storage and exports
 
−
* meier, miyamoto: backup servers
 
−
* monitor: server monitoring
 
  
−
We have spare nodes on the old al-salam cluster’s rack. These should be used for services that can handle minutes to hours of downtime, as they only have one power supply.
 
  
−
==== Specialized resources ====
+
There are 6 machines currently not in use in the 6 spaces above Monitor on the Equitorial Guinea rack
 +
=== Specialized resources ===
  
 
Specialized computing applications are supported on the following machines:
 
Specialized computing applications are supported on the following machines:
  
−
* GPU’s for AI/ML/data science: layout cluster
+
* [[Sysadmin:GPGPU|GPU’s for AI/ML/data science]]: layout cluster
 
* virtualization: smiley
 
* virtualization: smiley
 
* containers: bowie
 
* containers: bowie
  
−
=== Network ===
+
== Network ==
  
 
We have two network fabrics linking the machines together. There are three subdomains.
 
We have two network fabrics linking the machines together. There are three subdomains.
  
−
==== 10 Gb ====
+
=== 10 Gb ===
  
 
We have 10Gb fabric to mount files over NFS. Machines with 10Gb support have an IP address in the class C range 10.10.10.0/24 and we want to add DNS to these addresses.
 
We have 10Gb fabric to mount files over NFS. Machines with 10Gb support have an IP address in the class C range 10.10.10.0/24 and we want to add DNS to these addresses.
  
−
==== 1 Gb (cluster, cs) ====
+
=== 1 Gb (cluster, cs) ===
  
 
We have two class C subnets on the 1Gb fabric: 159.28.22.0/24 (CS) and 159.28.23.0/24 (cluster). This means we have double the IP addresses on the 1Gb fabric that we have on the 10Gb fabric.
 
We have two class C subnets on the 1Gb fabric: 159.28.22.0/24 (CS) and 159.28.23.0/24 (cluster). This means we have double the IP addresses on the 1Gb fabric that we have on the 10Gb fabric.
Line 89: Line 86:
 
Any user accessing *.cluster.earlham.edu and *.cs.earlham.edu is making calls on a 1Gb network.
 
Any user accessing *.cluster.earlham.edu and *.cs.earlham.edu is making calls on a 1Gb network.
  
−
==== Intra-cluster fabrics ====
+
=== Intra-cluster fabrics ===
  
−
The layout cluster has an Infiniband infrastructure. Whedon has only a 1Gb infrastructure.
+
The layout cluster has an Infiniband infrastructure. Wachowski has only a 1Gb infrastructure.
  
−
=== Power ===
+
== Power ==
  
 
We have a backup power supply, with batteries last upgraded in 2019 (?). We’ve had a few outages since then and power has held up well.
 
We have a backup power supply, with batteries last upgraded in 2019 (?). We’ve had a few outages since then and power has held up well.
  
−
=== HVAC ===
+
== HVAC ==
  
 
HVAC systems are static and are largely managed by Facilities.
 
HVAC systems are static and are largely managed by Facilities.
Line 104: Line 101:
  
 
[[Sysadmin:Layers of abstraction for filesystems|A word about what's happening between files and the drives they live on.]]
 
[[Sysadmin:Layers of abstraction for filesystems|A word about what's happening between files and the drives they live on.]]
 +
 +
 +
= New sysadmins =
 +
 +
These pages will be helpful for you if you're just starting in the group:
 +
 +
* [[Sysadmin:New Sysadmins | Welcoming a new sysadmin ]]
 +
* [[Sysadmin:Troubleshooting|General troubleshooting tips for admins]]
 +
* [[Sandbox Notes|Sandbox Notes]]
 +
* [[Password managers]]
 +
* [[Server safety]]
 +
* [https://code.cs.earlham.edu/sysadmin/ticket-tracker Ticket tracking for current projects]
 +
 +
Note: you'll need to log in with wiki credentials to see most Sysadmin pages.
 +
 +
= Additional information =
 +
 +
These pages contain a lot of the most important information about our systems and how we operate.
 +
 +
===Technical docs===
 +
 +
* [https://code.cs.earlham.edu/sysadmin/ticket-tracker Ticket tracking for current projects]
 +
* [[Server safety]]
 +
* [[Sysadmin:Backup|Backup]]
 +
* [[Sysadmin:Monitoring | Monitoring ]]
 +
* [[Sysadmin:SSH|SSH info relevant to admins]]
 +
* [[Sysadmin:User Management | User Management]] and [[Sysadmin:LDAP|LDAP]] generally
 +
* [[Sysadmin:Jupyterhub Notebook Server|Jupyterhub]] and [[Nbgrader notes|NBGrader]]
 +
* [[Sysadmin:MailStack|Email service]]
 +
* [[Sysadmin:XenDocs | Xen Server]]
 +
* [[Sysadmin:NFS|Network File System (NFS)]]
 +
* [[Sysadmin:Web Servers|Web Servers and Websites]]
 +
* [[Sysadmin:Services:Databases|Databases]]
 +
* [[Sysadmin:DNS & DHCP|DNS and DHCP]]
 +
* [[Sysadmin:AWS|AWS]]
 +
* [[Bash_start_up_script|Bash startup scripts]]
 +
* [[Sysadmin:VirtualBox | VirtualBox]]
 +
* [[X Applications]]
 +
* [[Sysadmin:Services:ClusterOverview|Cluster Overview]] and [[Sysadmin:Ccg-admin|additional details]]
 +
* [[Sysadmin:Firewall|Firewall]] running on babbage.cs.e.e
 +
 +
===Common tasks===
 +
* [[Sysadmin:Recurring Tasks | Recurring tasks - e.g. software updates, hardware replacements]]
 +
* [[Sysadmin:Contacting all users|Contacting all users]]
 +
* [[Reset password]]
 +
* [[Sysadmin:Software installation | Software installation]]
 +
* [[Modules | Installing software under modules ]]
 +
* [[Sysadmin:AddComputer|Add a computer to CS or cluster domains]]
 +
* [[Senior projects|Supporting senior projects]]
 +
* [[ShutdownProcedure|How to do a planned shutdown and reboot of the system]]
 +
** [[Sysadmin:TestingServices | Testing services]] (after a reboot, upgrade, change in the phase of the moon, etc.)
 +
* [[Sysadmin:Upgrading SSL Certificate | Upgrading SSL Certificates ]]
 +
* [[Sysadmin:Launch at startup|Launch a process at startup]]
 +
 +
===Group and institution information===
 +
* [[Sysadmin:CS-ITS Interoperability|Working with ITS]]
 +
* [[Sysadmin:Recurring spending | Recurring spending ]]
 +
* [[Sysadmin:SlackAndGitLab | Slack and GitLab integration]]

Revision as of 20:16, 2 December 2021

This is the hub for the CS sysadmins on the wiki.

Overview

If you're visually inclined, we have a colorful and easy-to-edit map of our servers here!

Server room

Our servers are in Noyes, the science building that predates the CST. For general information about the server room and how to use it, check out this page.

Columns: machine name, IPs, type (virtual, metal), purpose, dies, cores, RAM

Compute (servers and clusters)

CS machines and cluster machines
Machine name 159 Ip Address 10Gb Ip address Operating System Metal or Virtual Description
Bowie 159.28.22.5 10.10.10.15 Debian 9 Metal hosts and exports user files; Jupyterhub; landing server
Smiley 159.28.22.251 10.10.10.252 Ubuntu 18.04 Metal VM host, not accessible to regular users
Web 159.28.22.2 10.10.10.200 Ubuntu 18.04 Virtual Website host
Auth 159.28.22.39 No 10Gb internet CentOS 7 Virtual host of LDAP user database
Code 159.28.22.42 10.10.10.42 Ubuntu 18.04 Virtual Gitlab host
Net 159.28.22.1 10.10.10.100 Ubuntu 18.04 Virtual network administration host for CS
Lovelace 159.28.23.35 10.10.10.35 CentOS 7 Metal Example
Hopper 159.28.23.1 10.10.10.1 Debian 10 Metal landing server, NFS host for cluster
Sakurai 159.23.23.3 10.10.10.3 Debian 10 Metal Runs Backup
Miyamoto 159.28.23.45 Fill in Debian 10 Metal Runs Backup
HopperPrime 159.28.23.142 10.10.10.142 Debian 10 Metal Runs Backup
Monitor 159.28.23.250 No 10Gb internet CentOS 7 Metal Server Monitoring
Bronte 159.28.23.140 No 10Gb internet CentOS 7 Metal Example
Layout 0 159.28.23.2 10.10.10.2 CentOS 7 Metal Example
Layout 1 fill in fill in CentOS 7 Metal Example
Layout 2 fill in fill in CentOS 7 Metal Example
Layout 3 fill in fill in CentOS 7 Metal Example
Whedon 159.28.23.4 No 10Gb internet CentOS 7 Metal Example
Pollock 159.28.23.8 10.10.10.8 CentOS 7 Metal Example

CS machines: bowie.cs.earlham.edu web.cs.earlham.edu auth.cs.earlham.edu code.cs.earlham.edu net.cs.earlham.edu

Cluster Machines: lovelace.cluster.earlham.edu hopper.cluster.earlham.edu hopperprime.cluster.earlham.edu sakurai.cluster.earlham.edu bronte.cluster.earlham.edu whedon.cluster.earlham.edu pollock.cluster.earlham.edu layout.cluster.earlham.edu monitor.cluster.earlham.edu miyamoto.cluster.earlham.edu


There are 6 machines currently not in use in the 6 spaces above Monitor on the Equitorial Guinea rack

Specialized resources

Specialized computing applications are supported on the following machines:

Network

We have two network fabrics linking the machines together. There are three subdomains.

10 Gb

We have 10Gb fabric to mount files over NFS. Machines with 10Gb support have an IP address in the class C range 10.10.10.0/24 and we want to add DNS to these addresses.

1 Gb (cluster, cs)

We have two class C subnets on the 1Gb fabric: 159.28.22.0/24 (CS) and 159.28.23.0/24 (cluster). This means we have double the IP addresses on the 1Gb fabric that we have on the 10Gb fabric.

Any user accessing *.cluster.earlham.edu and *.cs.earlham.edu is making calls on a 1Gb network.

Intra-cluster fabrics

The layout cluster has an Infiniband infrastructure. Wachowski has only a 1Gb infrastructure.

Power

We have a backup power supply, with batteries last upgraded in 2019 (?). We’ve had a few outages since then and power has held up well.

HVAC

HVAC systems are static and are largely managed by Facilities.

See full topology diagrams here.

A word about what's happening between files and the drives they live on.


New sysadmins

These pages will be helpful for you if you're just starting in the group:

Note: you'll need to log in with wiki credentials to see most Sysadmin pages.

Additional information

These pages contain a lot of the most important information about our systems and how we operate.

Technical docs

Common tasks

Group and institution information

Retrieved from "https://wiki.cs.earlham.edu/index.php?title=Sysadmin&oldid=17980"