Versions Compared
Key
- This line was added.
- This line was removed.
- Formatting was changed.
Introduction
Info | ||
---|---|---|
| ||
The Server Maintenance Policy in intended to describe procedures and guidelines around how the Infrastructure Systems Team maintains the server infrastructure necessary to sustain SPU's Enterprise Applications & Services |
Table of Contents
Table of Contents | ||||
---|---|---|---|---|
|
Effective Date: February 1, 2017
Last Review/Update: October 2020May 2021
Architecture Overview
The Core Services Infrastructure Systems Team manages over 270 260 virtual servers along with , the campus perimeter firewall, and several dedicated physical servers and storage platforms (Vidnet, DFS, Faith&Co). We use VMware ESXi as our preferred virtual environment hypervisor. We use a "template" process in VMware heavily use server virtualization software along with machine image "templates" to standardize our server builds, to automate the creation of newly requested servers, and dynamically manage compute and storage resources to best serve the SPU community. We currently offer Windows and Linux server builds in our virtual environment.
Twice-Annual Review
Beginning Fall 2020, IST began the process to review and bring all systems up to current stable versions twice per year, duringSystem Reviews and Updates
For systems and services that cannot be interrupted during the normal school year, IST updates these machines during the Christmas and Summer breaks. During these review periods we will examine each system and install all this time, IST reviews and then updates these systems with all the necessary cumulative firmware, OS and application patches and updates. In addition to these times of annual review, we conduct ad hoc assessments of needed system maintenance activities as recommended by the system vendor or industry advisories as noted below:
- OSSecurity Patches
All SPU server builds are configured to install
OSsecurity patches automatically. Linux machines check for and install patches nightly; Windows machines check for and install patches
nightly when possible, but no less frequently than weekly.- Applicationweekly based on a staggered schedule defined by machine group policy. Perimeter systems are updated automatically as pushed from our firewall vendor.
Application and Firmware Patches
Application and firmware patches are reviewed as we're notified of their availability from the respective vendors. Our general process for application and firmware patches involves:
-- Immediate installation of high-level (zero day) security patches that are recommended and verified by the vendor;
- Feature/functionality patches and step releases are applied as needed/recommended, but not necessarily immediately;
Unless there are extenuating circumstances, our goal is to keep systems on the latest major versions of software and firmware, with discretionary application of point/step releases between major revs. In most instances, major updates will be scheduled during the twice-annual lift rather than risk bringing systems down during times of peak utilization.
lowerLower-risk step upgrades will be considered on a case by case basis.
Backups
All SPU servers run daily backups; please see our Backup and Recovery Policy for more detail
Monitoring
We use PRTG to monitor over 2000 data points across our server fleet. These metrics include criteria such as:
- Network availability (Ping)
- Disk space / usage trends
- CPU Load
- Memory Usage
- Website Availability
- Custom SQL Queries
We use this data to establish baselines for what is deemed "normal" behavior – we then have alerting configured so that when the metrics report data outside the norm, the branches of CIS responsible for the particular server / service are notified for further investigation and remediation.
Log Files
Server log files are aggregated and copied off the servers directly to a centralized platform. We currently do not have any active monitoring / alerting on this data. This process and architecture is currently under review.