SAAS Software Checklist

Assessment Purpose


The purpose of this checklist is to provide guidance to assess and evaluate SAAS solution’s security and other features and determine key risks and considerations. These questions act as guidelines to consider in the evaluation of the software acquisition. Many of these are addressed by the Higher Ed Community Vendor Assessment Toolkit (HECVAT), and this document serves to define many of the more technical areas to review.

Instructions

Computer and Information Systems will work with the area interested in the Cloud / Hosted / SaaS solution to review the software and gather the answers to the applicable questions below via: direct inquiry, vendor responses on the HECVAT, and reference calls to other customers.


When referencing the solution provider’s policy(ies) in the questionnaire below, include the policy name, section number and hyperlink (if available).

Table of Contents


SAAS Software Checklist


Project Information

What is the service provided?

What is the business purpose?

Who are the key SPU stakeholders? (name, title, email)

Evaluate the importance of the SAAS tool / system to SPU. Consider how SPU would be harmed if:

  • The tool / system became widely public and / or widely distributed
  • An employee of the provider accessed the tool / system in an unauthorized manner
  • The data / process / function were manipulated by an outsider
  • The process / function failed to provide expected results
  • The information/data were unexpectedly changed
  • The asset was unavailable for a period of time

What institutional data will be stored as or transmitted by the provider?


SAAS Provider

Proposed provider(s) / company name(s)

Who is our primary contact? (name, email, phone)

Who is our technical contact? (name, email, phone)

Is the solution provider an industry leader, small player, niche player or new-comer?

What is the size of the solution provider’s operations – consider number of employees, annual revenues, etc.

History: how long has the solution provider been in business?

List the provider’s current / prior higher education clients, if known

What are your financials? Are you funded by a VC firm? Whom?

Does the vendor have a trial or demo program?


Terms of Service / SLA

Is there a cap on liability?

Does the provider have cyber risk insurance in place? If so, please provide coverage details.

Does the provider have an active SLA in place that identifies minimum performance (e.g., up time, etc.)?

Does the provider provide regular service management reports (e.g., SLA performance)? If so, state the frequency of such reporting.

Describe penalties associated with SLA non-compliance.

Does the provider use a third party to provide the required services? If so, explain the services to be provided by the third party, and the type of relationship between the provider and the third party.


Data Transmission and Integration

What integration services / technologies are provided? Flat-file, JSON, API?

What are limitations on integration services? Field limitations, read/write only data, number of API calls, bandwidth caps, limited transmissions / day, etc.

How is data transmission secured? What level of encryption is used?

Are data transfers manual or can they be automated?

Will the service / solution require integration with other SPU data systems, either on-premise or in the cloud?

Do clients have direct database access? How can data be retrieved for analysis in analytics or BI tools?

What are the provider's terms when it comes to ownership of data? How about any metadata generate while using the application?

Which, if any transmission or integration services, have extra fees?


Backup and Disaster Recovery

Does the provider have a business continuity or disaster recovery plan? If so, attach, if possible and indicate when it was last tested.

Does the provider have a failover site? If so, is the failover site certified to the same standards as the primary facility? Please describe.

What is the provider's backup & recovery SLA?  What are the actual results/metrics vs. the SLA for the last 12 months?

What is the frequency of client data back-ups? Are backups encrypted?

Are data back-ups stored on-site or off-site? If the data is stored off-site, does a sub-contractor store it? If so, list all relevant sub-contractors.

What is the process to restore data from the provider’s back-up?


Security and Confidentiality

Does your company have a corporate security policy?

Does your company have a dedicated security team? If so, roughly how many people are on it?

What application security measures are used in the production environment (e.g., application-level firewall, database logging / auditing, etc.)?

What security scanning, monitoring or testing is done to ensure the services is resilient against malicious code, data breach, or other attacks?

Does the provider have access to SPU data, and if so, what restrictions are there over this level of access?

Can any third party access SPU data, and if so, how?

Do the intellectual property rights of SPU data remain intact (if applicable)? Does the provider retain rights to SPU data even if data is removed from the provider?

What security features exist for data transmitted back and forth between the user and the provider? Third parties?

What is the provider’s incident response procedure for handling a security or data breach. Does the provider have a cyber security plan in place? If so, please provide details.

Have there been any major security incident(s) reported with the provider in the last two years? If so, detail the incident(s) and resolution(s).

Does the provider perform regular vulnerability assessments / penetration tests to determine security gaps? Will SPU have access the provider's SOC 1/2 and / or any other independent security / penetration test / control audit / assessment report(s)?

Describe the provider’s reporting mechanism for security and / or other incidents. In what time frame do notification go out and what information do they contain?


Maintenance and Support

What are the provider’s customer support hours? Do these work for the University area considering the solution?

Does the provider have meaningful problem response, timeline, and resolution commitments?

Does the service provider have change management policies in place?

What level of testing is available for customers prior to updates? How much lead-time will the service provider give SPU of upcoming changes? How will we notified of changes?

Is a test environment provided to clients?

Are security risks identified and addressed during the software development life -cycle?

How does the provider accommodate customer requirements into their product strategy? Is there a customer advisory council?

What is the provider's product road-map and strategy?

Do any support options have extra fees?


Legal and Compliance

Have all regulatory requirements been identified? If so, by whom? Outline all regulatory requirements.

Provide / attach evidence of PCI-DSS compliance, if applicable. Does the contract state that the provider will provide evidence of compliance to Brock as soon as finalized? If not, why not?

Is the provider SAS70, FERPA, HIPAA compliant?

Where is the providers hosting environment located? Is any client data stored or transmitted outside the United States?

What are your data ownership and retention policies?


Operational Controls

Does the service provider outsource hosting of their application and data storage servers to a third-party? Provide details.

Does the service provider have an information security audit (SSAE16) or evaluation program for their operation? If yes, provide a copy.

What technology platform(s) is your system built upon? Are you using modern technology and industry best practices to build and maintain your system infrastructure? 

Are customers single or multi-tenant?

What are your uptime/availability statistics? 


Authentication

What authentication services does the provider support?

What Single-Sign-On solutions does the provider support?

How is account provisioning and revocation handled? Can accounts and access controls be management by an external identity management system?

Are there extra fees for any authentication services or options?


Contract Termination

Describe the process to terminate the service.

What happens to SPU's data at service termination?

Can SPU retrieve its data post-contract termination? Are export utilities available and easy to use?

Will SPU data be permanently erased from the solution, including any backup storage, when this data is deleted or the service ended?

Specify any fees that may be incurred at the end of the service.

Does SPU have the right to terminate if the provider introduces material modifications to service terms? Is there a right of termination for material breach of applicable privacy and security obligations?