Designing a self assessment Survey

Before you start to design a self-assessment survey, check to see if your agency has a self-assessment template that already exists that they would like you to use. Since you’re probably under a deadline, don’t recreate a brand-new self-assessment survey if a pretty good one already exists at your agency. Also, it may be against the agency security policies to use a survey that is different than the one they provide. If your agency does not have a self-assessment survey template, you will need to develop one before you can answer the questions. Special Publication 800-26 contains a fairly comprehensive sample survey and it’s a great starting point for developing one for your Certification Package.You’ll likely want to modify the survey you find in Special Publication 800-26 to make it more apropos to the objectives of your agency and information system.

Special Publication 800-26 recommends that your survey be designed for five levels of compliance. However, since almost every C&A program includes four levels of compliance, from a practical standpoint, it makes more sense to build four levels of compliance into your survey.The recommendation for five levels of compliance originated from a document published on November 28, 2000 known as the Federal Information Technology Security Assessment Framework (FITSAF).1 Since most C&A programs have only four levels of compliance, it is possible that if the FITSAF had been published after FISMA was passed, it may have included only four levels of compliance.

Levels of Compliance

To keep things simple, I suggest referring to the levels of compliance on your survey simply as L1, L2, L3, and L4.These levels of compliance should be consistent with the C&A levels. We refer to each question of the survey simply as a control.The compliance levels are simply boxes to check off next to the survey question, and should be interpreted as follows:

■ L1 indicates the security control is written into policy
■ L2 indicates that the security control is implemented
■ L3 indicates that the security control is tested
■ L4 indicates that the security control has passed all tests and is tightly integrated

Each compliance level includes the requirements from the prior level. It makes it very easy to understand at which C&A level your information systems are able to be certified if you design your self-assessment this way. Keep in mind that even if your information systems can be certified and accredited at a higher level, and each control item complies up to L4, that doesn’t mean that you should certify and accredit your information systems at the highest level. As already discussed, you should never certify and accredit your information system at a level higher than what is necessary. Table below illustrates a commonly used format for self-assessment survey questions.

Self-Assessment Survey Format

A checkmark in the L1 box indicates that there is a security policy that requires the control to exist, and a checkmark in the L2 box indicates that the control has been implemented. A checkmark in the L3 box indicates that tests have been performed on the implementation, and a checkmark in the L4 box indicates that all tests have been passed and that the control is tightly integrated into the information system. You’re probably wondering what is meant by “tightly integrated.”Tight integration is a somewhat nebulous term, and to be sure, although NIST and many security experts use this terminology commonly, there is no one agreed upon definition for what it means. My recommendation is that tight integration is something that you can justify through evidence and demonstration in one of the following ways:

■ Automated technical features
■ A strict change control process
■ A robust configuration management process
■ An online workflow process that includes levels of approvals and sign-offs

For example, if a network vulnerability assessment is performed automatically on a regular basis, according to a published schedule, it may qualify as “implemented” in compliance with L2 requirements. If all vulnerabilities are mitigated each time an automated network vulnerability assessment occurs, you could then claim that risk assessments are performed on a regular basis up to compliance level L4. If a network vulnerability assessment is performed now and then, but not on any regular schedule, and known vulnerabilities are recorded but not immediately mitigated, you could claim that risk assessments are in compliance with level L3. If network vulnerability assessments are required to be completed, and one is scheduled to occur but hasn’t occurred yet, then you could claim that the information system was in compliance up to level L2. If there was simply a policy that existed for network vulnerability assessments to be completed, whether any vulnerability assessments were actually completed or not, you could justifiably claim your information system is in compliance with level L1.

Management Controls

The survey questions should be designed to discover the truths about the management controls and should be focused on the following key areas:
■ Risk mitigation
■ Reporting and review by management
■ System lifecycle requirements
■ Security planning
■ Security oversight
■ Documentation for managers

You want the survey questions to uncover how well the management team complies with agency security policies, and how well they manage and oversee the operational and technical controls. Management is about budgeting, tracking, reporting, communications, accountability, and analysis. Questions surrounding security management controls should be designed with these elements in mind.

Operational Controls

Operational controls focus on processes and procedures that are implemented by people.The survey questions surrounding the operational controls should be geared toward finding out if the processes and procedures designed to control security work as planned.The discoveries made from responses to operational survey questions should be concerned with how well the people who administer the systems carry out their daily duties. Survey questions related to operational security controls should be focused on the following key areas:

■ Personnel security
■ Physical and environmental operations and safeguards
■ Administration and implementation
■ Preventative maintenance
■ Contingency and disaster recovery planning
■ Training and security awareness
■ Incident response procedures
■ Preservation of data integrity (antivirus, intrusion detection, etc.)
■ Network and system security operations
■ Documentation for operational staff

An important part of surveying operational controls is to find out if there is a clear separation of duties between the different administrative roles. In general, duties should be separated so that access to operations is available according to the principle of least privilege—users should be given no more privileges than absolutely necessary to do their jobs.

Preservation of data integrity and confidentiality issues should be investigated by questions regarding operational controls. As one example, data integrity questions should be designed to find out how antivirus programs are managed. An example of confidentiality questions would include those questions designed to find out about background checking processes for key personnel.

Technical Controls

Technical controls refer to the security safeguards that are built into the information systems. Survey questions should be designed to find out the status of the built-in technical controls. The type of information that you are trying to uncover is to find out if technical controls exist, and if they do, whether they are effective.

The key areas that technical controls focus on are:
■ Authentication and identity verification
■ Logical access controls
■ Secure configurations
■ Interconnectivity security
■ Audit mechanisms

If a Security Self-Assessment has been designed to fit all the information systems in a particular agency, it will likely be the case that some of the questions designed for technical control assurances will not be applicable. Not all information systems will require the same types of technical controls.

Correlation with Security Policies and Laws

When putting together the self-assessment, it can be helpful to have agency security policies and federal guidance that require the controls, to be listed with the question. Controls that are required by FISCAM and OMB Circulars will be items that GAO inspectors will look for during an audit. Controls that are required by the agency are of interest to the agency OIG auditors. Guidance from NIST is also worth citing for reference purposes. OMB Circular A-130 often is used as guide for developing the self-assessment questions.

Answering the Questions

Once a survey is developed, it needs to be completed. You will need to interview the developers, subject matter experts, and management team in order to find out the answers to the questions. Interviews can be performed either in person or electronically. There are many nice survey tools that can be implemented that are designed to collect this information through a Web portal. It’s often the case that some of the survey respondents may be in disparate geographic locations. By setting up the survey through a Web portal, you can simply send out an e-mail asking the required participants to login and answer the questions. Many of the online survey tools offer roll-up scores and advanced graphs that allow you to see which control areas require more attention. As organizations refine their self-assessment methodology, implementing the survey via an online portal is really the way to go.

Oversight Requirements Depicted in Survey Questions

Self-assessments should not be done in an accusatory way that implies wrong-doing has occurred. The point of a self-assessment is to collect information designed for agency or organizational self-improvement. Respondents to the questions should not be made to feel guilty if a particular area is not in compliance. You want the respondents to give honest answers. It is very important that respondents understand up front that no negative repercussions will occur as a result of their answers. If honest answers are not provided on the self-assessment, it becomes useless.

In May of 2000, the Department of Energy released a report2 regarding various security incidents that had occurred at Los Alamos National Laboratory. It was discovered that respondents to security self-assessments answered survey questions purposefully incorrectly because they felt pressured to give the “right” answer as opposed to the truthful answer. As a result, various security vulnerabilities were never discovered, and so nothing was ever done to mitigate them. Since the security vulnerabilities were never mitigated, security incidents occurred that exploited the vulnerabilities. Not only is it unethical to intimidate self-assessment respondents into answering the questions untruthfully, it defeats the purpose of the exercise.

Similar to what happened at Los Alamos, in September of 2003, a report put out by the Office of Inspector General at the Environmental Protection Agency found that 36 percent of the responses to security self-assessments contained inaccurate information. Submission of inaccurate security self-assessments is a known problem. The intent of a self-assessment is for the ISSO and information system owner to use the self-assessment surveys internally, though auditors may try to find out if the information contained in them is accurate. As more attention is being cast on inaccurate security self-assessments, scrupulous auditors will be spending more time trying to verify the accuracy of the information contained in them. Truthfulness conveys trust, and if auditors discover that security self-assessments are not accurate, they may scrutinize other parts of the Certification Package more so than they would otherwise.

Evaluators may ask questions to try to ascertain if the ISSO and information system owner actually use their own self-assessments to take corrective actions. Presumably, if an information system owner finds out from the self-assessment process that the vulnerabilities discovered do not warrant pursuing C&A, an ethically upstanding information system owner would put the C&A process on hold until proper mitigation of vulnerabilities occurs. Keep in mind that information system owners do not need to wait until a C&A deadline is looming to conduct a security self-assessment. It is probably best to get started on your self-assessment long before the three-year C&A deadline is looming overhead.