Requirements & Specifications by Eushiuan Tran (Carnegie Mellon University )

QA: Requirements, Specifications, Scenarios, Test Cases and Test Sets
-------------------------------
The requirements pyramid
-------------------------------
✤ Needs :: contains high-level requirements (vague user desires and wishes)
✤ Features :: while needs come from the end user, the features are formulated by business analysts and customer's marketing department.
✤ Scenarios :: Any situation that may occur in production, or any sequence of events which may occur when the end user is in interaction with the software system
✤ Use cases :: Use Case is one specific scenario, which are written by business analysts on the basis of customer needs and requirements.
✤ Input data sets :: based on the use case document and delivered by the customer, it is also useful to have up front Input/corresponding Output data sets for testing
✤ Test Cases :: engineers prepare test cases, based on the use case document
✤ supplementary requirements
✤ The User interface proposal :: GUI, or CLI, or NUI
✤ Technical details of hardware or software
✤ Glossary of specific terms
✤ The User Manual proposal


Whereas Test Case are the documented steps, which are written on the basis of use cases.
Use cases are prepared from the Functionality requirement document.
At the top level are stakeholder needs. On the lower levels are features, use cases, and supplementary requirements. Quite often, at different levels of these requirements, different levels of detail are captured. The lower the level, the more detailed the requirement.

--------------
Abstract:
--------------
Defining requirements to establish specifications is the first step in the development. However, in many situations, not enough care is taken in establishing correct requirements up front. This causes problems when ambiguities in requirements surface later in the life cycle, and more time and money is spent in fixing these ambiguities. Therefore, it is necessary that requirements are established in a systematic way to ensure their accuracy and completeness, but this is not always an easy task. This difficulty in establishing good requirements often makes it more of an art than a science. The difficulty arises from the fact that establishing requirements is a tough abstraction problem and often the implementation gets mixed with the requirements. In addition, it requires people with both communication and technical skills. As requirements are often weak about what a system should not do, this poses potential problems in the development of dependable systems, where these requirements are necessary to ensure that the system does not enter an undefined state. The development of dependable embedded systems requires even more complicated requirements as the embedded system not only interacts with the software but also with the outside world. Therefore, the importance of establishing good requirements is even greater in embedded systems design.

--------------
Introduction
--------------
Requirements and specifications are very important components in the development of any embedded system. Requirements analysis is the first step in the system design process, where a user's requirements should be clarified and documented to generate the corresponding specifications. While it is a common tendency for designers to be anxious about starting the design and implementation, discussing requirements with the customer is vital in the construction of safety-critical systems. For activities in this first stage has significant impact on the downstream results in the system life cycle. For example, errors developed during the requirements and specifications stage may lead to errors in the design stage. When this error is discovered, the engineers must revisit the requirements and specifications to fix the problem. This leads not only to more time wasted but also the possibility of other requirements and specifications errors. Many accidents are traced to requirements flaws, incomplete implementation of specifications, or wrong assumptions about the requirements. While these problems may be acceptable in non-safety-critical systems, safety-critical systems cannot tolerate errors due to requirements and specifications. Therefore, it is necessary that the requirements are specified correctly to generate clear and accurate specifications.
There is a distinct difference between requirements and specifications. A requirement is a condition needed by a user to solve a problem or achieve an objective. A specification is a document that specifies, in a complete, precise, verifiable manner, the requirements, design, behavior, or other characteristics of a system, and often, the procedures for determining whether these provisions have been satisfied. For example, a requirement for a car could be that the maximum speed to be at least 120mph. The specification for this requirement would include technical information about specific design aspects. Another term that is commonly seen in books and papers is requirements specification which is a document that specifies the requirements for a system or component. It includes functional requirements, performance requirements, interface requirements, design requirements, and developement standards. So the requirements specification is simply the requirements written down on paper.

------------------------------------
requirements and specifications
------------------------------------
There is a distinct difference between requirements and specifications. A requirement is a condition needed by a user to solve a problem or achieve an objective. A specification is a document that specifies, in a complete, precise, verifiable manner, the requirements, design, behavior, or other characteristics of a system, and often, the procedures for determining whether these provisions have been satisfied. For example, a requirement for a car could be that the maximum speed to be at least 120mph. The specification for this requirement would include technical information about specific design aspects.
Another term that is commonly seen in books and papers is requirements specification which is a document that specifies the requirements for a system or component. It includes functional requirements, performance requirements, interface requirements, design requirements, and developement standards. So the requirements specification is simply the requirements written down on paper.
http://users.ece.cmu.edu/~koopman/des_s99/requirements_specs/
...........................................
Good Test Cases /Cem Kaner
............................................
In practice, many things are referred to as test cases even though they are far from being fully documented.

Brian Marick uses a related term to describe the lightly documented test case, the test idea:
“A test idea is a brief statement of something that should be tested. For example, if you're testing a square root function, one idea for a test would be ‘test a number less than zero’. The idea is to check if the code handles an error case.”

In my view, a test case is a question that you ask of the system. The point of running the test is to gain information, for example whether the program will pass or fail the test.
If the documentation is an essential aspect of a test case, in your vocabulary, please substitute the term “test idea” for “test case” in everything that follows.

----------------------
Acceptance testing
----------------------
Acceptance testing is a testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria of a given use case. It is usually performed by the expert-user, customer or other authorized entity to determine whether or not the system is acceptable. A test pilot is an aviator who tests new aircraft by fling specific maneuvers. In aeronautics top pilots, navigators and engineers conduct flight tests and at the end of the test missions they will provide evaluation and certification data.

---------------------------------------
Establishing Correct Requirements
---------------------------------------
The first step toward developing accurate and complete specifications is to establish correct requirements. As easy as this sounds, establishing correct requirements is extremely difficult and is more of an art than a science. There are different steps one can take toward establishing correct requirements. Although some of the suggestions sound fairly obvious, actually putting them into practice may not be as easy as it sounds. The first step is to negotiate a common concept with the customer/stakeholder.

There is a quote by John von Neumann that states "There's no sense being exact about something if you don't even know what you're talking about."

Communication between the business analysts, system designer/architect and customer is vital. There is no point in trying to establish exact specifications if the business analysts and customers cannot even agree on what the requirements are.
Problem stems from ambiguities in stating requirements. Ambiguous requirements can be caused by missing requirements, ambiguous words, or introduced elements. There are also missing requirements. "a group of people" in the above requirement is an example of ambiguous words. What exactly does "group" imply? A group can consist of 5 people, 100 people, 1000 people, etc.
It is important to eliminate or at least reduce ambiguities as early as possible because the cost of them increases as we progress in the development life cycle.
Often the problem one has in establishing correct requirements is how to get started. One of the most important things in getting started is to ask questions.
what is the reason for solving this problem?,
what environment is this product likely to encounter?,
and what are the trade-offs ?.
These questions force both sides, to look at the higher issues. Also, since these questions are appropriate for any project, they can be prepared in advance. Another important point is to get the right people involved. There is no point in discussing requirements if the appropriate people are not involved in the discussion. Having effective meetings is not as easy as it sounds. However, since they play a central role in establishing requirements it is essential to know how to make meetings work.

Exploring the possibilities is another important step toward generating correct requirements. Every project will also encounter conflicts. Conflicts can occur from personality clashes, intergroup prejudice such as those between technical people and marketing people, and level differences. It is important that a facilitator is present to help resolve conflicts.

In establishing requirements, it is important to specifically establish:
- functions,
- attributes,
- constraints,
- preferences,
- and expectations of the product.
Usually in the process of gaining information, functions are the first ones to be defined. Functions describe what the product is going to accomplish.
It is also important to determine the attributes of a product.
Attributes are characteristics desired by the client, and while 2 products can have similar functions, they can have completely different attributes. After all the attributes have been clarified and attached to functions, we must determine the constraints on each of the attributes. Preferences, which is a desirable but optional condition placed on an attribute, can also be defined in addition to its constraints.
Finally, we must determine what the client's expectations are. This will largely determine the success of the product.
Testing is the final step on the road to establishing correct requirements. There are several testing methods used, as listed below. [Gause89]
Ambiguity poll - Used to estimate the ambiguity in a requirement. This involves asking questions such as how fast?, how big?, how expensive?, and then determining if there is ambiguity between the high and low values.

Technical review - A testing tool for indicating the progress of the requirements work. It can be formal or informal and generally only deals with technical issues. Technical reviews are necessary because it is not possible to produce error-free requirements and usually it is difficult for the producers to see their own mistakes.
User satisfaction test - A test used on a regular basis to determine if a customer will be satisifed with a product.
Black box test cases - Constructed primarily to test the completeness, accuracy, clarity, and conciseness of the requirements.
Existing products - Useful in determining the desirable and undesirable characteristics of a new product.
At some point it is necessary to end the requirements process as the fear of ending can lead to an endless cycle. This does not mean that it is impossible to revisit the requirements at a later point in the development life cycle if necessary. However, it is important to end the process when all the necessary requirements have been determined, otherwise you will never proceed to the design cycle.
Establishing good requirements requires people with both technical and communication skills. Technical skills are required as the embedded system will be highly complex and may require knowledge from different engineering disciplines such as electrical engineering and mechanical engineering. Communication skills are necessary as there is a lot of exchange of information between the customer and the designer. Without either of these two skills, the requirements will be unclear or inaccurate.
It is essential that requirements in safety critical embedded systems are clear, accurate, and complete. The problem with requirements is that they are often weak about what a system should not do. In a dependable system, it is just as important to specify what a system is not suppose to do as to specfiy what a system is suppose to do. These systems have an even greater urgency that the requirements are complete because they will only be dependable if we know exactly what a system will do in a certain state and the actions that it should not perform. Requirements with no ambiguities will also make the system more dependable. Extra requirements will usually be required in developing a dependable embedded system. For example, in developing a dependable system for non-computer-literate people, extra requirements should be specified to make the system safe even in exceptional or abusive situations.

------------------------------------------------------
Difference between test cases and use cases
------------------------------------------------------
>Needs / prepared by Marketing, contains high-level customer requirements and vague desires
>>Use cases / prepared by Business Analysts, and are written on the basis of customer needs and desires.
>>>Test cases / prepared by engineers, and are based on the use case document.
Use case will explain desired walk-through in one use scenario of a software application or a system.
Use cases are generally prepared by Business Analysts, to better understand the functionality of the application, generally use case contains preconditions, field validations, event lists, postconditions, flow graph, prototype.
Based on the use case document test engineers prepare test cases.
Use Case are the various scenario, which are written on the basis of customer/supplier requirements.
Whereas Test Case are the documented steps, which are written on the basis of use cases.
Use cases are prepared from the Functionality requirement document.
For example, a need might be “Data should be persistent.” The feature can refine this requirement to be “System should use a relational database.” On the supplementary specification level, the requirement is even more specific: “System should use Postgres database.” The further down, the more detailed the requirement. One of the best practices of requirements management is to have at least two different levels of requirement abstraction. For example, the Vision contains high-level requirements (features), and the lower levels in the pyramid express the requirements at a detailed level. Senior stakeholders (such as vice presidents) do not have time to read 200 pages of detailed requirements but should be expected to read a 12-page Vision document.
However, it is up to business analysts to decide on the granularity of requirements at each level. Nothing is wrong with placing quite detailed requirements from stakeholders on the stakeholder needs level.
The main difference between needs and features is in the source of the requirement. Needs come from stakeholders, and features are formulated by business analysts.
The role of test cases is to check if use cases and supplementary requirements are implemented correctly. Scenarios help derive use cases from test cases and facilitate the design and implementation of specific paths through use cases. In RequisitePro we can define many other requirement types, such as glossary terms and actors. They are not pure requirements conforming to the definition provided at the beginning of this chapter, but if we represent them in RequisitePro as requirements, we gain flexibility to track their attributes and traceability using same mechanisms that are provided for other requirement types.