Quality Assurance & Testing

QA & Testing
1. Positive and Negative testings;
2. White Box vs Black Box
3. Functional, Non-Functional, UI and Acceptance testing;
4. Implicit verifications and explicit verifications;
5. Systematic testing and ad-hoc;
6. Ad-hoc and Exploratory testing
7. Unit (Component) - Integration - System - Acceptance testing

...................
Testing Levels
...................
There are four basic levels of testing that are commonly used within various models:
- Component (unit) testing
- Integration testing
- System testing
- Acceptance testing
Acceptance testing: Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
System testing: The process of testing an integrated system to verify that it meets specified requirements.
Integration testing: Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems.
Component testing: The testing of individual software components.

Acceptance testing is a testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria of a given use case. It is usually performed by the expert-user, customer or other authorized entity to determine whether or not the system is acceptable.
A test pilot is an aviator who tests new aircraft by fling specific maneuvers. In aeronautics top pilots, navigators and engineers conduct flight tests and at the end of the test missions they will provide evaluation and certification data.
........................
Test Types
There are several key types of tests that help improve the focus of the testing:
Functional testing: testing specific functions
Non-functional testing: testing characteristics of the software and system
Structural testing: testing the software structure or architecture
Re-testing (confirmation) and regression testing: testing related to changes
White and black box testing
Each test type focuses on a particular test objective
Test objective: A reason or purpose for designing and executing a test.
Test object: The component or system to be tested.
Test item: The individual element to be tested. There usually is one test object and many test items

......................
Functional testing:
Testing based on an analysis of the specification of the functionality of a component or system.
The functions are "what" the system does:
They are typically defined or described in work products such as a requirements specification, use cases, or a functional specification;
They may be undocumented;
Functional tests are based on both explicit and implicit features and functions;
They may occur at all test levels, e.g., tests for components may be based on a component specification;
Functional testing focuses on the external behavior of the software (black-box testing).

.........................................................
Non-Functional testing/ i.e. Quality testing:
Focuses on "how" the system works;
Non-functional tests are those tests required to measure characteristics of systems and software that can be quantified;
These quantifications can vary and include items such as: response times, throughput, capacity for performance testing etc.
Testing the attributes of a component or system that do not relate to functionality, e.g. reliability, efficiency, usability, maintainability, compatibility and portability


................................
Structural (White box) testing:
Testing based on an analysis of the internal structure of the component or system / architecture of the system, aspects such as a calling hierarchy, data flow diagram, design specification, etc.;
May may be performed at all test levels - system, system integration, or acceptance levels (e.g., to business models or menu structures);
Structural techniques are best used after specification-based techniques;
Can assist in measuring the thoroughness of testing by assessing the degree of coverage of a structure;
Tools can be used to measure the code coverage

..........................
Static Testing:
Testing of a component or system at specification or implementation level without execution of the software;
Non-execution based method for checking life cycle artifacts;
Manual static techniques: reviews (inspections, walkthroughs etc.), formal and informal;
Automated techniques: supporting reviews, static analysis tools (compilers);
Anything can be reviewed – PRD, Specs, Memos, Proposals, User Guides etc.

............................................
Catching an Error -> Bug Report
............................................
Reproducing an error;
Reporting an error:
Bug report – main elements – structure:
- ID #;
- Title/short description;
- What is the problem (what happened);
- Where the problem occurred;
- Long description (steps to reproduce);
- Fix Priority;
- Bug Severity;
- Development stage: Project ID / milestone / release / version.

-----------------------------------------------------------------------------
BLACK spots, Road Map, Development Plan

Induced crush testing before 1.0 release. We are committed to provide functionality and service that meet or exceed promised performance, and the use of real-life data with some random garbage input. Testing will be performed to prove such functionality in the induced stress conditions.

* Robustness: how well software solution reacts to invalid input problems not due to programmer error. This includes situations such as incorrect, inappropriate or corrupt data, unavailability of needed resources such as memory, operating system services and network connections, and user error.

* Usability: the ergonomics of a program: the ease with which a person can use the program for its intended purpose, or in some cases even unanticipated purposes. Such issues can make or break its success even regardless of other issues. This involves a wide range of textual, graphical and sometimes hardware elements that improve the clarity, intuitiveness, cohesiveness and completeness of a program's user interface.

Risks, issues and problems
All risks, issues and problems must be managed on an ongoing basis using appropriate management processes and forms and logs to record all these items.
A great tip is to try and resolve any issues or risks as soon as they are identified, in order to minimize the impact they may have on your project.
Problematic issues must be discussed and prioritized with the relevant stakeholders on an ongoing basis and their impact on the project must be track and monitored.

Project health checks
Tracking projects at a low level is part of ongoing project management, but you must also be able to every now and then during the project’s existence sit back and review the project from a higher level to gain a view of the project’s overall health and status. By taking a weekly summarized view of the project, you will be able to more effectively manage the project and lead the team.


..................................
What’s a test case?
..................................
Let’s start with the basics. What’s a test case?
IEEE Standard 610 (1990) defines test case as follows:
“(1) A set of test inputs, execution conditions, and expected results developed for a
particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

“(2) (IEEE Std 829-1983) Documentation specifying inputs, predicted results, and a set of execution conditions for a test item.”

According to Ron Patton (2001, p. 65),
“Test cases are the specific inputs that you’ll try and the procedures that you’ll follow when you test the software.”

Boris Beizer (1995, p. 3) defines a test as “A sequence of one or more subtests executed as a sequence because the outcome and/or final state of one subtest is the input and/or initial state of the next. The word ‘test’ is used to include subtests, tests proper, and test suites.

Good Test Cases /Cem Kaner

Bob Binder (1999, p. 47) defines test case:
“A test case specifies the pretest state of the IUT and its environment, the test inputs or conditions, and the expected result. The expected result specifies what the IUT should produce from the test inputs. This specification includes messages generated by the IUT, exceptions, returned values, and resultant state of the IUT and its environment. Test cases may also specify initial and resulting conditions for other objects that constitute the IUT and its environment.”

In practice, many things are referred to as test cases even though they are far from being fully documented.

Brian Marick uses a related term to describe the lightly documented test case, the test idea:
“A test idea is a brief statement of something that should be tested. For example, if you're testing a square root function, one idea for a test would be ‘test a number less than zero’. The idea is to check if the code handles an error case.”

In my view, a test case is a question that you ask of the program. The point of running the test is to gain information, for example whether the program will pass or fail the test.

It may or may not be specified in great procedural detail, as long as it is clear what is the idea of the test and how to apply that idea to some specific aspect (feature, for example) of the product.
If the documentation is an essential aspect of a test case, in your vocabulary, please substitute the term “test idea” for “test case” in everything that follows.
An important implication of defining a test case as a question is that a test case must be reasonably capable of revealing information.