Testing Programs

Testing verifies that a program behaves correctly — that it produces the expected output for all possible inputs, including unusual or invalid ones. Without testing, bugs reach users.

OCR J277 covers two aspects of testing: when to test (iterative vs final), and what data to use.

The purpose of testing:

• Confirm the program does what it is designed to do
• Find and fix errors (bugs) before the program is released
• Ensure the program handles invalid input without crashing
• Provide evidence that the program meets its specification

A program that works for the developer's test cases but fails for users has not been tested thoroughly. The choice of test data determines what errors are found.

Testing can only show the presence of bugs — it cannot prove a program is entirely bug-free. Thorough testing makes it much less likely that bugs remain.

Iterative testing happens during development.

(Extra context — OCR J277 uses the term "iterative testing"; "formative" and "modular" are alternative names used in other contexts but are not required OCR terminology.) Each component or subroutine is tested as soon as it is written, before moving on. Problems are caught and fixed early, when they are cheapest to address.

Final (terminal) testing happens when the complete program is finished. The whole system is tested together to ensure all components integrate correctly.

Feature	Iterative testing	Final/terminal testing
When	During development, module by module	At the end, on the complete program
What is tested	Individual subroutines or components	The full integrated system
When errors are found	Early — easier and cheaper to fix	Late — may require rework across modules
What it catches	Bugs in individual components	Integration bugs and end-to-end behaviour

Worked example: building a quiz program with three subroutines — displayQuestion, getAnswer, and updateScore. With iterative testing, each subroutine is tested immediately after writing. With final testing only, a bug in getAnswer may not be found until the entire program is assembled.

Most real development uses iterative testing throughout, with final testing as a final check before release.

Two error categories arise in programming:

Syntax error — the code breaks the grammatical rules of the programming language. The translator (compiler or interpreter) cannot parse it and the program will not run.

if score > 50    # missing colon — syntax error
    print("Pass")

Logic error — the code runs without crashing, but produces incorrect or unexpected output because the logic is wrong.

if score > 50:    # should be >= 50 for "pass at 50"
    print("Pass")
else:
    print("Fail")

A student with score exactly 50 would be told "Fail". No crash — just wrong output.

Error type	Detectable by	Effect
Syntax error	Translator (at compile/run time)	Program will not run at all
Logic error	Testing with appropriate data	Program runs but gives wrong results

A syntax error prevents execution. A logic error allows execution but produces wrong answers. You cannot find a logic error by reading the error messages — you find it by testing with carefully chosen data.

Test data is the set of inputs used to test a program. OCR J277 requires knowledge of four categories:

Category	Definition	Example (for age input, valid range 0–120)
Normal	Valid data that the program should accept without errors	25, 47, 100
Boundary	Valid data at the very edge of the acceptable range	0, 120 (the limits themselves)
Invalid	Correct data type but outside the acceptable range — should be rejected	-1, 121
Erroneous	Wrong data type entirely — should be rejected	"hello", 3.5, "" (empty)

Why test at the boundary? Boundary values are where off-by-one errors appear. A check written as age > 120 incorrectly accepts 120; a check written as age >= 120 incorrectly rejects 120. Normal values in the middle don't catch this bug — boundary values do.

A test plan documents the inputs to be tested, the expected results, and (after testing) the actual results. It provides evidence that systematic testing was carried out.

Worked example — test plan for a program that accepts scores from 0 to 100:

Test	Type	Input	Expected output	Actual output	Pass/Fail
1	Normal	50	"Valid score"
2	Normal	85	"Valid score"
3	Boundary	0	"Valid score"
4	Boundary	100	"Valid score"
5	Invalid	-1	"Invalid. Enter 0–100."
6	Invalid	101	"Invalid. Enter 0–100."
7	Erroneous	"abc"	Error or rejection message
8	Erroneous	45.5	Error or rejection message

A complete test plan includes at least one test from each of the four categories. Exam questions often provide a partially completed test plan and ask you to fill in missing entries.

Testing does not just find bugs — it drives algorithm refinement. When a test reveals unexpected output, the algorithm must be revised and re-tested.

Worked example — a sorting algorithm incorrectly handles a list with duplicate values:

Initial test:

• Input: [3, 1, 2] → Output: [1, 2, 3] ✓
• Input: [3, 1, 3] → Output: [3, 1, 3] ✗ (duplicates not handled)

The bug is found by testing a case the original developer did not consider. The algorithm is then revised and the test re-run to confirm the fix.

Refined test approach — always include test cases for:

(Extra context — the following edge-case list is good practice but goes beyond the OCR J277 2.3.2 requirement to "identify suitable test data for a given scenario.")

• Empty inputs (zero items)
• Lists with duplicate values
• Lists already in order
• Lists in reverse order
• Single-element lists

These represent the edge cases that reveal weaknesses in typical algorithm implementations.

(Extra context — OCR J277 2.3.2 only requires understanding that testing leads to algorithm revision; efficiency and robustness improvements are covered in other spec sections.) Refining algorithms means making them more correct (handles more cases), more efficient (faster or uses less memory), or more robust (handles bad input gracefully).

1. Confusing invalid and erroneous test data

Invalid data is the correct type but outside the valid range: for an age field, -1 is an integer (correct type) but invalid. Erroneous data is the wrong type entirely: "hello" is a string, not an integer — erroneous.

2. Forgetting boundary values on both sides

For a range of 0–100, the boundary values are 0 and 100 — both ends. Test both. Students often only test one boundary.

3. Classifying syntax errors as logic errors

If the program will not run, the error is syntax. If it runs but gives wrong output, it is logic. A missing colon in Python is a syntax error; the wrong comparison operator is a logic error.

4. Test plans with only normal data

A test plan that only includes values from the middle of the valid range misses boundary, invalid, and erroneous cases — exactly where bugs are most likely to be. Include all four categories.

Mistake	Correction
Using -1 as "erroneous" data for an age field	-1 is invalid (correct type, wrong range); erroneous would be "minus one" or 3.5
Boundary test for range 1–10 uses only 5	Boundary values are 1 and 10 — both ends
"Program crashes — that's a logic error"	A crash caused by unparseable code is a syntax error