Learning Objectives
By the end of this lesson, you will be able to:
- Define data integrity and explain its importance
- Differentiate between data validation and verification
- Describe and apply various data validation methods (range check, format check, length check, presence check, existence check, limit check, check digit)
- Explain and use data verification methods during data entry (visual check, double entry, check digits)
- Explain and use data verification methods during data transfer (parity check, checksum)
- Calculate check digits for codes like ISBN
- Differentiate between data security and data integrity
Key Terms
Data Integrity
Accuracy, completeness and consistency of data
Validation
Checking if entered data is reasonable and within given criteria
Verification
Checking that data has been accurately copied or transferred
Presence Check
Ensuring that an entry field is not left blank
Format Check
Checking that characters entered conform to a pre-defined pattern
Length Check
Checking that data contains an exact number of characters or numbers
Range Check
Checking that only numbers within a specified range are accepted
Limit Check
Checking only one of the limits such as upper limit or lower limit
Type Check
Ensuring that correct type of data is entered into a field
Existence Check
Checking that a file exists with filename referred to in data entry
Check Digit
Final digit included in a code calculated from all other digits
Double Entry
Data entered twice and compared to ensure accuracy
Visual Check
Manual check completed by user who is entering data
Checksum
Method to check if data has been changed or corrupted during transmission
Parity Check
Method to check whether data has been changed or corrupted after transmission
Parity Bit
Extra bit added to data to make number of 1-bits even or odd
Data Security
Protection of data from illegal access or loss
Introduction to Data Integrity
Data integrity means accuracy, completeness and consistency of data. It ensures that data received is the same as data sent or data copied is the same as the original. Data integrity maintains validity of data, making sure data is correct and not corrupted.
Why Data Integrity Matters
- Ensures reliable decision-making based on accurate data
- Prevents errors in critical systems (banking, healthcare, aviation)
- Maintains trust in digital systems and transactions
- Reduces costs associated with data errors and corrections
Threats to Data Integrity
- During data entry and data transmission stages
- By malicious attacks on data, for example caused by malware and hacking
- By accidental data loss caused through hardware issues
- Human errors during data input or processing
Real-Life Example: Online Banking
When you transfer money online, data integrity ensures that:
- The amount you enter (£100) is the same amount deducted from your account
- The account number you type reaches the bank's system unchanged
- Your balance is calculated correctly after the transaction
- No digits are swapped or lost during the transmission
Without data integrity, £100 could become £1000, or your money could go to the wrong account!
Important: Difference Between Data Security and Data Integrity
Data Security:
- Protects data from illegal access/loss
- Focuses on confidentiality and availability
- Examples: Passwords, encryption, firewalls
- Prevents unauthorized users from seeing or modifying data
Data Integrity:
- Deals with validity of data
- Ensures data is free from errors
- Examples: Validation checks, verification methods
- Prevents data corruption after input or transmission
Simple Analogy:
Think of a locked diary (security) vs a diary with accurate entries (integrity). Security keeps people out, integrity ensures what's written inside is correct.
Check Your Understanding: Data Integrity Basics
1. Define data integrity in your own words. [2 marks]
Answer
- [1 mark] Data integrity refers to the accuracy, completeness and consistency of data
- [1 mark] It ensures that data received is the same as data sent, or data copied is the same as the original
- [Additional] It maintains validity of data, making sure data is correct and not corrupted
2. Give three examples of how data integrity can be compromised. [3 marks]
Answer
- [1 mark] During data entry and data transmission stages
- [1 mark] By malicious attacks on data, for example caused by malware and hacking
- [1 mark] By accidental data loss caused through hardware issues
- [Additional] Human errors during typing or processing data
3. What is the main difference between data security and data integrity? [2 marks]
Answer
- [1 mark] Integrity deals with validity of data, data should be free from errors
- [1 mark] Security deals with protection of data from illegal access/loss
- [Additional] Security protects data from unauthorized access while integrity ensures data is not corrupted after input or transmission
Data Validation
Validation is a method of checking if entered data is reasonable and within a given criteria but it cannot check if data is correct or accurate. Validation is automatically carried out by computer software.
Important: What Validation CANNOT Do
Validation checks if data is reasonable but not if it's correct.
Example: If somebody accidentally enters their age as 62 instead of 26, it is reasonable (could be a person's age) but not accurate or correct.
Types of Validation Checks
Presence Check
Ensures that an entry field is not left blank.
Format Check
Checks that characters entered conform to a pre-defined pattern.
Length Check
Checks that data contains an exact number of characters or numbers.
Range Check
Checks that only numbers within a specified range are accepted.
Limit Check
Checks only one of the limits such as upper limit or lower limit.
Type Check
Ensures that correct type of data is entered into that field.
Existence Check
Checks that a file exists with filename referred to in data entry.
Data Validation Simulator
Test different validation checks on sample data. Try entering values that should pass or fail validation.
How it works: Each validation check tests specific rules:
- Format check: Pattern matching (dates, emails)
- Range check: Value within minimum and maximum
- Length check: Exact character count
- Type check: Correct data type (numbers, text)
Real-Life Example: Online Shopping Form
When you shop online, validation checks ensure your information is entered correctly:
Credit Card Details
Shipping Address
These validation checks prevent common errors but cannot detect if you accidentally enter wrong but valid information (like wrong card number that happens to be 16 digits).
Activity 1: Designing Validation Rules
Design appropriate validation rules for the following scenarios:
- A school registration form needs to accept student ages. What validation would you use?
- An online ticket booking system requires seat numbers from 1 to 50. What check is needed?
- A password field requires exactly 8 characters. What validation check applies?
- A date of birth field must be in the format DD-MM-YYYY. What check should be used?
- An email field cannot be left empty. What validation is required?
Solution:
- Student ages: Range check (e.g., 4-18) and Type check (numeric only)
- Seat numbers 1-50: Range check (1 to 50) and Type check (integers only)
- Password 8 characters: Length check (exactly 8 characters)
- Date format DD-MM-YYYY: Format check (pattern matching) and Range check for days (1-31) and months (1-12)
- Email cannot be empty: Presence check (field must not be blank)
Check Your Understanding: Data Validation
1. What is the main limitation of data validation? [2 marks]
Answer
- [1 mark] Validation checks if data is reasonable but cannot check if it is correct or accurate
- [1 mark] Example: Entering age as 62 instead of 26 passes validation (reasonable age) but is incorrect
2. Explain the difference between range check and limit check. [2 marks]
Answer
- [1 mark] Range check ensures data is within a specified range (e.g., month 1-12)
- [1 mark] Limit check checks only one limit such as upper or lower limit (e.g., maximum age 120)
3. Give an example of when you would use a format check. [2 marks]
Answer
- [1 mark] When data must follow a specific pattern or structure
- [1 mark] Example: Date field requiring dd/mm/yyyy format or email requiring name@domain.com
4. Why is type check important in data validation? [2 marks]
Answer
- [1 mark] Ensures the correct data type is entered (e.g., numbers for age, text for name)
- [1 mark] Prevents processing errors that could occur if wrong data type is used in calculations
5. When would you use an existence check? [1 mark]
Answer
- [1 mark] When checking that a file exists with filename referred to in data entry
- [Additional] Example: When attaching a file to an email or uploading a document
Data Verification
Verification is checking that data has accurately copied onto computer or transferred from one computer system to another. Verification can be used to make sure that data in your database is correct. Verification is a way of preventing errors when data is entered manually (using keyboard) or when data is transferred from one computer to another.
Verification During Data Entry
Double Entry
Data is entered twice, computer compares both entries and outputs an error message requesting that data is entered again if they are different.
Not ideal for large amounts of data. Could take a lot of time to enter data twice. User could enter same mistake twice and so it wouldn't get picked up.
Visual Check
A screen/visual check is a manual check completed by user who is entering data. When data entry is complete, data is displayed on screen and user is asked to confirm that it is correct before continuing.
Check Digits
A check digit is final digit included in a code; it is calculated from all other digits in code. When a number is entered into computer (with check digit), the check digit is recalculated and, if same value is not generated an error will occur.
Check Digits in Detail
When a series of numbers are used to identify something, it is possible to use a check digit method of verification.
Applications of Check Digits:
- Barcodes (product codes)
- International Standard Book Numbers (ISBN)
- Vehicle Identification Numbers (VIN)
- Credit card numbers
- Bank account numbers
Errors Detected by Check Digits:
- Phonetic Errors (e.g., 13 instead of 30)
- Misplaced or Extra Digits (e.g., 247 instead of 2407)
- Transposition Error (e.g., 4087 instead of 4807)
- Incorrect digit entered (e.g., 5327 instead of 5307)
Check Digit Calculation Example: ISBN
Calculating check digit for 10-digit ISBN: 1 84146 201 ?
| Step | Calculation | Result |
|---|---|---|
| 1. Assign weights | Digits weighted 10 to 2 from left | 1(10), 8(9), 4(8), 1(7), 4(6), 6(5), 2(4), 0(3), 1(2) |
| 2. Multiply & sum | (1×10)+(8×9)+(4×8)+(1×7)+(4×6)+(6×5)+(2×4)+(0×3)+(1×2) | 10+72+32+7+24+30+8+0+2 = 185 |
| 3. Divide by 11 | 185 ÷ 11 = 16 remainder 9 | Remainder = 9 |
| 4. Subtract from 11 | 11 - 9 = 2 | Check digit = 2 |
When remainder is 10, letter X is used as check digit. Full ISBN: 1 84146 201 2
Check Digit Calculator
Enter a 9-digit number to calculate its ISBN check digit:
Calculation Steps:
| Digit | - | - | - | - | - | - | - | - | - |
|---|---|---|---|---|---|---|---|---|---|
| Weight | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 |
| Product | - | - | - | - | - | - | - | - | - |
How it works: ISBN check digit calculation uses modulus 11 system:
- Multiply each digit by weights from 10 down to 2
- Sum all the products
- Divide sum by 11 and find remainder
- Check digit = 11 - remainder (if remainder 10, use X)
Verification During Data Transfer
When data is transferred electronically from one device to another, there is possibility of data corruption or even data loss. Several methods exist to minimize this risk.
Checksums
Checksum is method to check if data has been changed or corrupted during data transmission. Data is sent in blocks and an additional value, checksum, is sent at end of block of data.
- If sum of bytes ≤ 255, checksum = that exact value
- If sum of bytes > 255, checksum = remainder after dividing by 256
Bytes total 1,151 → 1,151 / 256 = remainder 127 → Checksum = 127
How Checksums Work
Parity Checks
Parity check is method to check whether data has been changed or corrupted following transmission from one device or medium to another.
Types of Parity:
- Even parity: Total number of 1-bits (including parity bit) is even
- Odd parity: Total number of 1-bits (including parity bit) is odd
Example Byte with Parity:
Data byte: 1 1 0 1 1 0 0 (seven 1-bits)
- Even parity: Parity bit = 1 (makes total 8 ones = even)
- Odd parity: Parity bit = 0 (keeps total 7 ones = odd)
Limitation of Parity Bit
If there are multiple errors in same byte/column that still produce same parity bit, error will not be detected.
"00001111" (4 ones) changes to "00000111" (3 ones) with even parity:
Original: 0 00001111 (even parity = 0)
Corrupted: 0 00000111 (even parity = 0) → Error not detected!
Parity Block Checks
More advanced method that can detect and correct single errors by checking both rows (bytes) and columns (bit positions).
Parity Check Simulator
How parity works: The parity bit is added to make total number of 1s either even or odd.
- Even parity: Parity bit = 1 if data has odd number of 1s, 0 if even
- Odd parity: Parity bit = 1 if data has even number of 1s, 0 if odd
- Receiver counts 1s, if count doesn't match expected parity, error detected
Real-Life Example: Wi-Fi Data Transmission
When you stream a video over Wi-Fi, multiple verification methods ensure data integrity:
Checksums
Parity Bits
Retransmission
Without these verification methods, your video would have glitches, frozen frames, or missing audio.
Activity 2: Verification Problems
Solve the following verification problems:
- Calculate the check digit for ISBN: 0 306 40615 ?
- A data block has bytes totaling 850. What is the checksum?
- For the 7-bit data "1011001", what parity bit is needed for even parity?
- Give two limitations of double entry verification.
- What type of error would a check digit detect if "13" was entered instead of "30"?
Solution:
-
ISBN 0 306 40615:
Weights: 0(10), 3(9), 0(8), 6(7), 4(6), 0(5), 6(4), 1(3), 5(2)
Sum: (0×10)+(3×9)+(0×8)+(6×7)+(4×6)+(0×5)+(6×4)+(1×3)+(5×2) = 0+27+0+42+24+0+24+3+10 = 130
130 ÷ 11 = 11 remainder 9
11 - 9 = 2
Check digit = 2 -
Checksum for 850:
850 ÷ 256 = 3 remainder 82
Checksum = 82 -
"1011001" with even parity:
Data has 4 ones (even)
For even parity, parity bit = 0 (keeps total even)
Result: 0 1011001 -
Limitations of double entry:
1. Not ideal for large amounts of data (time-consuming)
2. User could make same mistake twice, so error wouldn't be detected - Error type: Phonetic error (thirteen vs thirty)
Check Your Understanding: Data Verification
1. What is the main purpose of verification? [2 marks]
Answer
- [1 mark] Checking that data has been accurately copied onto computer or transferred between systems
- [1 mark] Making sure data in database is correct and preventing errors during manual entry or transfer
2. Explain how a check digit works. [3 marks]
Answer
- [1 mark] Check digit is final digit included in a code, calculated from all other digits
- [1 mark] When number is entered, check digit is recalculated
- [1 mark] If recalculated value doesn't match original check digit, an error is detected
3. What is the difference between checksum and parity check? [3 marks]
Answer
- [1 mark] Checksum calculates sum/value from data bytes sent at end of block
- [1 mark] Parity check adds extra bit to make total 1-bits even or odd
- [1 mark] Checksum works on blocks of data, parity works on individual bytes
4. Why might double entry verification fail? [2 marks]
Answer
- [1 mark] User could enter the same mistake twice
- [1 mark] Comparison would show they match, so error wouldn't be detected
- [Additional] Also time-consuming for large amounts of data
5. What happens during a parity check if a byte has an error? [2 marks]
Answer
- [1 mark] Receiver counts number of 1-bits in received byte (including parity bit)
- [1 mark] If count doesn't match expected parity (even/odd), error is detected
- [Additional] System may request retransmission of the data
Key Takeaways
- Data integrity means accuracy, completeness and consistency of data
- Validation checks if data is reasonable but cannot check if it's correct
- Verification checks that data has been accurately copied or transferred
- Validation methods include presence, format, length, range, limit, type, and existence checks
- Verification during data entry includes double entry, visual check, and check digits
- Check digits are calculated from other digits and can detect phonetic, transposition, and digit errors
- Verification during data transfer includes checksums and parity checks
- Checksums are calculated from data bytes and sent with the data block
- Parity checks add an extra bit to make number of 1-bits even or odd
- Even parity ensures even number of 1-bits; odd parity ensures odd number
- Data security protects data from illegal access; data integrity ensures data is free from errors
- ISBN check digits use modulus 11 system with weights from 10 down to 2
- Parity limitation: Multiple errors that maintain same parity won't be detected
Question Bank
1. Compare and contrast data validation and data verification. [6 marks]
Marking Scheme & Answer
| Aspect | Validation | Verification |
|---|---|---|
| Purpose | Checks if data is reasonable and within criteria | Checks data has been accurately copied/transferred |
| What it checks | If data follows rules/patterns (format, range, etc.) | If data matches original source |
| Limitation | Cannot check if data is correct (e.g., age 62 vs 26) | Can detect copying/transmission errors |
| When used | During data entry | During data entry AND data transfer |
| Methods | Presence, format, length, range, type checks | Double entry, visual check, check digits, checksums, parity |
| Automation | Automatically carried out by software | Can be manual (visual) or automatic (checksums) |
2. Describe three different validation checks with examples. [6 marks]
Marking Scheme & Answer
- [2 marks] Format check: Checks characters conform to pre-defined pattern. Example: Date must be dd/mm/yyyy
- [2 marks] Range check: Checks numbers within specified range. Example: Month in date must be 1-12
- [2 marks] Length check: Checks exact number of characters. Example: Telephone number must be 11 digits
- [Additional] Other valid answers: Presence check (field not blank), Type check (correct data type), Limit check (upper/lower limit), Existence check (file exists)
3. Calculate the check digit for ISBN: 0 7475 3269 ? Show all steps. [5 marks]
Marking Scheme & Answer
- [1 mark] Assign weights: 0(10), 7(9), 4(8), 7(7), 5(6), 3(5), 2(4), 6(3), 9(2)
- [2 marks] Multiply and sum: (0×10)+(7×9)+(4×8)+(7×7)+(5×6)+(3×5)+(2×4)+(6×3)+(9×2) = 0+63+32+49+30+15+8+18+18 = 233
- [1 mark] Divide by 11: 233 ÷ 11 = 21 remainder 2
- [1 mark] Check digit: 11 - 2 = 9
- [Additional] Full ISBN: 0 7475 3269 9
4. Explain how checksums are used in data transmission. [4 marks]
Marking Scheme & Answer
- [1 mark] Data is sent in blocks with checksum value at end
- [1 mark] Sender calculates checksum from data bytes before transmission
- [1 mark] Receiver recalculates checksum from received data
- [1 mark] If calculated checksum matches transmitted checksum, data is error-free; if not, retransmission is requested
- [Additional] Calculation: If sum ≤ 255, checksum = sum; if sum > 255, checksum = remainder after dividing by 256
5. Describe the parity check method and its limitations. [5 marks]
Marking Scheme & Answer
- [1 mark] Parity check adds extra bit (parity bit) to data
- [1 mark] Even parity: Total 1-bits (including parity) is even
- [1 mark] Odd parity: Total 1-bits (including parity) is odd
- [1 mark] Receiver counts 1-bits; if count doesn't match expected parity, error detected
- [1 mark] Limitation: Multiple errors that maintain same parity won't be detected (e.g., two bits flip)
- [Additional] Example: "00001111" (4 ones) → "00000111" (3 ones) with even parity: parity bit stays 0, error not detected
6. Design validation rules for a student registration system. [6 marks]
Marking Scheme & Answer
- [1 mark] Student ID: Format check (e.g., STU-XXXX where X=digit) and length check
- [1 mark] Name fields: Presence check (cannot be blank) and type check (text only)
- [1 mark] Date of birth: Format check (dd/mm/yyyy), range check (day 1-31, month 1-12), and limit check (e.g., must be at least 4 years old)
- [1 mark] Email: Format check (must contain @ and domain) and presence check
- [1 mark] Phone number: Length check (11 digits) and type check (numbers only)
- [1 mark] Course selection: Existence check (course must exist in system) and range check (max 5 courses)
7. Explain why both validation and verification are needed for data integrity. [4 marks]
Marking Scheme & Answer
- [1 mark] Validation prevents obviously wrong data from being entered (e.g., month 13, text in numeric field)
- [1 mark] Verification ensures data is accurately copied/transferred without corruption
- [1 mark] Validation alone cannot catch all errors (e.g., age 62 instead of 26 passes validation)
- [1 mark] Verification alone cannot prevent invalid data entry (e.g., would verify incorrect but accurately copied data)
- [Additional] Together they provide comprehensive data integrity: validation for input quality, verification for accuracy of copying/transmission
8. What types of errors can check digits detect? Give examples. [4 marks]
Marking Scheme & Answer
- [1 mark] Phonetic errors: 13 (thirteen) instead of 30 (thirty)
- [1 mark] Misplaced or extra digits: 247 instead of 2407 or 42107 instead of 4207
- [1 mark] Transposition errors: 4087 instead of 4807 (two numbers swapped)
- [1 mark] Incorrect digit entered: 5327 instead of 5307
9. Calculate the checksum for bytes totaling: a) 200 b) 500 c) 1200 [3 marks]
Marking Scheme & Answer
- [1 mark] a) 200: ≤ 255, so checksum = 200
- [1 mark] b) 500: 500 ÷ 256 = 1 remainder 244, checksum = 244
- [1 mark] c) 1200: 1200 ÷ 256 = 4 remainder 176, checksum = 176
10. Explain the difference between data security and data integrity with examples. [4 marks]
Marking Scheme & Answer
| Aspect | Data Security | Data Integrity |
|---|---|---|
| Focus | Protection from illegal access/loss | Validity of data, freedom from errors |
| What it does | Prevents unauthorized access, maintains confidentiality | Ensures data is accurate, complete, consistent |
| Examples | Passwords, encryption, firewalls, access controls | Validation checks, verification methods, check digits |
| When applied | Protects data throughout its lifecycle | Applied during data entry and transmission |
Simple analogy: Security is like a locked safe (keeps data safe), integrity is like an accurate ledger (ensures data is correct).