GUI Testing Checklist

Section 1 - Windows Compliance Standards

1.1. Application
1.2. For Each Window in the Application
1.3. Text Boxes
1.4. Option (Radio Buttons)
1.5. Check Boxes
1.6. Command Buttons
1.7. Drop Down List Boxes
1.8. Combo Boxes
1.9. List Boxes

Section 2 - Tester's Screen Validation Checklist

2.1. Aesthetic Conditions
2.2. Validation Conditions
2.3. Navigation Conditions
2.4. Usability Conditions
2.5. Data Integrity Conditions
2.6. Modes (Editable Read-only) Conditions
2.7. General Conditions
2.8. Specific Field Tests
       2.8.1. Date Field Checks
       2.8.2. Numeric Fields
       2.8.3. Alpha Field Checks

Section 3 - Validation Testing - Standard Actions

3.1. On every Screen
3.2. Shortcut keys / Hot Keys
3.3. Control Shortcut Keys

Section 4 - Origin & Inspiration

4.1. Document origin
4.2. Sources of Inspiration & information
4.3. Contacting the author

Automation and Testing Check List

Here's the scenario I'm thinking about: you need to deploy a standardized set of packages and configurations to a bunch of servers. You put together a checklist detailing the steps you need to take on each server -- kickstart the box, run some post-install scripts, do some configuration customization, etc. At this point, you're already ahead of the game, and you're not relying solely on human memory. However, if you rely on a human being going manually through each step of the checklist on each server, you're in for some surprises in the guise of missed steps. The answer of course is to automate as many steps as you can, ideally all of them.

Now we're getting to the main point of my post: assuming you did automate all the steps of the checklist, and you ran your scripts on each server, do you REALLY have that warm and fuzzy feeling that everything is OK? You don't, unless you also have a comprehensive automated test suite that runs on every server and actually checks that stuff happened the way you intended.

Here are some concrete examples of stuff I'm verifying after the deployment of a certain type of our servers (running Apache/Tomcat).

OS-specific tests

1. Does the sudoers file contain certain users that need those rights
2. Is the sshd process set to start at boot time
3. Is the ClientAliveInterval variable set correctly in /etc/sshd/sshd_config
4. Are certain NFS mount points defined in /etc/fstab, and do they actually exist on the server
5. Is sendmail set to start at boot time, and running