Things certainly have changed.  I remember a day when testing meant writing up a set of scenarios to manually test, or having a checklist of features to repeatedly test against on deployment day.  Now, with modern tools, there are lots more ways to test your code.  Automation can repeatedly test those scenarios for you, and ensure you didn’t break your code when new changes are made.  You can use these automation tests in lots of different ways, along with some basic things to keep in mind to ensure those tests are as valuable as possible.

Basic Types of Automation

1. Unit Testing with Code Coverage – Write reusable test cases that test your functions, methods, and classes.  Integrate them into your automated build pipeline, and measure how much of your code is tested.  Aim for 100%, but expect a bit less.  Some unit testing tools are jUnit, nUnit, CA LISA, Rhino Mocks, and many more.
2. Functional / System Testing – Write tests from your customer’s perspective.  walk through your application in the browser, fill in the fields with predefined values, and submit.  Test if the data is correct, if the validations worked, if the data was submitted.  Common tools for this are Selenium, Cucumber, CA Lisa, Rhino Mocks, etc.
3. Browser and Device Testing – The Browser Wars may be over, but we still have a fragmented landscape of browsers used, and you have to test them all – Chrome, Edge, IE, Safari, Firefox, Android, iOS, phones, tablets, desktops… To cover them all, you can automate your tests and integrate with tools like BrowserStack, SauceLabs, Selenium, etc.
4. Beta / Acceptance Testing – You need to ensure that you have built a quality project that meets the requirements of the business and the customer.  This is usually a set of manual tests completed by your customer before and / or after launch.
5. Install Testing – This subset of existing automation tests ensures you exercise as much of your codebase as possible in a new environment, particularly production, to ensure your server installation has completed properly.
6. Smoke Testing – A small subset of existing automation tests, most probably used in other cases, that ensures any environment changes have not had an impact on your codebase and its functionality.  These tests usually follow the happy paths (or red routes).
7. Regression Testing – a complete and thorough suite of tests that exercise all of your code, new features and old, happy path and all known edge cases, including past and present bugs, ensuring you are prepared for a launch to production.
8. Performance / Load / Stress Tests – Identify break points and thresholds in your software and hardware, and improve.  Common tools include jMeter, LoadRunner, Silk Performer, and many others.
9. Security / Penetration tests – Make your code more secure, prevent hacking and attacks.  Tools include SonarQube, OWASP ZAP, Nexpose, WireShark, Retina, Aquila, and many more.

Tips To Keep In Mind

10. Keep your code simple – Keep your code focused on solving one problem at a time.  This makes it easier to read and understand, code correctly, debug, and maintain.
11. Pair Programming – Two heads are better than one.  Write your code cleaner, faster, and with less bugs.  This method comes with built in code reviews.
12. Use Design Patterns and Architecture Patterns – Follow these, and you are using proven methods that other developers will understand.  The Gang of Four is a good place to start learning design patterns, and there are many others.  MVC, MVP, MVVM, and VIPER are some common architecture patterns that you can use to organize your code at an application level.
13. Comment and Document your Code – Do this as close to the code as possible.  This makes your code easier to read and understand.  One way I like to do this is write pseudocode first, then fill in with actual code.  Then the pseudocode becomes natural code commenting.
14. Automated Builds and Continuous Integration – Run your code and test cases as often as possible, make sure they all work together.  Jenkins is the most common way to achieve this.  Another common tool is Bamboo.
15. Demos – this forces you to think holistically about your code, how you will communicate your solution to others, and gather feedback from your audience.  If you follow an agile methodology, chances are this is already part of your process.

4E38RY7KZA4J

I just got an email about a really cool new tool built into Windows 7 that Microsoft used to debug their new platform. It is called Problem Step Recorder. The best thing to do is to post a snippet of the email right here. I think it says everything perfectly:

“In case you’re not aware of this, here is a little known Microsoft tool bundled with Windows 7 that can be extremely useful to illustrate a problem when testing an application. The diagnostic tool called “Problem Step Recorder” was originally produced by Microsoft during the development of Windows 7 Beta to assist their Quality Assurance team in debugging the OS. It uses a combination of screen captures with mouse tracking to record your actions and can be a great way of describing a problem to others. The program is launched from the Start menu by typing ‘psr’ or ‘psr.exe’ in the search field. You’ll get a floating applet that looks like this: When you hit the Record button, the applet tracks your mouse and keyboard input while taking screenshots that correspond to each new action. When you stop recording your session is saved to an HTML slide show that recreates your steps. It also allows you to add comments to further document the problem. I think it can be very useful as an attachment in [your bug tracking tool] for those hard to describe issues or as a “How To” document for end users.”

Which leads to other ways of doing this… you could youse WebEx or Windows Media Encoder to document any bug as a step-by-step. If you use WatiN, Selenium, or VS2010, you can also use their recorders to document any bugs you may find in a web application, hand that to the dev team, and then there is no guessing how to reproduce the bug.

Kudos to Microsoft, and to the folks who uncovered this!