Now that we have walked through the basics in my posts on Testing 101, 200 Level, and 300 Level testing, the number of bugs in your production environment should be falling. To continue to eliminate the nasty bugs, it is time to dig deep, get creative, and use skills from other fields. Here are some out-of-the-box ideas to continue the momentum and focus on quality and stability of your software products.

1. Shift Left – this is the general principle to find bugs as soon as possible in the software development process – write automation tests to find bugs in QA, write unit tests to find bugs in development, use TDD to find bugs before you write code, architect testability into your platform before you even create the solution.  TechBeacon has a great article about Shift Left.
2. Crowdsource Testing – sometimes you need to get a fresh, new perspective on testing.  Using the general public can help you get there.  I have used a  company called uTest, and they were great.  They have all kinds of purchase options.  There are lots of other crowdsource testing companies to choose from.
3. Fuzzing – I have done this before and never knew it was called fuzzing.  Build automation tests that randomly generate characters (ISO-8859-1 and UTF-8) and submit forms, to test positive and negative cases you never thought of.  Run it thousands of times, and you are bound to find some errors.  Read F-Secure to learn more about fuzzing.
4. Innovation – When most people think of innovation, they think of a bolt of lightning, striking randomly. While this may be true some of the time, that isn’t always the case. There are methods to developing and trying new ideas, and that kind of innovation needs a culture to support it. This great article from The Medium outlines some of the methodologies to use to find innovation.
5. Design Thinking – Over the last few years, I have been working with Rutgers on their Advisory Board for their Design Thinking certificate program. I stay on top of the subject, and tweet a lot of great articles to read on the topic. When I came across this article on DT and Software Quality, I knew I needed to share it.
6. Bug Bash / Bug Hunt / Bug Day – Make finding bugs a social event. Invite your team, or other teams, to find bugs in your product. Set aside an afternoon, order some pizza, plan some prizes, and find some bugs! Read more about how to run a Bug Hunt.
7. Quality Hackathon – another way to generate ideas about new approaches to testing and quality, you could hold a hackathon, one that focuses specifically on testing. Are there new ways to automate? New tools to try? New approaches to testing? Challenge your team, set some parameters, pick some judges, give away some prizes, and make it fun!
8. Apply Gamification to Testing – In World of Warcraft, there are monsters to kill, skills to learn, reputation to gain, equipment to discover.  You can do the same thing with Quality and Gamification – each person on the team has a QE Score, and gets points for bugs found, classes taken, Tech Talks conducted, junior teammates mentored.   Define a reward system, how to grow, and publicly praise the growth – different levels, badges, and achievements to earn and the steps to get there.  Read more about gamification on CIO.com.
9. Machine Learning and Testing – Can we teach a machine to find bugs in software that humans write? Or identify what sections of our software systems will most likely have the most bugs? Or what sections of code are the most complex? Maybe we should give it a shot…
10. Measure your Test Maturity – According to CIO Magazine, “The Capability Maturity Model Integration (CMMI) is a process and behavioral model that helps organizations streamline process improvement and encourage productive, efficient behaviors that decrease risks in software, product and service development.” Initially developed by the Software Engineering Institute at Carnegie Mellon University, in conjunction with the DoD and U.S. Government, CMMI is currently administered by the CMMI Institute as a process improvement tool for projects, divisions or organizations. CMM defines 5 levels of maturity – (1) Initial, (2) Repeatable, (3) Defined, (4) Managed, and (5) Optimizing. If thought of in a different way, this maturity model could be specifically applied to software quality and testing, and used to measure current state and set goals for improvement.
11. Think Different – While this may be the slogan for a famous ad campaign for a large tech company named after Newton’s fruit, it is also a technique to help people solve problems by taking a different mental approach – Creative, Analytical, Critical. Concrete, Abstract, Divergent, Convergent, Sequential, and Holistic thinking.  Read more about software testing and different thinking types on testingexcellence.com.
12. Complexity = Errors – Exploit the correlation between complexity and errors. A rule of thumb is that one component should do one thing, and do it well. Make complex classes smaller. Break monolithic services into micro services. Break database tables into multiple, smaller tables. The same is also true for tests. Make unit tests smaller. Test smaller sections of the user journey. And aim your spotlight of testing onto complex areas of your application – UI, services, or data, integration points, service signatures, common failure points.
13. Wear different Thinking Caps – It’s a common enough expression… “put on your thinking cap.” The 1985 book Six Thinking Hats by Edward de Bono promotes multi-dimensional thinking by mapping different colored hats to different cognitive styles – blue for an overview, white for facts, red for emotion, yellow for optimism, black for risks, green for creativity. Put on different thinking caps, and innovate in whole new ways.
14. World Quality Report – “The World Quality Report is the only global report analyzing software testing and quality engineering trends. It presents an analysis of developments in agile and DevOps, artificial intelligence, automation, test environments, data, security and budgets, showing once again the importance of quality, and of the measures that are put in place to maintain it.” – sogeti (part of capgemini)
15. Online Resources – Things change, evolve, improve. It is our obligation as technologists to stay on top of those changes. There are a lot of resources that can help you do that. Training, conferences, certifications, and industry trends are some ways to stay on top of those changes. Sometimes it is just staying on tho of the news as it happens. While doing research for my blog posts, some of the best resources for me were Ministry of Testing, Software Testing Help, StickyMinds, InfoWorld, and Software Testing Magazine.
16. Training – Continuing your education as a technologist and quality engineer is critical. There are lots of online video training resources available online. Pick your favorite and keep learning. Two of my favorite are Udemy (and their class on Software Testing and Innovation): and Lynda (and their class on Exploratory Testing).
17. Conferences – Learn about new techniques, hear from industry experts, plan for future innovation, look at products from vendors, continue your education. Check out a list of conferences from TechBeacon and testingconferences.org .
18. Certifications – Certifications are more than just a test and a score. They are managed by a company with resources, continuing education, advanced certifications, and a community of professionals you can tap into. Check out a list of certifications for Software Testing professionals.
19. Industry Trends – Remain on top of industry trends allows you to adapt to current challenges, leverage new tools and techniques, and benefit your skills and your company. Read more about quality assurance trends in 2019.
20. Beyond The Bugs – What I have focused on in these last four posts is how to find bugs in your code. Ones that would functionally prevent users from using your site. But there are lots of non-functional ways to test your site. Security testing, penetration testing, performance testing, reliability testing, efficiency testing, maintainability testing, portability testing.. and plenty more I am sure. Keep on expanding your knowledge, wide and deep.

In the list above, I just scratch the surface on each one of these ideas. Spend time researching each of them, dig deeper into the ideas, or come up with your own. Have an idea, technique, or best practice I haven’t covered? Leave me some comments and let me know.