Category Archives: Management

Brian T. Whaley – A README File

============================================================

Who I Am

I am currently the Director of Engineering responsible for the Global Prospects Application Experience Team.

Prior to American Express, I led global digital web development departments for healthcare, financial, telecommunications, and media companies.  I bring with me expertise in UI Development, Software Project Management, and Agile Methodologies.  Some of my most recent projects outside AmEx include launching custom developed marketing, E-Commerce, and Content Management systems, focusing on SEO, Analytics, and User Experience.

I hold an MSIS degree from the Stevens Institute of Technology, a Bachelor’s degree in Art and Art History from Rutgers University, am a Certified Project Management Professional, and am a Certified Scrum Master.  In addition, I serve on the Advisory Board of the Design Thinking Certificate program at Rutgers University.

I live in Morristown, NJ with my wife of 18 years, Mary Ann. My hobbies include photography, scuba diving and reading Sci-Fi and Fantasy novels.

============================================================

Why A README Document

The objective of my REAME is to describe who I am, set some expectations, and management  styles.  What I don’t want this document to do is dictate how to work with me.  I believe that good  leaders adjust to the personality styles of the people reporting to them.

============================================================

My Job

Talent Management

My primary function is to attract, hire, retain, and grow the best talent I can for our team.  We need  a heterogeneous mix of backgrounds, interests, and experience.  We need to work within a safe,  transparent, and productive culture.  Without the right people on the team and culture to work  within, we cannot deliver our products effectively.

Technology Management

I am also responsible to develop a technology roadmap, lay out the right architecture, advancements, and improvements  to deliver our product with little friction, the best user experience, and fastest time to market.

============================================================

What You can Expect from Me

  • Focus on Servant Leadership 
  • Develop leadership at all levels in all roles, from the bottom up
  • People over Process, as the Agile Manifesto recommends
  • Give and receive timely and regular feedback – recognition in public, criticism in private
  • I will book 1:1s only when I have something I need to share, or for performance reviews.
  • Deliver continuous improvement in everything we do – code, processes, policies, meetings, etc.
  • Provide transparency and candor wherever and whenever possible
  • A bias toward action
  • Due to my long train commute, I plan to be in the office at 9am, leave at 5pm
  • Work from the office whenever possible, but when working from home, be available on all  channels (phone, text, email, slack, WebEx, etc.)
  • I am a night owl, so I may be working later than you.  Don’t feel obligated to respond during  non-business hours, unless it is related to a production issue.

============================================================

What I Expect from You

  • Book your 1:1s if you want them.  Regularly scheduled or ad-hoc, the schedule and agenda is up to you.  They are your time with me.
  • Provide feedback regularly – to your staff, your peers, and your boss.  That’s me!
  • Display honesty and transparency in all that you do.
  • Focus on People First, Product Second, and Everything Else Third.
  • Do your best work all the time.
  • Try something new.  Educate yourself.
  • Don’t be afraid to fail.  Trying and failing should be celebrated.
  • Don’t be afraid to ask for help.  But don’t bring problems without a proposed solution.
  • Show you care.  Emotions included.
  • Exhibit leadership from your position.  You don’t need to be a manager to be a leader.
  • Display an extremely high sense of ownership, with a bias for action.
  • Continually point out things we can improve, in everything we do.

============================================================

Personality Test Results

People learn a lot about themselves and the people around them when they take personality tests.  It is an excellent exercise in introspection and self understanding.  To that end, here are my results from some popular personality tests.

Strengths Finder Results

  1. Achiever (EXECUTING)
  2. Ideation (STRATEGIC THINKING)
  3. Positivity (RELATIONSHIP BUILDING)
  4. Woo (INFLUENCING)
  5. Learner (STRATEGIC THINKING)

MBTI Results

============================================================

Books

I am an avid reader.  Unfortunately, I do read books in waves.  I primarily read Science Fiction, Fantasy, and Management books.  The book I recommend most often to people at work is Turn The Ship Around by L. David Marquet.  It is about leading from the bottom up, fostering individual contributors as leaders with high levels of ownership, and scaling these ideas across the organization.  Just the kind of culture I try to foster in my organization.  Here is my Goodreads book library for more information.

============================================================

Favorite Quote

“It doesn’t make sense to hire smart people and tell them what to do; we hire smart people so they can tell us what to do.” – Steve Jobs – https://www.inc.com/marcel-schwantes/this-classic-quote-from-steve-jobs-about-hiring-employees-describes-what-great-leadership-looks-like.html

Managing the boxes, and the space in between

As a leader, it is incumbent upon me to manage my team in every way. The word team has lots of different meanings for people. Some definitions require a team to be a temporary, project based group. Others do not make the distinction. I do not think it matters. In my mind, a team is a group of people who work together regularly for a common goal, and is not limited by time, space, or organizational boundary.

There are the five phases of development that teams go through, whether it is a short term assignment or a long term functional group. Those phases are Forming, Storming, Norming, Performing, and Adjourning.  Any change to that team “resets” the status quo, and sends the team through those phases again to some degree. This could be due to staff changes, a reorganization, attrition, major shifts in work assignments, or a host of other reasons. It is my job to guide my team through those phases of team development, and emerge on the other side as a better, stronger team.

Staffing changes are one of the most devastating changes to the team balance. Losing someone the team depends on in any way upsets the balance, changing the work and the way it is done for each individual member. Adding new players to the mix changes the political landscape, and a grab for power is only natural. A newly forming department is a chance to move up the organizational ladder. The team will be storming until a steady state is achieved.

Shifts in leadership give the team an entirely new tone, and this is driven not only by those at the top, but the relationships he or she fosters around him. How does the leader treat their team members? How well do they get along with partners outside the team? Does the leader delegate all phases of work, or are they in the trenches with their team, achieving their goals together? All this affects what people think about when commuting to the workplace.

Changes to the type and volume of work also disrupt team balance. A large project is a chance to prove yourself. New technologies are an opportunity to distinguish yourself apart from other developers. When too much work is assigned, team members struggle to balance out the overload evenly to minimize their personal stress levels.

My job is to manage each person on the team through these inflection points. Are there things that interest them that they are assigned to do? Are they challenged in their position enough, but not too much? Is there a planned career path? Are there defined ways for them to show success, and allowed to fail (with the ability to recover) for personal improvement? Do they enjoy coming into work every day?

Sometimes it is not managing the work in front of someone that is important, but the things around them. Are they finding ways to improve performance of themselves and those around them? How well does the team work with other teams? Are they disconnected, does the output of one team become the input of another, or is the process shared? Are meetings limited to the boundaries of your team? Are the successes shared across teams? Are the areas for improvement owned across teams as well? Are we measuring our work so that we can show success, improvement, and areas to work on next? Do you define your team as people only in your department, or has the definition of your “team” been broadened to include other teams you work with regularly?

These are the things that occupy my mind lately. Projects will constantly come and go. It is the team, it’s members, and how they work together that will make the work and it’s results better.

Reflections from the Other Side of the Interview Table

Over the last 10 years, I have been involved with many, many interviews.  I have blogged about good interview booksresources for interviewinghow to prepare for an interview as the interviewer, and how to conduct an interview.  Being on the other side of the interview table is a bit different.  It definitely added some serious insight into how I conduct my interviews, and reinforced a lot of the process I have put in place.

There are different things to do before the interview,  to prepare for the interview itself, and to follow up with after the interview.  This is my brief step-by-step guide to navigate the interview process as an interviewee.  It seemed to be successful, as I have moved my cheese recently to the Associated Press!

Should you stay or should you go?

This is a big decision.  In this rough economy, having a job at all is a blessing.  But decide to leave a stable job for something else is risky.  But, with great risk comes great reward.  This move has been great for me mentally, personally, and professionally.  Here are a few sites that I used to help me decide to take the leap.

Preparing your resume

Your resume is your potential new boss’s first glimpse at who you are.  If you decide to make the leap and look for a new job, be sure that you spend the time to present yourself as best you can.  This is where the bulk of candidates will be cut from the running.  Make sure you stay on the short list with these resources.

Once your resume is just the way you want it, be sure to update your profiles on FacebookLinkedInDiceMonster, your Google Profile, or any of the other places where your work history might be stored online. Your new employer will check all of these, and consistency in you message and timeline is very important.

Where to look for a new tech job

Once your resume is all tidy, and your profiles are updated, now it’s time to start looking for that perfect new job.  Here are a couple of articles that will help you find that perfect new home.

One of my favorite places I liked to search for new jobs was indeed.com.  You could search across DiceMonsterCareerBuilder, and lots of Fortune 500 corporate career sites.  Personally, I created a search on each of these sites, and added the RSS feed from the search to Google Reader, and checked it each day.  This made job searching simpler, and centralized it for me all in one place.

Preparing for the interview

When preparing for the interview, you should anticipate the questions you are going to be asked.  you should expect technical, managerial, project management, style, and soft skills questions.  Here are a few books that I recommend to prepare for your interview questions:

Follow up afterwards

After the interview, you need to thank your interviewers and let them know that you are interested in the job.  Be sure to follow up with them after the interview.  Thank them for their time, build on some points or strengths discussed in the interview, and express your enthusiasm in the position.  There are lots of good sample thank you letters out there.  Be sure to customize it to your interviewer, the interview, and to you.

Research the salary band for the position

You are going to be talking about salary at some point with your potential new boss or HR department.  You need to be prepared.  Be sure to research the salary band for your title, position, region, and level of responsibility.  Salary.com is a great place to do this.

How to navigate the job offer

Job offers can be complex, confusing, and a very touchy situation.  This is very far along in the process, and you now know whether you want the job or not. Negotiations.com offers some good advice in negotiating your offer package.

Wrap Up

Overall, the objective of an interview is to get to know your new possible employer, and let them get to know you.  If you are a match for them, and they are a match for you, things will work out fine.  If not, then don’t be disheartened – you and your interviewing company were not a match for each other, and you are better off finding a job that will make everyone happy.  You move on to the next interview.  My piece of advice to find that perfect match is to be yourself, be honest, and be prepared.

Software Metrics

Being able to measure success for a software development group is a difficult thing. But not being able to show the success of your development group is a dangerous thing. Your management team will want to be able to measure quality, show improvement, and benchmark against industry standards. All the popular maturity models (if you put stock into them) emphasize the ability to measure, react, and improve your quality and processes. My department is no different. We try to remain as flexible and lightweight as possible, and add formality where improvement is necessary. Here are some of the metrics that we collect to measure our success and find ares of improvement.

Code Coverage

There will always be bugs. No matter how hard you try, there will always be bugs in your software. That does not mean that you cannot try to prevent them. One of the easiest ways to do that is to write automated unit tests to validate your code is doing what is expected. You can write your unit tests lots of different ways, and Steve Cornett’s Code Coverage Analysis paper gives lots of different ways to break down code coverage. A great place to start is to aim for 50% coverage of all of your lines of code. And, as Tony Obermeit says in his Code Coverage Lessons paper, your ultimate goal of course is to always hit 100% coverage. You will need to pick a code coverage tool to help measure your success. In my department, developing in a Visual Studio and C# environment lends itself to the nCover open source solution. This solution works well with our CruiseControl environment. Test Driven Development methodologies and mocking tools can help you get closer and closer to covering as much of your code as possible with automated tests.

Defect Tracking

I use the words defects and bugs interchangeably. This I am sure is something that some people would disagree with, but I think that it is close enough. If it is not working as expected, then it is a defect, and it is a bug. Regardless, defects are identified in the development process, system and user acceptance testing process, and in the production environment. The objective is to minimize the number of bugs that are found in the live environments. To do that, you need to encourage the identification and mitigation of bugs in earlier and earlier stages. This sounds fundamental, but becomes difficult to implement. There are lots of methods that you can do to identify, solve, remove, and prevent bugs. You must have a way to measure that these methods are improving your success rate. And that means measuring the number of defects found in each environment – development, system test, user acceptance test, and production. The easiest set of numbers to get in a corporate environment is production defects. There is always a problem management system or help desk that tracks these kinds of things. But as a software development organization, you need to implement bug tracking throughout the entire lifecycle of your software. You can trend the numbers, and make sure you are finding more bugs before UAT, particularly in development. Tools like Bugzilla, an open source bug tracking tool, can help you track, trend, and manage defects in your software throughout its lifecycle.

Support Ticket Management

Software is not a static entity. It is always changing. Just think of all the patches, updates, service packs, and bug fixes that Microsoft releases on its suite of software. In a corporate environment, it is no different. Software management does not end once it is released. Teams of developers will be constantly updating desktop, web based, and console applications based on new requirements and requests from their clients. Problem Management software can be used to help track and trend all of these requests bydata points such as severity (Urgent, High, Medium, Low), priority (Urgent, High, Medium, Low), assessment (Customer Assistance, Required Modification, Elective Enhancement, Break Fix), difficulty (High, Medium, Low), etc. You can measure success agains more complex metrics such as the number of tickets created, number of open tickets, time to resolution, etc. All of these metrics will help you determine how fluid, stable, usable, sustainable, and maintainable your software is. Do not ignore your software or its users once it is released to production.

Analytics

Web Analytics tools can tell you how many users you have had on your site, how long they visited, where they came from, where they went, how they found your site, did they reach your goal pages, did they convert, and did they return. There are free web based tools like Google Analytics, and over the shelf packages like WebTrends and CoreMetrics that can help you measure site activity. Do not ignore these metrics to help you define current activity, make improvements to your site, track your new results, and continue to improve. They directly measure your clients’ interaction with yoru software, and can identify trends that with simple changes can vastly improve your software and development processes.

Conclusion

So… these are some of the ways that we track the success of our software. There are a host of other methods to measure software, such as function points, lines of code, complexity, interfaces, velocity, etc. What ways do you measure your software? how do you define success? What are your plans for the future?

Leave me a comment and let me know what you think.

Book Review – How Would You Move Mount Fuji?

Over the holiday break I decided to tackle some of the books that I have stacking up next to my bedside.  One of them was How Would You Move Mount Fuji by William Poundstone.  This was a book that I know some of my colleagues had read already, and they recommended it highly, so I decided it was my turn. 

The book was a quick read.  The author kept my interest with not only the topic, but also with his concise explanations and his witty comments. 

Poundstone describes the history of the intelligence tests, and how it was developed.  They were used by our military to determine qualification for different job roles.  This led to the popular use of intelligence tests in the corporate world, particularly in the use of Silicon Valley.  During the civil rights movement, intelligence tests were determined to have a racial bias in the questions, so were banned as a hiring practice by the federal government.

The ban of intelligence tests did not deter those types of questions from remaining in interviews, however.  Looking for more people with minds like Bill Gates, puzzles made their way into the interviews at Microsoft.  They have popularized the use of logic puzzles and impossible questions.  Poundstone also describes the grueling day-long series of interviews at Microsoft and how you are rated throughout the process. 

My most important takeaways from the book was these nuggets of golden advice –

  1. When the technology you use is changing rapidly, you must hire for problem-solving stills, not just for the technology.
  2. A bad hiring decision is likely to hurt the company more than a  good hiring decision will help it.
  3. If you ask puzzle questions in your interview, make sure they are worth the effort by asking yourself these two questions:
    1. Are you willing to hire someone because of a good answer to this question?
    2. Are you willing to reject someone because of a bad answer?

I highly recommend this book to any hiring manager who plans on asking any puzzle type questions.  I also think the book adds insight into the overall interview process, even if you don’t plan on asking them that type of question.

Have you read the book?  Do you have an opinion on puzzle questions in interviews?  Leave me your feedback and let me know what you think.

The Lost Art of Debugging – Part 3 – Things To Do

As I have said before, debugging is a complex and time consuming process. I have outlined 10 resources for debugging, and provided a primer for things not to do when debugging. Now, we get to the meat and potatoes of debugging. This is a guide of things to do when debugging. I have broken this guide into three sections – a description of the Scientific Method of Debugging, Tips for Hunting for Bugs, and Bug Prevention Methods.

Scientific Method of Debugging

This is a parallel to the Scientific Method that you learned in your grade school science class. The book Code Complete by Steve McDonnell outlines this methodology to debug your applications.

  • Stabilize the bug by identifying the simplest test case that reproduces the error
  • Locate the source of the bug
    • Gather data about the bug and its behavior
    • Analyze the data regarding the bug
    • Form an hypothesis about the bug
    • Write test cases that would prove or disprove your hypothesis
    • Run the tests and prove your hypothesis, or begin again with a new hypothesis
  • Fix the defect
  • Test the fix with all of the new unit tests you have written
  • Continue to look for similar, cascading, or peripheral errors

For a more detailed breakdown of the Scientific Method of Debugging, read Code Complete by Steve McDonnell.

Bug Hunting Tips

These tips for finding bugs, broken out into different areas, will help you narrow down where your bug is.

General Tips

  • Be sure to save the original source code
  • Be sure the fix the problem, not the symptom
  • Make one change at a time
  • Check and Double Check your fix
  • Consider the possibility that the bug is generated from multiple sources
  • Divide and Conquer
  • Check other places where bugs have existed
  • Check other places where code has changed recently
  • Compile the application again

Logging

  • Insert Trace, Print, or Alert Statements to help track the bug
  • Create a logging methodology to trace the application
  • Check the Log Files of the servers, etc.
  • Search the web for the stack trace

Unit Testing

  • Design By Contract
  • Write unit tests that cause the error
  • Try to reproduce the bug in different ways
  • Try boundary cases and special cases

More Complex Methods

  • Recompile with a different compiler
  • Create a mini version of the application
  • Sequentialize the Parallel to see if it is a timing or resource issue
  • Try the code in different environments (local, dev, test, prod, other developers’ machines, etc.)
  • Grab someone else to talk through the bug without looking at the code (explaining the problem may trigger some ideas)
  • Do a full, in-depth code review on the broken code

Last Resort

  • Rewrite the whole section of code that is causing the bug
  • Take a break for a few minutes, or an hour, or until the next day, to give your mind time to process the data

Bug Prevention Methods

These are things that you should be doing during the planning and development of your applications that will help you identify, fix, and test your bugs after you are done with development.

  • Identify and track consistent bug types within your own code
  • Introduce debugging methodologies early
  • Implement loose coupling techniques
  • Implement Information Hiding Techniques
  • Write a regression test that tests the bug you just fixed

So… these are some different ways to attack debugging. Are there methods that have proven themselves that you have used? Do you take a different, more unique approach to debugging? What have your experiences shown to be your best practices? Please leave your feedback and let me know what you think.

The Lost Art of Debugging – Part 2 – Things Not To Do

Debugging is a complex and time consuming process.  In my last post I listed 10 Resources for Debugging, both web sites and books, that every software developer should read to keep their debugging skills sharp.  Knowing what not to do is just as important as knowing what you should do.  Here is a list of things not to do when you are debugging your application

  • Don’t guess what the problem is
  • Don’t dive into making changes without a plan
  • Don’t code right away without a thorough analysis
  • Don’t fix the shell of the problem, not the problem itself
  • Don’t trust line numbers in compiler messages
  • In fact, don’t trust the compiler messages at all
  • Don’t ignore the value of automated unit testing
  • Don’t delete the original code
  • Don’t ignore recent changes to your application
  • Don’t ignore resources available to you, like the Internet or the Library

This post is a teaser for the one that is to come – what thins should I do when debugging?  Is there a more formal process for debugging?  What steps should I follow?  What things should I look for?  What place should I look?

5 Benefits of Internal Corporate Blogs

Blogs are not new anymore.  They have been around for a few years, and have proliferated fairly deep into the technology culture.  If people are not writing one of their own, they are reading one, or a handful of them, or have an RSS aggregator where they are reading dozens or hundreds of blogs. 

Blogs have lots of uses.  They are great for viral advertising, news publication, syndication, and collection of public opinion about a topic.  You can have a personal blog to share photos of your vacations, a corporate blog for press releases, a news blog for niche news, and a host of other reasons to have a blog.  Blogs are an easy way for anyone to produce and consume any information about anything from anywhere. 

But blogs have taken off within corporations as well.  That is my interest here.  We have deployed a blogging pilot in the workplace, and I have become a big proponent.  At first it was part of my yearly objectives to help pilot and proliferate the use of blogs.  I have blocked time on my calendar twice a week for a half hour to post to my blog.  Now I am one of the most frequent bloggers in our pilot, and I blog both inside and outside the company.  I do this because I see such a value in the process.  Blogging is such an important tool in my manager arsenal, and I thought I would list the benefits of blogging from my experiences for the team and for the company.

Knowledge Sharing

Blogging inherently is a way to share information with its readers.    The writer composes the message, and it is consumed by many readers in the blogosphere.  These messages are (obviously) either bottom-up or top-down.  Bottom-up messages are written from the perspective of the line workers and low to mid level managers to express ideas or experiences they have had.  Top-down blogs are a way to deliver executive messages, such as strategic direction, corporate status, and announcements.  Blogs act as a written memory of events, lessons learned, experiences, successes, or announcements.  Blogs can also be a way to share expertise on a specific topic, such as data management, service oriented architecture, design patterns, or search engine optimization. 

Idea Solidification

The writer has to take the time to think about the message they want to deliver, collect their thoughts, organize them, and express them clearly.  That way the readers can read a focused message that delivers a clear and understandable message.  Blogs can also be a way to develop, solidify, and evolve an idea by posing a question, idea, or concept, and collect and narrow an idea through constructive comments. 

Project Management

Using a blog to document a project and its progress is a bit unorthodox.  Typically a wiki or another more collaborative environment is more appropriate.  But blogs are so easy to use, they make for a good medium to capture progress and communicate them easily.  Again, they act as a written memory of the project’s progress, successes, and lessons learned.

Cross Team Communication

Blogs are a great way to provide a medium for dynamic networks of conversations across teams.  These teams could be local or distributed, days, nights or weekends, and it will not matter.  A shared blog space will allow for ideas to be spread, information to be distributed, questions to be asked and answers to be shared.  Blogs crisscross people, departments, silos, grade levels, and experiences.

TeamBuilding

The simplest thing that blogs do is to bridge the boundaries of distance and culture by being always on, able to be shared and viewed at any time.  When launching a blog or posting a comment, a person takes themselves in written form and puts a bit of themselves out for everyone to look at, examine, scrutinize, comment on, and critique.  Successful corporate blogging creates an atmosphere of trust, so that people are not threatened, intimidated, or frightened to expose themselves.  Successfully sharing and collaborating this way fosters a spirit of participation that encourages people to continue to contribute their thoughts and ideas.  This leads to team learning and growth across the team, from within the team, and makes them stronger together. 

 

So what is your perspective on blogging within corporations, or blogging in general?  Is there value?  Is it a waste of time? Leave me feedback and let me know your thoughts.

 

Resources

7 Sources to Help You Conduct a Peer Code Review

The end of the year is swiftly approaching, and as the budget cycle is coming to a close, the work piles up quickly.  I have had a staff of three from January through August, and with the piling work, we have had to triple in size.  A staff of nine is very different to manage compared to a staff of three.  There are many things to wrangle during ramp-up: a new corporate culture, new policies, new development environment, new standards, new team members.  One of the tools in the technology manager’s arsenal is the peer review. 

The CodingHorror web site outlines why they think software development departments should perform code reviews in their article called “Code Reviews: Just Do It.”  Jared Richardson also describes the value of peer code reviews on his blog.  The basic reasons for implementing code reviews are obvious: reduced bugs, reduced rework, cross training, quicker ramp-up time, sharing of ideas, developer growth, mentorship opportunities, and team building.

Karl E. Wiegers has written a great article called “Seven Truths About Peer Reviews” available on processimpact.com that outlines a number of different methods to conduct your peer code review.  Some of the peer review methods methods he describes are inspection (both Fagan Method and Gilb/Graham Method), team reviews, walkthroughs, pair programming, peer deskchecks, and passarounds.  It’s a good read, and you should take a look.

Peer reviews do not need to be long, drawn out, complicated processes.  Jason Cohen describes some best practices for peer code review on his web site, SmartBear.com.  He has written a great looking book called “Best Kept Secrets of Peer Code Review” that you can order free on his site.  He recommends only reviewing code for five to ten minutes at a time, to keep everyone sharp, focused, and on point. 

You should definitely walk into your peer code review with a plan in mind.  Here is a Code Review Checklist for Macadamian that will help you organize your approach and keep you focused.

Finally, peer code reviews can be rough on the ego of the developer.  Here are “Ten Commandments of Ego-less Programming” based on Jerry Weinberg’s book , The Psychology of Computer Programming, from 1971.  This article describes how peer code reviews result in shorter project completion times, lower defect counts, and how developers had to set their egos aside to make it happen.

I am sure that code reviews will be good for my team, our projects, and our clients.  The hurdle is making them part of our development routine.  What do you think of code reviews?  Do you use them in your technology department?  What results have you seen?  Leave me feedback and let me know.

3 Indispensable Tools for Candidate Review

Tools are a great way to shorten time to complete tasks and improve quality in a process.  Hiring, Candidate Review, and Performance Review are no different.  Here are three tools that I have founds myself using while reviewing candidates.

SkillSurvey

So I got an automated email last week.  Big surprise, eh?  If you are like me, you get dozens of these a day.  This one was a little bit different.  It was from a company asking me to provide a reference for one of my old consultants.  That piqued my interest.  I opened it up, followed the link to SkillSurvey, and filled out the questions.  The simple instructions walked me through the process, and reassured me that my comments were anonymous and would be aggregated.  There were less than 10 questions, and were simple radio button scales.  I had the ability to put in freeform comments, and send it off. 

360Metrics

 Every other year, I have been encouraged by my management to do a 360 degree review of myself.  I log into 360Metrics and choose up to 3 direct reports, clients, peers, and managers.  Each of them receive emails to complete a set of predefined questions that rate and rank my skills on on a set of core values.  They are periodically reminded over the course of the review cycle.  When the review period closes, I receive an email and can generate an aggregate report of my ratings and comments. 

BrainBench

Every time I open up a requisition to find a new candidate for our team, I get a flood of resumes.  Eighty percent of those resumes have some sort of certification.  Most certifications we see are from Microsoft.  But some certifications come from BrainBench.  They offer certifications for individuals on a wide variety of specializations from Computer Software to Management, from Aptitude to Office Skills, from Communication to Industry Knowledge.  Obviously, the ones we see most often from BrainBench are technical in nature.  BrainBench also provides Pre-Hire Testing and Employee Development services to Employers.

Benefits

  • These tools are simple to use
  • The questions can be asked and are easy to analyze quantitatively
  • The same questions are asked of all candidates, so the questions and the delivery are consistent
  • They have open ended questions that allow you to analyze free-form text
  • Since these tools are web based, they can be leveraged (both from the candidate and from the hiring manager) 24 hours a day, 7 days a week.

Drawbacks

  • The tools may not be measuring what you are looking for
  • It is not difficult to “game” the system to produce phony or inaccurate results
  • The perception of these tools is that they measure which candidate is “better” than another
  • Some of these tools may give an advantage to good test-takers
  • The process may be too objective, and not interactive enough

Conclusion

I think that these tools can be a big help, and a great time saver.  Tools like SkillSurvey leave the evaluation up to the hiring manager, and help facilitate and focus the collection of critical hiring information.  However, the risk of relying on tools like BrainBench is that you take the interpersonal aspect out of the process.  You may miss an enthusiastic, bright candidate who is a bad test-taker, or who has the right attitude, but a different set of experiences. 

I am sure that there are lots of other tools available in this arena.  What tools have you used?  What have your experience been with them?  How do you use them in your process?