Category Archives: Leadership

Outsourcing 102 – The New Team Members

The Plan

The initial plan was to train the liaison within my department, as there was an immediate need for assistance. After, he would be cycle through the teams of my peers, to have him learn about each of their departments. The liaison was to learn about the environment, gather documentation, and set up an environment offshore for development. As demand increased, we would slowly grow a team offshore. My focus was to start them off slow, as content managers, support staff, and graphic designers. As experience grew, we would spread the offshore team into other, more complex areas of development, such as new development and project management.

Step One – Training and Environment

My team has always been ahead of the game in terms of documentation. We use a shared platform across all of our sites, so the structure of our projects are all similar. Each of the services and the methods are documented with code samples on their usage. We use a project template to jump-start project development, with documentation on the functionality of each of the DLLs, aspx pages, and ashx handlers. We also have templates for project estimation, requirements documentation, and system testing. We have a checklist to guide the developer through the project process. All of our code is stored in VSS, all of our DDL and DML SQL is stored with the solution, and all the documentation is stored in our Documentum instance. Promotes are handled throught CruiseControl for .Net code, and SQL Navigator for the SQL code. All pur projects are initiated through a centralized PMO, and our Client Facing team are the overall project managers. Providing all of this information about our code and our process was the easiest part of the transition.

Getting the offshore team a working environment was not as easy. All of our servers are in the United States, behind our firewall. Working with our security team, VPN access was ruled out as too risly to provide without a more diligent legal contract. This left few options. The one that was chosen was a web based Citrix solution. Tools such as Visual Studio, VSS, and SQL Navigator, were provided to the development, test, and production environment through the Citrix Business Partner site. Some other tools were provided directly to the offshore team, such as local copies of Visual Studio and a local version of our database.

Step Two – Content

Some of the first tasks assigned to the offshore were to help manage content for new projects. I figured that this was a simple set of tasks that would expose the outsourced developers to our development environment. An offshore developer was added to a new project, paired up with an existing onshore developer, and work was divided. Each new page has to be registered in the database, and assigned to the proper application, with all the right properties and URL rewriting information. Our page content comes from advertising agencies in the form of HTML. We take that HTML, slice it into reusable blocks and unique blocks, and insert them into our custom content management database. Each of the pages then has to be associated to each of the content blocks. All of this data is inserted into the database via Data Manipulation (DML) SQL scripts. Once inserted, the pages need to be tested with the solution. These activities were the responsibility of the offshore team members.

Step Three – Support

A significant part of our development activity is support. Each project allocates a 5 year budget to support that application, and we need to manage those activities just as well as projects, if not better. In fact, support tickets are like mini projects. They contain an analysis, design, implement, test, deploy, and stabilize phase just like projects do. The only real difference is that upport activities last less than 15 days. This is another great way to get new staff familiar with a new environment. We set aside two people offshore to be responsible for support. The tickets were divided across the two people. And they had all the same resources as the project based developers. These two very quickly got a cross section of all projects and sites, tools and technologies, and processes and procedures.

We had also implemented a role onshore that would be responsible for coordinating all support, both across technologies and across the globe. This person was responsible for support assignment, workload, quality, and process, all as it related to support. This was received well. Support functions now had a champion; a clear voice representing their perspective.

Next

In my next post, I will cover the results of our new outsourcing contract, and how some other teams were impacted by the outsourcing contract. I will also cover and how it became the most important decision of the year for the team.

Outsourcing 101 – Introduction

The World is Flat

Outsourcing and offshoring is a mainstream business practice in today’s economy. Companies reach to outsourcing and offshoring to find cost savings, find expertise outside of their core business, and provide a follow-the-sun workforce. Blended costs for outsourced companies is lower than a purely domestic team by leveraging lower resource costs in other countries like India, Brazil, and the Philippines. Things are no exception where I work.

The Story

I have decided to document the transformation of my department into a global organization that embraces outsourcing and offshoring. I am also hoping that those that went through this process with me who read my blog will provide comments of their own, and keep me honest.

In The Beginning…

In the middle of 2007, my department decided to globalize our work force and find an outsourcing vendor. Other departments had experience with Satyam, Intellogroup, and Accenture. We decided to start with Satyam, as we heard the most positive reviews of their performance.

We put together a brief meeting and walked through our objectives of blending a global team to drive down costs. Over the course of the next 4 months, we focused our interviewing skills at a half dozen candidates for our first outsourced team member. The first candidate seemed to be a good fit for our team, and so we made short order in making an offer and getting a contract signed. But, in the 24 hours between interview and offer, the candidate mysteriously became unavailable. Interview followed interview, and all the candidates seemed to fall short of our expectations. It reached a point where the resumes stopped coming in. With only two months left in the year, we resolved ourselves to try another outsourcing company.

After reaching out to Intelligroup, we held a brief meeting with them. The meeting turned into an ad-hoc interview for one of the attendees. He seemed to be strong in .Net technologies, have a solid background in software architecture, knew enough about web technologies, and had project management experience. We made an offer, signed a contract, and on December 3, 2007, we had officially taken the plunge into offshore outsourcing and hired our onshore liaison.

Next…

My next post will talk about how we expanded the team to include our first offshore resources located in India, how we integrated them into the team, and some of the bumps and bruises we experienced along the way.

Software Metrics

Being able to measure success for a software development group is a difficult thing. But not being able to show the success of your development group is a dangerous thing. Your management team will want to be able to measure quality, show improvement, and benchmark against industry standards. All the popular maturity models (if you put stock into them) emphasize the ability to measure, react, and improve your quality and processes. My department is no different. We try to remain as flexible and lightweight as possible, and add formality where improvement is necessary. Here are some of the metrics that we collect to measure our success and find ares of improvement.

Code Coverage

There will always be bugs. No matter how hard you try, there will always be bugs in your software. That does not mean that you cannot try to prevent them. One of the easiest ways to do that is to write automated unit tests to validate your code is doing what is expected. You can write your unit tests lots of different ways, and Steve Cornett’s Code Coverage Analysis paper gives lots of different ways to break down code coverage. A great place to start is to aim for 50% coverage of all of your lines of code. And, as Tony Obermeit says in his Code Coverage Lessons paper, your ultimate goal of course is to always hit 100% coverage. You will need to pick a code coverage tool to help measure your success. In my department, developing in a Visual Studio and C# environment lends itself to the nCover open source solution. This solution works well with our CruiseControl environment. Test Driven Development methodologies and mocking tools can help you get closer and closer to covering as much of your code as possible with automated tests.

Defect Tracking

I use the words defects and bugs interchangeably. This I am sure is something that some people would disagree with, but I think that it is close enough. If it is not working as expected, then it is a defect, and it is a bug. Regardless, defects are identified in the development process, system and user acceptance testing process, and in the production environment. The objective is to minimize the number of bugs that are found in the live environments. To do that, you need to encourage the identification and mitigation of bugs in earlier and earlier stages. This sounds fundamental, but becomes difficult to implement. There are lots of methods that you can do to identify, solve, remove, and prevent bugs. You must have a way to measure that these methods are improving your success rate. And that means measuring the number of defects found in each environment – development, system test, user acceptance test, and production. The easiest set of numbers to get in a corporate environment is production defects. There is always a problem management system or help desk that tracks these kinds of things. But as a software development organization, you need to implement bug tracking throughout the entire lifecycle of your software. You can trend the numbers, and make sure you are finding more bugs before UAT, particularly in development. Tools like Bugzilla, an open source bug tracking tool, can help you track, trend, and manage defects in your software throughout its lifecycle.

Support Ticket Management

Software is not a static entity. It is always changing. Just think of all the patches, updates, service packs, and bug fixes that Microsoft releases on its suite of software. In a corporate environment, it is no different. Software management does not end once it is released. Teams of developers will be constantly updating desktop, web based, and console applications based on new requirements and requests from their clients. Problem Management software can be used to help track and trend all of these requests bydata points such as severity (Urgent, High, Medium, Low), priority (Urgent, High, Medium, Low), assessment (Customer Assistance, Required Modification, Elective Enhancement, Break Fix), difficulty (High, Medium, Low), etc. You can measure success agains more complex metrics such as the number of tickets created, number of open tickets, time to resolution, etc. All of these metrics will help you determine how fluid, stable, usable, sustainable, and maintainable your software is. Do not ignore your software or its users once it is released to production.

Analytics

Web Analytics tools can tell you how many users you have had on your site, how long they visited, where they came from, where they went, how they found your site, did they reach your goal pages, did they convert, and did they return. There are free web based tools like Google Analytics, and over the shelf packages like WebTrends and CoreMetrics that can help you measure site activity. Do not ignore these metrics to help you define current activity, make improvements to your site, track your new results, and continue to improve. They directly measure your clients’ interaction with yoru software, and can identify trends that with simple changes can vastly improve your software and development processes.

Conclusion

So… these are some of the ways that we track the success of our software. There are a host of other methods to measure software, such as function points, lines of code, complexity, interfaces, velocity, etc. What ways do you measure your software? how do you define success? What are your plans for the future?

Leave me a comment and let me know what you think.

Book Review – How Would You Move Mount Fuji?

Over the holiday break I decided to tackle some of the books that I have stacking up next to my bedside.  One of them was How Would You Move Mount Fuji by William Poundstone.  This was a book that I know some of my colleagues had read already, and they recommended it highly, so I decided it was my turn. 

The book was a quick read.  The author kept my interest with not only the topic, but also with his concise explanations and his witty comments. 

Poundstone describes the history of the intelligence tests, and how it was developed.  They were used by our military to determine qualification for different job roles.  This led to the popular use of intelligence tests in the corporate world, particularly in the use of Silicon Valley.  During the civil rights movement, intelligence tests were determined to have a racial bias in the questions, so were banned as a hiring practice by the federal government.

The ban of intelligence tests did not deter those types of questions from remaining in interviews, however.  Looking for more people with minds like Bill Gates, puzzles made their way into the interviews at Microsoft.  They have popularized the use of logic puzzles and impossible questions.  Poundstone also describes the grueling day-long series of interviews at Microsoft and how you are rated throughout the process. 

My most important takeaways from the book was these nuggets of golden advice –

  1. When the technology you use is changing rapidly, you must hire for problem-solving stills, not just for the technology.
  2. A bad hiring decision is likely to hurt the company more than a  good hiring decision will help it.
  3. If you ask puzzle questions in your interview, make sure they are worth the effort by asking yourself these two questions:
    1. Are you willing to hire someone because of a good answer to this question?
    2. Are you willing to reject someone because of a bad answer?

I highly recommend this book to any hiring manager who plans on asking any puzzle type questions.  I also think the book adds insight into the overall interview process, even if you don’t plan on asking them that type of question.

Have you read the book?  Do you have an opinion on puzzle questions in interviews?  Leave me your feedback and let me know what you think.

The Lost Art of Debugging – Part 3 – Things To Do

As I have said before, debugging is a complex and time consuming process. I have outlined 10 resources for debugging, and provided a primer for things not to do when debugging. Now, we get to the meat and potatoes of debugging. This is a guide of things to do when debugging. I have broken this guide into three sections – a description of the Scientific Method of Debugging, Tips for Hunting for Bugs, and Bug Prevention Methods.

Scientific Method of Debugging

This is a parallel to the Scientific Method that you learned in your grade school science class. The book Code Complete by Steve McDonnell outlines this methodology to debug your applications.

  • Stabilize the bug by identifying the simplest test case that reproduces the error
  • Locate the source of the bug
    • Gather data about the bug and its behavior
    • Analyze the data regarding the bug
    • Form an hypothesis about the bug
    • Write test cases that would prove or disprove your hypothesis
    • Run the tests and prove your hypothesis, or begin again with a new hypothesis
  • Fix the defect
  • Test the fix with all of the new unit tests you have written
  • Continue to look for similar, cascading, or peripheral errors

For a more detailed breakdown of the Scientific Method of Debugging, read Code Complete by Steve McDonnell.

Bug Hunting Tips

These tips for finding bugs, broken out into different areas, will help you narrow down where your bug is.

General Tips

  • Be sure to save the original source code
  • Be sure the fix the problem, not the symptom
  • Make one change at a time
  • Check and Double Check your fix
  • Consider the possibility that the bug is generated from multiple sources
  • Divide and Conquer
  • Check other places where bugs have existed
  • Check other places where code has changed recently
  • Compile the application again

Logging

  • Insert Trace, Print, or Alert Statements to help track the bug
  • Create a logging methodology to trace the application
  • Check the Log Files of the servers, etc.
  • Search the web for the stack trace

Unit Testing

  • Design By Contract
  • Write unit tests that cause the error
  • Try to reproduce the bug in different ways
  • Try boundary cases and special cases

More Complex Methods

  • Recompile with a different compiler
  • Create a mini version of the application
  • Sequentialize the Parallel to see if it is a timing or resource issue
  • Try the code in different environments (local, dev, test, prod, other developers’ machines, etc.)
  • Grab someone else to talk through the bug without looking at the code (explaining the problem may trigger some ideas)
  • Do a full, in-depth code review on the broken code

Last Resort

  • Rewrite the whole section of code that is causing the bug
  • Take a break for a few minutes, or an hour, or until the next day, to give your mind time to process the data

Bug Prevention Methods

These are things that you should be doing during the planning and development of your applications that will help you identify, fix, and test your bugs after you are done with development.

  • Identify and track consistent bug types within your own code
  • Introduce debugging methodologies early
  • Implement loose coupling techniques
  • Implement Information Hiding Techniques
  • Write a regression test that tests the bug you just fixed

So… these are some different ways to attack debugging. Are there methods that have proven themselves that you have used? Do you take a different, more unique approach to debugging? What have your experiences shown to be your best practices? Please leave your feedback and let me know what you think.

The Lost Art of Debugging – Part 2 – Things Not To Do

Debugging is a complex and time consuming process.  In my last post I listed 10 Resources for Debugging, both web sites and books, that every software developer should read to keep their debugging skills sharp.  Knowing what not to do is just as important as knowing what you should do.  Here is a list of things not to do when you are debugging your application

  • Don’t guess what the problem is
  • Don’t dive into making changes without a plan
  • Don’t code right away without a thorough analysis
  • Don’t fix the shell of the problem, not the problem itself
  • Don’t trust line numbers in compiler messages
  • In fact, don’t trust the compiler messages at all
  • Don’t ignore the value of automated unit testing
  • Don’t delete the original code
  • Don’t ignore recent changes to your application
  • Don’t ignore resources available to you, like the Internet or the Library

This post is a teaser for the one that is to come – what thins should I do when debugging?  Is there a more formal process for debugging?  What steps should I follow?  What things should I look for?  What place should I look?

5 Benefits of Internal Corporate Blogs

Blogs are not new anymore.  They have been around for a few years, and have proliferated fairly deep into the technology culture.  If people are not writing one of their own, they are reading one, or a handful of them, or have an RSS aggregator where they are reading dozens or hundreds of blogs. 

Blogs have lots of uses.  They are great for viral advertising, news publication, syndication, and collection of public opinion about a topic.  You can have a personal blog to share photos of your vacations, a corporate blog for press releases, a news blog for niche news, and a host of other reasons to have a blog.  Blogs are an easy way for anyone to produce and consume any information about anything from anywhere. 

But blogs have taken off within corporations as well.  That is my interest here.  We have deployed a blogging pilot in the workplace, and I have become a big proponent.  At first it was part of my yearly objectives to help pilot and proliferate the use of blogs.  I have blocked time on my calendar twice a week for a half hour to post to my blog.  Now I am one of the most frequent bloggers in our pilot, and I blog both inside and outside the company.  I do this because I see such a value in the process.  Blogging is such an important tool in my manager arsenal, and I thought I would list the benefits of blogging from my experiences for the team and for the company.

Knowledge Sharing

Blogging inherently is a way to share information with its readers.    The writer composes the message, and it is consumed by many readers in the blogosphere.  These messages are (obviously) either bottom-up or top-down.  Bottom-up messages are written from the perspective of the line workers and low to mid level managers to express ideas or experiences they have had.  Top-down blogs are a way to deliver executive messages, such as strategic direction, corporate status, and announcements.  Blogs act as a written memory of events, lessons learned, experiences, successes, or announcements.  Blogs can also be a way to share expertise on a specific topic, such as data management, service oriented architecture, design patterns, or search engine optimization. 

Idea Solidification

The writer has to take the time to think about the message they want to deliver, collect their thoughts, organize them, and express them clearly.  That way the readers can read a focused message that delivers a clear and understandable message.  Blogs can also be a way to develop, solidify, and evolve an idea by posing a question, idea, or concept, and collect and narrow an idea through constructive comments. 

Project Management

Using a blog to document a project and its progress is a bit unorthodox.  Typically a wiki or another more collaborative environment is more appropriate.  But blogs are so easy to use, they make for a good medium to capture progress and communicate them easily.  Again, they act as a written memory of the project’s progress, successes, and lessons learned.

Cross Team Communication

Blogs are a great way to provide a medium for dynamic networks of conversations across teams.  These teams could be local or distributed, days, nights or weekends, and it will not matter.  A shared blog space will allow for ideas to be spread, information to be distributed, questions to be asked and answers to be shared.  Blogs crisscross people, departments, silos, grade levels, and experiences.

TeamBuilding

The simplest thing that blogs do is to bridge the boundaries of distance and culture by being always on, able to be shared and viewed at any time.  When launching a blog or posting a comment, a person takes themselves in written form and puts a bit of themselves out for everyone to look at, examine, scrutinize, comment on, and critique.  Successful corporate blogging creates an atmosphere of trust, so that people are not threatened, intimidated, or frightened to expose themselves.  Successfully sharing and collaborating this way fosters a spirit of participation that encourages people to continue to contribute their thoughts and ideas.  This leads to team learning and growth across the team, from within the team, and makes them stronger together. 

 

So what is your perspective on blogging within corporations, or blogging in general?  Is there value?  Is it a waste of time? Leave me feedback and let me know your thoughts.

 

Resources

7 Sources to Help You Conduct a Peer Code Review

The end of the year is swiftly approaching, and as the budget cycle is coming to a close, the work piles up quickly.  I have had a staff of three from January through August, and with the piling work, we have had to triple in size.  A staff of nine is very different to manage compared to a staff of three.  There are many things to wrangle during ramp-up: a new corporate culture, new policies, new development environment, new standards, new team members.  One of the tools in the technology manager’s arsenal is the peer review. 

The CodingHorror web site outlines why they think software development departments should perform code reviews in their article called “Code Reviews: Just Do It.”  Jared Richardson also describes the value of peer code reviews on his blog.  The basic reasons for implementing code reviews are obvious: reduced bugs, reduced rework, cross training, quicker ramp-up time, sharing of ideas, developer growth, mentorship opportunities, and team building.

Karl E. Wiegers has written a great article called “Seven Truths About Peer Reviews” available on processimpact.com that outlines a number of different methods to conduct your peer code review.  Some of the peer review methods methods he describes are inspection (both Fagan Method and Gilb/Graham Method), team reviews, walkthroughs, pair programming, peer deskchecks, and passarounds.  It’s a good read, and you should take a look.

Peer reviews do not need to be long, drawn out, complicated processes.  Jason Cohen describes some best practices for peer code review on his web site, SmartBear.com.  He has written a great looking book called “Best Kept Secrets of Peer Code Review” that you can order free on his site.  He recommends only reviewing code for five to ten minutes at a time, to keep everyone sharp, focused, and on point. 

You should definitely walk into your peer code review with a plan in mind.  Here is a Code Review Checklist for Macadamian that will help you organize your approach and keep you focused.

Finally, peer code reviews can be rough on the ego of the developer.  Here are “Ten Commandments of Ego-less Programming” based on Jerry Weinberg’s book , The Psychology of Computer Programming, from 1971.  This article describes how peer code reviews result in shorter project completion times, lower defect counts, and how developers had to set their egos aside to make it happen.

I am sure that code reviews will be good for my team, our projects, and our clients.  The hurdle is making them part of our development routine.  What do you think of code reviews?  Do you use them in your technology department?  What results have you seen?  Leave me feedback and let me know.

3 Indispensable Tools for Candidate Review

Tools are a great way to shorten time to complete tasks and improve quality in a process.  Hiring, Candidate Review, and Performance Review are no different.  Here are three tools that I have founds myself using while reviewing candidates.

SkillSurvey

So I got an automated email last week.  Big surprise, eh?  If you are like me, you get dozens of these a day.  This one was a little bit different.  It was from a company asking me to provide a reference for one of my old consultants.  That piqued my interest.  I opened it up, followed the link to SkillSurvey, and filled out the questions.  The simple instructions walked me through the process, and reassured me that my comments were anonymous and would be aggregated.  There were less than 10 questions, and were simple radio button scales.  I had the ability to put in freeform comments, and send it off. 

360Metrics

 Every other year, I have been encouraged by my management to do a 360 degree review of myself.  I log into 360Metrics and choose up to 3 direct reports, clients, peers, and managers.  Each of them receive emails to complete a set of predefined questions that rate and rank my skills on on a set of core values.  They are periodically reminded over the course of the review cycle.  When the review period closes, I receive an email and can generate an aggregate report of my ratings and comments. 

BrainBench

Every time I open up a requisition to find a new candidate for our team, I get a flood of resumes.  Eighty percent of those resumes have some sort of certification.  Most certifications we see are from Microsoft.  But some certifications come from BrainBench.  They offer certifications for individuals on a wide variety of specializations from Computer Software to Management, from Aptitude to Office Skills, from Communication to Industry Knowledge.  Obviously, the ones we see most often from BrainBench are technical in nature.  BrainBench also provides Pre-Hire Testing and Employee Development services to Employers.

Benefits

  • These tools are simple to use
  • The questions can be asked and are easy to analyze quantitatively
  • The same questions are asked of all candidates, so the questions and the delivery are consistent
  • They have open ended questions that allow you to analyze free-form text
  • Since these tools are web based, they can be leveraged (both from the candidate and from the hiring manager) 24 hours a day, 7 days a week.

Drawbacks

  • The tools may not be measuring what you are looking for
  • It is not difficult to “game” the system to produce phony or inaccurate results
  • The perception of these tools is that they measure which candidate is “better” than another
  • Some of these tools may give an advantage to good test-takers
  • The process may be too objective, and not interactive enough

Conclusion

I think that these tools can be a big help, and a great time saver.  Tools like SkillSurvey leave the evaluation up to the hiring manager, and help facilitate and focus the collection of critical hiring information.  However, the risk of relying on tools like BrainBench is that you take the interpersonal aspect out of the process.  You may miss an enthusiastic, bright candidate who is a bad test-taker, or who has the right attitude, but a different set of experiences. 

I am sure that there are lots of other tools available in this arena.  What tools have you used?  What have your experience been with them?  How do you use them in your process?

10 Steps to Conduct a Successful .Net Job Interview

This is a follow-up post to my posts on .Net Hiring Manager Resources and on Preparing for a .Net Interview. I will be interviewing a number of candidates next week for open positions in our department. I thought it would be good to review the process that we have typically followed, and get feedback.

1. Introduction

Someone should meet the candidate at the receptionist’s desk. It is a good idea to have the hiring manager do this. Look them in they eye, introduce yourself, and shake their hands firmly. On the walk to the interview room, share some small talk about the weather and the drive. This gives you an idea if they will mind how far they will have to drive to work. It also gives you the opportunity to check out how they dress and how they carry themselves. Once you are in the interview room, let them know that the interview will be about an hour long. Ask them if they would like something to drink, and to get more comfortable. Introduce them to everyone that they are interviewing.

2. Discuss the Open Position

Once everyone has introduced themselves and gotten comfortable, the hiring manager should ask how much they know about the open position. It is good to discuss the company’s goals, the division or department you work for, the specific project they would be working on (or describe a typical project the department works on), and describe the requirements of the position.

3. Review the Candidate’s Resume

Be prepared with questions about job positions or projects listed on the candidate’s resume. Open the floor, and let all those participating in the interview ask questions. This may be about specific technologies or techniques of interest, corporate culture differences, or specific challenges that were overcome. Give the candidate the chance to show what they have done.

4. .Net Trivia

This section of the interview should be driven by your technical gurus. Getting the people involved that your candidate would work with, and giving them ownership of the interview process, gives them buy-in on the decision. The purpose of these questions is to judge the specific experiences of the candidate. They are not intended as the be all and end all of measuring knowledge, but should be geared to give you the interviewer a good handle of what the candidate has seen or done.

5. HTML / JavaScript / CSS Questions

It is not uncommon for .Net developers to be lacking in experience when it comes to HTML, Cascading Style Sheets and JavaScript. Any good web developer will need to know these thing, however. If you are hiring for web development work, be sure to cover the basics, and make sure they understand how these all blend together.

6. General Interviewing Questions

In most cases, your candidate will not be working alone. Understanding how they work on a team is critical to their success, and yours, after they are hired. This is your opportunity to ask non-technical questions that focus on personality, teamwork, flexibility, communication, project management, leadership, and responsibility.

7. Whiteboard Questions

Ask your candidate questions that make them get up in front of a group, diagram their ideas, and explain why his ideas are the right approach. This will show you what the candidate is like when speaking in front of other people, like clients or project managers. You see their communication and persuasion skills, as well as their technical ability and diagram skills.

8. Puzzles & Riddles

This is a fun part of the interview. Be sure the candidate is relaxed, and make sure they understand that they are not expected to get the questions right. You give them a riddle or a puzzle, and have them talk through their thought process. This will give you an opportunity to see their creative, out-of-the-box thinking potential.

9. Questions from the Candidate

Expect questions from the candidate. If they have no questions for you, there may be cause for concern. They are not thinking very hard about what you have told them and about what might be coming next for them.

10. Wrap Up

Thank the candidate for their time. If possible, give them an idea about when they or their consulting company will hear back from you. Walk them back to the receptionist, and ask if they need any directions. Again, this will let you see how far in advance they have thought, how much hand-holding they will need, and how much they can think independently.

So what do you think of these steps? Are there things that I have missed that should be covered? What do you do differently (or the same) that you find valuable?