General Rules

• Lessons Learned sessions tend to bring out the negatives, be sure to ask "What Went Right"
• Conduct Lessons Learned meetings
• Depending on the size and type of project there can be one or more LL session.
• Use survey questions (recommended below) to solicit feedback
• Meetings and questions should address both PM and non-PM Lessons Learned

Before the meeting

• Use online anonymous survey to collect feedback, distill down to areas of discussion and focus actual meeting(s) around identifying specific actions to follow
• Prepare the team in advance of the meeting for what will be discussed
• Take steps to ensure that the meeting will be comfortable and non-confrontational
• Involve a "scribe" or two to ensure that all notes get captured - the PM could be considered for this role
• Project Manager of the project should not be facilitator of the lessons learned - get another PM or strong facilitator
• Who is invited?  Should include team members, customers, other stakeholders
• Try to plan to feed people (when possible), breaks may be needed as well
• Encourage creative thinking - make it a relaxed environment for having the discussion - go to a less familiar, interesting place

During the meeting

Introduction

• Be clear about the agenda, what the goals are for the meeting
• Develop a list of ground rules - have the team develop these (suggested set - be nice, don't blame, be constructive)
• Hand out set of topics and ask team which items they would like to talk about - go around the table to discuss - make sure each person gets a turn
• Start session with an overview of the project - goals, what we accomplished, any particular challenges encountered

During discussion

• Focusing the discussion around actions/solutions rather than rehashing the past
• Identify top items to be worked on

Closing

• Identify clear actions and owners where appropriate
• If you use a meeting to collect the feedback, save a period of time at the end to focus on identifying solutions for the most important issues

After the meeting

• Followup on actions agreed upon

Other considerations

• Start out focusing on the positive lessons learned - what went well
• Be prepared to discuss specific problems (problem, perceived cause, what should have been done differently)
• For large projects, don't do it only at the end of the project - do LL at various key milestones
• Develop standard LL templates for online surveys

You can conduct surveys using questions selected from the following

Sample Questions for Lessons Learned Sessions

Mid-Project Lessons Learned

• What is going well that we want to continue?
• What isn't going well that we want to fix?
• What can we do to improve?

Key End of Project Questions

• General
  • List top significant project successes? (keep to a manageable number)
  • What obstacles or unanticipated circumstances made it difficult to complete the project?
  • Based on what you know now, what should have been done differently in this project?
• Communications
  • Were communications adequate in all activities? (right time, right audience, right information, right type)  If not, in what ways could communications be improved?
  • Was documentation appropriate, timely, and clear?  If not, what should be included for future efforts?
• Project Management
  • What value did project management bring to the project? 
• Project Planning
  • How well was the project staffed to complete the project?
  • How well was the project planned?
  • How well were roles and responsibilities defined?
  • How well was stakeholder input factored into the project planning process?
• Project Execution
  • How well did we execute according to plan?
  • How well were the stakeholders involved in the execution of the project? 
  • How well were issues resolved?
  • How well did the teams/sub-teams work together?
• Testing
  • Was adequate testing performed (if applicable)?  If not, what should be done differently next time?
• Cutover
  • Was cutover planned well?  Executed well?  If not, what should be done differently next time?

Additional questions - use as appropriate for your project

Questions should be selected based on the type and size of the project.

• Project Planning
  • Was the project plan realistic or unrealistic?
  • Did the planning effort include key stakeholders?
  • Were all major activities accounted for?
  • Did the individual team members responsible for the work provide the estimates?
  • Were appropriate staff involved in the planning effort?
  • Do you believe the technology PM applied provided a better ulimate outcome for this project?
• Communications
  • Were communications effective?
  • Were meetings effective?
  • Were expectations clear?
  • Did you know where to go to get information about the project?
  • Did communications go to the right people, at the right time?
  • Were communications with our consultant effective?
  • Did we escalate effectively and at the right time?
  • Were issues dealt with effectively?
• Staffing, Roles, and Responsibilities
  • Were people appropriately assigned to the project?
  • Was staffing sufficient?
  • Were roles and responsibilities clear?
  • Did team members and stakeholder fulfill their roles appropriately?
  • Was leadership and governance effective?
  • Have you participated in this type of technology project before?
• Project Execution
  • Was the project executed according to plan?
  • Was there an appropriate level of commitment and urgency?
  • Was change management performed appropriately?
  • Was risk management performed appropriately?
  • Was issue management performed appropriately
  • Would you recommend this technology PM service to a peer?
  • Was the project flexible enough to adapt to unforeseen circumstances?
  • Did the project unduly stray from identified goals and objectives?
  • How well were scope changes managed
• Teamwork
  • Did the teams/sub-teams work effectively together?
  • Did the teams share a common goal/mission?
  • All stakeholders were appropriately engaged, involved, and available?
• Cutover
  • Was the cutover successful?
  • Was the cutover well planned?
  • Were cutover communications effective?
• Anything else?
  • What tools or techniques worked well?
  • How well did the project meet objectives?

End of Document

Following data is remaining work that team members will be cleaning up.

Irina Naydich

• What worked well either for this project or for the project team?
• What did not work well either for this project or for the project team?
• What needs to be done over or differently?
• What surprises did the team have to deal with?
• What project circumstances were not anticipated?
• Were the project goals attained? If not, what changes need to be made to meet goals in the future?

• List top three significant project successes.
• Other notable project successes.
• Project shortcomings and Solutions.
  

Micci Bogard

Space for Micci

Lisa Stensland

Survey Questions from PS 8.9 Upgrade

These questions were used in an on-line survey and folks had to answer on a scale from 1-5 (1=Strongly Disagree, 5=Strongly Agree) except where otherwise noted.
OVERALL PROJECT ASSESSMENT

• Overall, this project was a success.
• I would want to be involved in another project of this nature.
• Project successes were recognized.
• (Open-Ended) What factors most contributed to the success of this project?
• (Open-Ended) What was difficult or frustrating in this project that we could have prevented?
• (Open-Ended) Based on what you know now, what should have been done differently in this project?

LEADERSHIP

• The project manager was involved at the right level of the project.
• The lead for my area was involved at the right level of the project.
• The lead for my area clearly defined assignments.
• The lead for my area kept me well informed of the project status.
• Issues raised to the project leadership were addressed appropriately.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

COMMUNICATION

• For the overall project, I received the right information from the right people at the right time.
• Within my area, I received the right information from the right people at the right time.
• Meetings I attended regarding the project were useful.
• Project status reports were useful.
• I knew where to go to get information on the project.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

STAKEHOLDER PARTICIPATION

• The users of the PeopleSoft system were adequately involved in this project.
• The PeopleSoft directors were adequately involved in this project.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

PROJECT PLANNING

• I was adequately involved in the planning process.
• Being involved in the project planning was beneficial for me.
• The project planning effort resulted in a realistic plan.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

PROJECT CONTROL

• Management of project scope was effective.
• Management of project budget was effective.
• Management of project schedule was effective.
• Management of project resourcing was effective.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

PROJECT RESOURCING

• Roles and responsibilities were clearly defined.
• Staffing in my area was sufficient for this project.
• The appropriate people were assigned to the tasks in my area.
• My ideas and concerns were fairly addressed.
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

PROJECT SCHEDULE

• The project schedule was clearly defined.
• Adjustments to the schedule were made when needed.
• I knew what I needed to do and by when.
• I received deliverables from my co-workers in a timely manner.
• I was able to do my work at a comfortable pace.

TESTING

• Developing the test strategy was valuable.
• Developing the test plan was valuable.
• The system/parallel testing effort was well-coordinated.
• The system/parallel testing effort throughly covered all areas.
• The system/parallel testing effort prevented major surprises from arising in production.
• Issues uncovered in testing were escalated to CIT effectively.
• Issues uncovered in testing were resolved promptly.
• Issues uncovered in testing were resolved correctly.
• (Open-Ended) In what areas should we have tested more?
• (Open-Ended) In what areas did we spend too much time testing?
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

CUTOVER

• For cutover weekend, planning was done well.
• For cutover weekend, execution was done well.
• For cutover weekend, communication was done well.
• I was prepared to do my job on Day 1.
• I was able to do my job on Day 1 without any major issues.

RELIABILITY AND QUALITY

• We have now been in production with PeopleSoft 8.9 for about 2 months. Compared to what you expected, how is the system quality and reliability? (Possible answers: "Better than I expected", "About what I expected", "Worse than I expected")
• (Open-Ended) What details, if any, would you like to provide regarding the answers you chose?

More generic survey questions

This set of questions was used for a Blackboard Upgrade project. Again, feedback was collected via online survey.

For each topic listed below, 3 questions were asked:

• What went well?
• What didn't go well?
• What would you do differently?

Topics (sub-bullet questions were "considerations" were examples given to the responder to help provide some perspective)