How to Know if It Worked

 It is not enough to provide career-help and career resources at the library. It is also essential to determine how effective programs and resources are for today's job seekers.

"Outcomes allow us to know something about the extent to which we have (or haven’t) reached our audiences. Information about outcomes allows us to strengthen our services. Equally important, it communicates the value of museums and libraries to the broadest spectrum of those to whom we account," (Outcome Based Evaluation).

In fact, according to Lear (2002), evaluation should really begin before a program even starts. This means determining user needs and expectations. Sometimes it may even mean identifying groups that may benefit from library services but are not currently using the library. One of the most basic ways to evaluate something is to count who is using it. Circulation, program attendance and website hits can all be used. However, a true evaluation will dig a little deeper--moving past the "what?" into the "why?" and "how come?".


Looking at Users & Needs
One way to do this is look at who is currently attending programs. Ask if these folks normally use the library or if they are being drawn in by a need (Lear, 2002). In Joseph Matthew's (2007) text on evaluating and measuring library services, he discusses segmenting library users by their needs, including experience seekers, scholars, facilitators and problem solvers. People may be coming to the library for social connections, research, helping a friend or with a specific question--and those are only half of the users Matthews lists (Matthews, 2007).

Surveys can be a way to look at programming needs. While surveys can be problematic, they can also be a place to start (Lear, 2002). It is important to remember that evaluation is not necessarily an objective activity. In other words, evaluation methods may only look at positive aspects of the program or be sabotaged by staff who do not want to participate or have not had the proper training (Matthews, 2007).

Job Seeking Behavior Research
In order to truly serve users needs, it may be helpful to skim through the vast body of research available on job-seeking behaviors. It is probably not necessary to do in-depth research, but understanding how patrons may be coming to the table is important. The following articles offer insight into just a few groups of people the library may be dealing with: Coaching the Job Seeker with Special Needs by Freddie Cheek; The New (and Ongoing) Job Crisis for Adult Learners: How Adult Educators Can Respond by Paul Jurmo; Do It Now! The Job Search Kit for Those 50 and Over, Third Edition by Marilyn Tellez.

  • Evaluation Forms: One common way to evaluate a program is to have participants fill out evaluation forms. The forms should balance demographic and program related questions. You do want to know who is attending programs, but you do not want to invade a user's privacy (Lear, 2002). Rating systems and comment fields are important to include. Check out the Resources for Librarians page on where to find example evaluation forms. The library will want to keep a programs and services file and also develop a method for dealing with negative feedback (Lear, 2002).

  • Outcome Based Evaluation: According to the Institute of Library and Museum Services (IMLS), it is important to evaluate the benefits to people including, "achievements or changes in skill, knowledge, attitude, behavior, condition, or life status for program participants" (Outcome Based Evaluation). The outcome based evaluation can include observation, questionnaires and survey answers. The librarian does not, however, have to talk to every user. With OBE, it is enough to get an idea of how you are effecting change in the community. If a program is done well, the outcomes should match up with the purposes originally stated. For example, the Computer Career-Help center should increase a user's familiarity with three major job searching sites. If users report an increased familiarity, one program outcome has been reached.
  • OBE is most effective with a target audience. Perhaps this kind of evaluation could be used if the library is gearing career-help toward a specific type of worker hit hard by the recession. One other advantage of OBE is that it can be modified as programs continue. If the goals or purposes of a career-help center at the public library change over time, the outcomes may also change. However, this type of evaluation also assumes libraries are not shooting in the dark. In order to know if the library has reached the right outcomes, there has to be a sense of where the library wants to go. For more concrete examples and a better idea of how to frame OBE, visit the IMLS evaluation site.

  • SWOT Analysis: Conducting a SWOT analysis involves listing strengths, weaknesses, opportunities and threats. This is another way to bring some dimension to the usage statistics compiled by the library. Librarians should ask themselves what strengths can be converted to weaknesses and what can be done better next time (Stein, 2009). It may be a good idea to gather all staff and volunteers who work on the program for a SWOT brainstorming session. It is likely that websites and other resources for career-help will be up for a long time and may need to be evaluated more than once as the economy and the community changes.

  • Cost Benefit Analysis: Sometimes it can be hard to articulate the less tangible benefits of libraries. The St. Louis Public Library developed an effective cost benefit analysis to try and get at the heart of this problem. Holt, Elliott and Moore (2010) found that the St. Louis Public Library provided $4 in services for every $1 in taxpayer money they received. Part of the study looked at what schools and businesses would have to spend to make up for the services being provided. The totals were in the millions, and reference services, which may include fielding career-help questions, were valued at $50 an hour (Holt, Elliott & Moore, 2010). One of the key points of this analysis is to provide something that is not already being done in your community. If the public library is not filling a genuine need for career-help, the cost may outweigh the benefits. If the library is supplementing existing community services, think carefully about the staff time and other resources being spent. Are the benefits still worth the cost?

Checking Out the Resources Themselves
Outside resources including websites, videos, books, speakers and handouts will probably be used for some aspect of your career-help. According to Jerrard (2009), though, many libraries are offering patrons resources that may hurt more than they will help in today's postmodern economy. Multimedia resources may be helpful to job seekers. On the other hand, if patrons do not have home computers, the library may want to make examples and tip sheets available in print as well (Jerrard, 2009). Qualifications for workers are not the same as they were 20 years ago, it is a good idea to make sure resources reflect the newest possible information. A good way to do that might be to ask local workforce development offices or companies to evaluate circulating materials and recommend the best websites. Also, check out the Examples and Resources pages for an idea of what the latest resources may look like.