Evaluation

Statement of Competency:
Evaluate programs and services, including the use of appropriate methods and tools to assess effectiveness.

Introduction

Evaluation is a critical component of professional practice in library and information science (LIS), ensuring that programs, services, and systems effectively meet user needs and organizational goals. Competency N focuses on the ability to assess effectiveness using structured methods and apply those findings to improve outcomes. This requires not only gathering information, but also interpreting results, identifying patterns, and making informed decisions based on evidence.

Evaluation in LIS often combines both qualitative and quantitative approaches. Quantitative measures, such as response time or usage statistics, provide measurable indicators of performance, while qualitative observations—such as user satisfaction, clarity of communication, and overall experience—offer insight into how services function in practice (Hernon & Altman, 2010; Wildemuth, 2017). Using both approaches allows for a more comprehensive understanding of effectiveness.

Structured evaluation criteria are also essential. Frameworks such as the CRAAP test emphasize evaluating information based on currency, relevance, authority, accuracy, and purpose (Blakeslee, 2004). While originally designed for source evaluation, these principles translate directly into evaluating services and systems by providing consistent criteria for assessing quality and reliability. Similarly, user-centered evaluation approaches emphasize that services must be assessed not only for efficiency, but also for how well they support user needs and experiences (Fisher & Fulton, 2022).

Evaluation is not a one-time process but an ongoing practice that supports continuous improvement. As Hernon and Altman (2010) note, assessment findings should inform decision-making, allowing organizations to refine services and better serve their communities. Through my coursework, I have applied evaluation methods across multiple contexts, including service interactions, system-level resource management, and instructional design. These experiences demonstrate my ability to evaluate effectiveness and apply findings in meaningful ways.

Evidence

My evaluation skills developed through coursework that required me to assess real-world services, systems, and learning tools. Across these projects, I applied structured criteria, compared outcomes, and reflected on effectiveness from both a user and professional perspective. The following artifacts demonstrate my ability to evaluate services, systems, and programs using appropriate methods and apply those findings to practice.

Artifact #1

Evaluation of Virtual Reference Services (INFO 287: Library Services in the Digital Age)

Justification

This project required me to evaluate virtual reference services across multiple libraries using consistent criteria, including response time, helpfulness, usability, and overall user experience. By asking the same or similar questions across different systems, I was able to reduce variability and make direct comparisons between services.

The evaluation incorporated both quantitative and qualitative measures. I tracked response times—often within seconds—and compared how quickly users were connected to librarians, while also analyzing the quality of responses and level of engagement. For example, some interactions were described as “transactional,” with limited follow-up, while others included clarifying questions and more conversational support. This distinction reflects the importance of user-centered evaluation, where effectiveness is measured not only by speed but also by depth of interaction (Hernon & Altman, 2010).

I also evaluated system limitations, including restricted access for non-affiliated users and differences in service availability, such as reduced weekend support. These observations demonstrate how institutional policies and staffing impact service effectiveness. By comparing multiple libraries, I identified patterns in how services are delivered and how those differences shape user experience.

This project reflects evaluation practices that emphasize structured comparison, user experience, and contextual factors. It supports Competency N by demonstrating my ability to systematically evaluate services and apply findings to assess effectiveness and identify areas for improvement.

Artifact #2

Electronic Resource Management (CORAL System) (INFO 287: Library Services in the Digital Age)

Justification

This artifact demonstrates my ability to evaluate information resources and systems at an organizational level. Working within the CORAL electronic resource management system, I assessed a digital journal resource by examining metadata, access conditions, licensing, and overall usability.

This evaluation required applying criteria related to accessibility, relevance, and system organization. Rather than focusing only on content, I evaluated how effectively the system supports access and management of resources. This aligns with evaluation practices that emphasize both system functionality and user access (Wildemuth, 2017).

Additionally, this work highlights how evaluation supports decision-making in LIS environments. Determining whether a resource is valuable involves considering factors such as cost, accessibility, and user needs. These considerations reflect broader evaluation frameworks that connect assessment to organizational priorities and service effectiveness (Hernon & Altman, 2010).

This artifact also reinforced that evaluation is often ongoing. Resource management systems require continual review to ensure that materials remain relevant and accessible. This reflects the iterative nature of evaluation, where findings inform future decisions and adjustments.

This project supports Competency N by demonstrating my ability to evaluate systems using structured criteria and apply those evaluations to resource management and service improvement.

Artifact #3

Interactive Information Literacy Escape Room (INFO 287: Library Services in the Digital Age)

Justification

This project demonstrates my ability to evaluate learning outcomes through the design of an interactive assessment tool. My teammate and I developed an escape room activity based on The Crucible that requires participants to apply information literacy concepts, including accuracy, authority, purpose, and proper citation, in order to progress.

The structure of the activity functioned as a series of evaluation checkpoints. Each “lock” requires users to demonstrate understanding before moving forward, effectively assessing their ability to apply concepts such as MLA citation, source credibility, and plagiarism recognition. This aligns with established evaluation practices where learning is assessed through application rather than memorization (Booth, 2011).

The project also incorporates immediate feedback. Users who selected incorrect answers are prompted to try again, while correct responses allow progression. This iterative feedback loop reflects evaluation methods that support both assessment and improvement, reinforcing learning through repeated engagement (Wildemuth, 2017).

Additionally, the use of the CRAAP framework within the activity provides structured evaluation criteria, ensuring that the assessment is consistent and aligned with information literacy standards (Blakeslee, 2004). This demonstrates how evaluation tools can be embedded within programs to measure effectiveness while supporting user learning.

This project supports Competency N by demonstrating my ability to design and apply evaluation tools that assess learning outcomes and measure program effectiveness.

Conclusion

Through these projects, I developed the ability to evaluate services, systems, and programs using structured methods and criteria. My work shows that evaluation in LIS is not limited to measuring performance, but also involves understanding user experience, identifying limitations, and applying findings to improve outcomes. By working across different contexts—virtual reference, resource management systems, and instructional design—I developed a well-rounded approach to evaluation.

As I move forward in my career, I will continue to apply evaluation methods to assess effectiveness and support continuous improvement. Engaging with LIS research and best practices will allow me to refine these skills and ensure that my work remains responsive, user-centered, and evidence-based.

References

Blakeslee, S. (2004). The CRAAP test. California State University, Chico.

Booth, C. (2011). Reflective teaching, effective learning: Instructional literacy for library educators. American Library Association.

Fisher, K. E., & Fulton, C. (2022). Information communities. In S. Hirsh (Ed.), Information services today: An introduction (3rd ed.). Rowman & Littlefield.

Hernon, P., & Altman, E. (2010). Assessing service quality: Satisfying the expectations of library customers (2nd ed.). American Library Association.

Hirsh, S. (2022). Information services today: An introduction (3rd ed.). Rowman & Littlefield.

Wildemuth, B. M. (2017). Applications of social research methods to questions in information and library science (2nd ed.). Libraries Unlimited.