1688421 – SAGE Publications, Inc. (US) ©
evaluation design will still yield useful information. Yet a strong logic model within a rigorousevaluation design will enable much stronger conclusions regarding program effectiveness andimpact. As you have likely surmised, a weak logic model within a strong evaluation designprovides little useful information, just as an unreadable treasure map within a sturdy homebrings you no closer to the treasure. That said, in this section you will add strength and depthto your logic model by continuing to build upon the evaluation matrix you began in Chapter 7.Methods and tools will be identified or developed for each indicator on your logic model,addressing the question, How will you collect your data?
Although there are many evaluation methods, most are classified as qualitative, quantitative, orboth. Qualitative methods rely primarily on noncategorical, free response, observational, ornarrative descriptions of a program, collected through methods such as open-ended surveyitems, interviews, or observations. Quantitative methods, on the other hand, rely primarily ondiscrete categories, such as counts, numbers, and multiple-choice responses. Qualitative andquantitative methods reinforce each other in an evaluation, as qualitative data can help todescribe, illuminate, and provide a depth of understanding to quantitative findings. For thisreason, you may want to choose an evaluation design that includes a combination ofqualitative and quantitative methods, commonly referred to as mixed methods. Some commonevaluation methods are discussed below and include assessments and tests, surveys andquestionnaires, interviews and focus groups, observations, existing data, portfolios, and casestudies. Rubrics are also included as an evaluation tool that is often used to score, categorize,or code interviews, observations, portfolios, qualitative assessments, and case studies.
Qualitative methods: evaluation methods that rely on noncategorical data and freeresponse, observational, or narrative descriptions.
Quantitative methods: evaluation methods that rely on categorical or numerical data.
Mixed methods: evaluation methods that rely on both quantitative and qualitativedata.
Before delving in to different methods, it is worth mentioning the ways in which the termsassessment and survey are sometimes used and misused. First, while the term “survey” issometimes used synonymously with “evaluation,” evaluation does not mean survey. A survey isa tool that can be used in an evaluation and it is perhaps one of the most common tools usedin evaluation, but it is just one tool nonetheless.
Another terminology confusion is between “assessment” and “evaluation.” These too are oftenused interchangeably. However, many in the field of evaluation would argue that assessmenthas a quantitative connotation, while evaluation can be mixed method.
Similarly, the term “measurement” is often used synonymously with “assessment,” andmeasurement too has a quantitative connotation. I believe the confusion lies in the terms“assess,” “evaluate,” and “measure”; they are synonyms. So, it only makes sense thatassessment and evaluation, and sometimes measurement, are used synonymously. And whilethere is nothing inherently wrong with using these terms interchangeably, it is a good idea toask for clarification when the terms assessment and measurement are used. Some majorfunders use the term “assessment plan” to mean “evaluation plan,” but others may use the termassessment as an indication that they would like quantitative measurement. The takeawayfrom this is to communicate with stakeholders such that the evaluation (or assessment) youdesign meets their information needs and expectations.
8.2.1 Qualitative Methods
Qualitative methods focus on noncategorical, observational, or narrative data. Evaluation usingqualitative methods is primarily inductive, in that data are collected and examined for patterns.These patterns are then used to make generalizations and formulate hypotheses based onthese generalizations. Qualitative methods include interviews and focus groups, observations,some types of existing data, portfolios, and case studies. Each method is described in thefollowing paragraphs.