Lesson 3: Evaluation Methodologies
A few main concepts covered:
Essential Question: What are some ways to evaluate information?
Guiding Questions: Is there a standardized method to do this? What are some skills that might help me evaluate information? What does the research say about the way students evaluate information?
Introduction:
Is there a quick and easy, four or five-step process, to effectively and efficiently evaluate web resources?
Short answer: nope!
Longer answer: There are checklists to help you navigate the evaluation process. Are checklists alone enough? Wichowski & Kohl, (2013, pp. 229-230) librarians [Note: Googled them- they're both library directors] writing in favor of using the CRAAP test (with modification- looking at "digital ethos," or the online communication characteristics that establish the author's credibility, authority, and reputation) to evaluate blogs and microblogs written by researchers state that " While some scholars argue that checklists like the CRAAP test are inappropriate and mechanistic evaluation tools, we refute this assessment, arguing that this checklist is a useful device especially for students new to research or scholars new to social media resources." Additionally, Wichowski & Kohl, (2013, p. 230) add that "We see the CRAAP test criteria as the most concise, flexible, and memorable evaluation tool of
the series of checklist tests that have been proposed since the late 1990s."
Here is an outline of the CRAAP test:
Source: (Nolan, May 2017)
Marc Meola, (2004, p.332) humanities librarian at the College of New Jersey, NJ, (at the time of the published paper) writes against using a checklist model approach alone, offering an alternative process: "a contextual approach- that uses peer review, comparison, and corroboration as methods for teaching Web-site evaluation." Among the arguments against using the checklist method alone, Meola, (2004) states that librarians overstate their expertise in evaluating information (and calls the idea that librarians alone should teach students how to evaluate websites dogma); students are not gullible simpletons when it comes to evaluating sources, and librarians should not assume so; promotes like "is the information reliable and error free" doesn't tell you anything about figuring it out (p. 336); the checklist approach promotes "a mechanical and algorithmic way of evaluation that is at odds with the higher-level judgement and intuition that we [librarians] presumably seek to cultivate as part of critical thinking," that is, it doesn't do the critical thinking for you (p. 337).
As far as this author is concerned, checklist method with modification is the way to go if you’re just starting out with evaluating sources.
We will use both the CRAAP test and the WWWDOT methods (who wrote it, why it was written, when it was written, does it help meet my needs, organization of the site, and to-do list for the future) to explore a web page and decide how credible and authoritative it is.
Zhang and Duke, (2011) developed the WWWDOT framework for use by elementary school students. Research shows that students do not approach internet information critically. Zhang and Duke, (2011, p. 145) wanted to know what impact the WWWDOT framework, would have, "if any, on fourth- and fifth-grade students’ awareness of the need and their ability to critically evaluate Web sites as sources of information? The results show that just four 30-minute sessions of instruction in the WWWDOT framework did make fourth- and fifth-grade students more aware of the need to evaluate information on the Internet for credibility and better able to evaluate the trustworthiness of Web sites on multiple dimensions"
They used an experimental design (with a control and experimental group of 242 fourth- and fifth-grade students). The experimental group was taught to use the WWWDOT framework, while the control group was not. They used three assessments to rank answers (a questionnaire, a Single Web Site Evaluation Task, and a Web Site Ranking Task). These first-of-their kind assessment to measure elementary school students' website evaluation skills studied: 1) awareness to critically evaluate web resources; use of WWWDOT to evaluate web sites; web browser and info-seeking skills (by using a questionnaire); 2) how students evaluate a website's trustworthiness (by using the Single Web Site Evaluation Task); and 3) how students identify trustworthy vs untrustworthy websites (by using a Web Site Ranking Task form).
The authors conclude that: “students who experienced the WWWDOT lessons came to realize that information on the Internet is not always accurate or true. Arguably, this awareness is a fundamental undergirding for anything the students might learn about the evaluation of Web sites as sources of information. As such, we believe this is especially appropriate and important to develop in elementary-age students." (Zhang and Duke, 2011, p. 148). Some limitations of this study include: it was only tested on fourth and fifth grade students (and may or may not apply to other age groups); WWWDOT did not improve students’ self-perceived Web site evaluation skills (to judge whether it met their information needs, overall judgement of a site's trustworthiness, ranking of websites by their relative trustworthiness).
In order to address some of the possible setbacks, this lesson plans to incorporate more time on analyzing the concept of creator authority, analyzing arguments, and metacognitive thinking on information needs.
References:
Meola, M. (2004). Chucking the checklist: A contextual approach to teaching undergraduates web-site evaluation. Portal: Libraries and the Academy, 4(3), 331-344. doi:10.1353/pla.2004.0055
Nolan, S.A. (May 2017). Critical thinking and information fluency: Fake news in the classroom. [Webpage]. https://www.apa.org/ed/precollege/ptn/2017/05/fake-news
Wichowski, D. & Kohl, L. (2013). Establishing credibility in the information jungle: Blogs, microblogs, and the CRAAP test. Library Staff Publications, Presentations & Journal Articles. https://digitalcommons.bryant.edu/cgi/viewcontent.cgi?article=1002&context=libr_jou
Zhang, S., & Duke, N. K. (2011). The impact of instruction in the WWWDOT framework on students’ disposition and ability to evaluate web sites as sources of information. The Elementary School Journal, 112(1), 132-154. doi:10.1086/660687
Standards
Note: To address college readiness, several state and national standards have been selected, emphasizing related skills and concepts. However, while this lesson touches on several of these, not all of these will be assessed here.
|
Association for College and Research Libraries (ACRL). Information Literacy Competency Standards for Higher Education.
http://www.ala.org/acrl/standards/ilframework Frame(s): Authority Is Constructed and Contextual; Information Creation as a Process; Information Has Value; Research as Inquiry; Scholarship as Conversation; Searching as Strategic Exploration Knowledge Practices: explore the concept of authority; assess info product's fit to your info. need; be aware of how info. is published/make informed online choices; formulate questions for research...; identify contribution of specific authors/info sources; determine scope of info-seeking task Dispositions: keep an open mind when evaluating info; assess info product's fit to your info. need; accept ambiguity/value of info. creation expressed in emerging formats or modes; respect the original ideas of others/see yourself as a contributor to the info. marketplace; research as exploration; research as ongoing conversation; exhibit mental flexibility and creativity |
|
American Association for School Librarians (AASL). National School Library Standards for Learners, School Librarians, and School Libraries (AASL Standards) https://standards.aasl.org/wp-content/uploads/2018/08/180206-AASL-framework-for-learners-2.pdf Domain(s): Think, Create, Share, and Grow Shared Foundation(s): Inquire, Include, Collaborate, Curate, Explore, Engage Think (2. Adopting a discerning stance toward points of view and opinions expressed in information resources and learning products. Etc.); Create (2. Collecting information representing diverse perspectives); Share (2. Involving diverse perspectives in their own inquiry processes.); Grow (1. Performing ongoing analysis of and reflection on the quality, usefulness, and accuracy of curated resources.). Etc. |
|
International Society for Technology in Education (ISTE). Standards for Students and Educators https://www.iste.org/standards/for-students
ISTE for Students- Knowledge Constructor. 3b. Students evaluate the accuracy, perspective, credibility and relevance of information, media, data or other resources. Innovative Designer: 4a. Students know and use a deliberate design process for generating ideas, testing theories, creating innovative artifacts or solving authentic problems; 4d. Students exhibit a tolerance for ambiguity, perseverance and the capacity to work with open-ended problems. |
|
Texas School Library Standards Strand 1: Information Literacy 1.2.4 show evidence of the ability to evaluate and validate information for: • Authority • Bias • Credibility • Currency • Relevance |
|
Lesson Objectives/What am I learning? The information literate student…
Learning Outcomes/Why did I learn this? [Demonstration of Learning] • At the end of this lesson, my students will be able to evaluate the authority, accuracy, perspective, credibility and relevance of an online resource by using a method. |
Part I- Find an Article
Instructions: Use the resources below and find an article that interests you.
Resources:
1 ) Please make sure you have access to TexShare to access the databases listed below.
El Paso Public Library (EPPL) website, get access: http://www.elpasolibrary.org/research/online-research
TexShare login: https://texshare.net/
Part II- Find an Article
Instructions: Scan, skim, and use the SMOG method to determine the article’s readability.
The SMOG (Simple Measure of Gobbledygook) Reading Level Calculator provides a measure of readability, the indication of number of years of education that a person needs to be able to understand the text easily on the first reading.
https://www.online-utility.org/english/readability_test_and_improve.jsp
You can also look up the Lexile level: https://hub.lexile.com/analyzer
Part III- Use the WWWDOT or CRAAP method to analyze your article.
Part IV- Use the WWWDOT or CRAAP method to analyze a website.
Instructions: Click on a recommended website below.
The Occupational Information Network
https://www.onetonline.org/
Pros and Cons of Current Issues
https://www.procon.org/
The Universes of Max Tegmark
https://space.mit.edu/home/tegmark/
WWWDOT and CRAAP Test Methods:
WWWDOT: A Tool for Supporting Critical Reading of Internet Sites
Who wrote this (and what credentials do they have?)
Why did they write it?
When was it written and updated?
Does this help my needs (and how)?
Organization of site (you can write and/or draw)
To do list for the future
Source: (Zhang & Duke, 2011)
The CRAAP test:
Source: (Nolan, May 2017)
