Related Links

Progressive Era History (v.9)

Labor History (v.8)

View All Issues

PHRC's rating system

Evaluating Web Sites

Rating System for Evaluating Public History Web Sites
By Debra DeRuyver, Jennifer Evans, James Melzer, and Emma Wilmer
Written & Mounted April 30, 2000

Table of Contents:

Intro
Explanation of PHRC Rating System
Application of PHRC Rating System
A Note About the Rating System
Rating Scale
Basic Criteria
Public History Specific Criteria

Intro

Regular visitors to public history Web sites include historians and non-historians; academic and non-academic publics. Oft times, Web sites are used as the sole source of historical information on particular topics, particularly by K-12 and undergraduate students. Therefore, critical analysis of these sites is of the utmost importance. We as public historians have a responsibility to critique and evaluate these online resources, both to help improve the specific sites under review and to raise the bar for the entire field of public history. Site analyses provide benefits to several different groups:

    • The user gains an idea of the strengths and weaknesses of a site and the validity and reliability of the information contained therein.
    • The reviewed site benefits from a constructive outside appraisal.
    • Un-reviewed public history sites benefit from having models of best practices and common pitfalls.
    • Critical site analyses encourage better scholarship and more engaging presentations of public history.
    • Ultimately, the historical record is served by site analyses; site reviews provide traces of presentations of public history on the Web during its first decade of existence.

Explanation of PHRC Rating System

In order to facilitate Web site analysis and to create a uniform standard for rating sites, PHRC has developed a list of criteria for evaluating public history Web sites. The criteria have been divided into two major sections: basic and public history specific. This division helps the reviewer analyze a site from different perspectives and leads to a more thorough analysis. Basic criteria for evaluating the quality of Web sites are readily available online. PHRC's managing editors reviewed a number of different evaluation checklists and compiled the criteria that we thought were most useful for our purposes into six subsections (see below). The public history specific criteria have been created by PHRC and are criteria that we feel are crucial aspects of the ideal public history Web site.

PHRC has developed a weighted point system to standardize site ratings and to create consistent ratings across sites. The Basic Criteria and the Public History Specific Criteria each contain one hundred possible points. Therefore, the maximum rating a site can receive is 200 points. The Basic Criteria section is divided into six subsections, with each subsection worth 15 points for a total of 90 points. The last ten points are available for the reviewer's discretionary use in order to reflect his or her overall impression of the site.

The division of points in the Public History Specific Criteria section is content oriented and driven by current practices as well as our conception of the ideal public history Web site. This section is divided into four subsections. Because public history’s unique contribution to the public sphere and historical discourse is its "Interpretation of Materials," this subsection is given the most weight. The remaining points are divided equally between the final three subsections. "Primary Source Documents," "Education & Outreach," and "Promotion of a Community of Interest" are all indicative of the direction in which public history should be moving and in which the Web is positioned to make a unique contribution.

Application of PHRC Rating System

This rating system works in the following way: a user reviews a site and assigns points for each subcategory. These points are added up, and the total is applied to a scale that indicates the rating that a site will receive (see below). The bullets beneath each subcategory suggest specific facets of the broader criteria to examine. They are not intended to be used as a strict method for assigning points. For example, in the Timeliness/Permanence subsection (worth 15 points total) there are three bulleted suggestions; these three suggestions are not necessarily worth five points each.

The user is encouraged to apply the point system rigorously and to award points judiciously. Rating sites is and should be a difficult task, requiring a good deal of thought and analysis. Although it may be clear that the creators have expended significant resources on their site, points should not be awarded for "sweat of the brow," but rather for overall site quality. The goal of PHRC is to be rigorous and analytic. With these reviews we hope to encourage better scholarship rather than discourage the practice of public history on the Web.

We recommend reading Issue Number 7 of our Web site reviews: PHRC's Rating System in the Classroom: Three Reviews of Canadian Web Sites to learn more about the application of the rating system. In this special issue, we have published three reviews as they were originally received by us, followed by comments from PHRC's editors regarding how the rating system was applied, where the editors agreed or disagreed with the author's assessment, and what the points that the editors would have awarded the site. The introduction to the issue is also helpful in its summary of some of the typical errors made in applying the rating system.

A Note About the Rating System

These ratings are only applied to reviews of Web sites that present public history. PHRC is currently in the process of developing a checklist for the review of Web sites of public history consulting firms. Also, only reviews posted on or after April 30, 2000, reflect the current ratings criteria; reviews posted before this date use PHRC’s previous rating system. Prior to March 2003, PHRC used the symbol of the statue of liberty to designate the levels of our rating system. A photo of the earth is now used in order to better signify the international aims of the site.

The Education (formerly called Education/Outreach) and Promotion of a Community of Interest sections of the criteria were revised in June 2003 in order to more explicitly describe the editorial assumptions that had governed PHRC's application of these criteria since April 2000.

Rating Scale

1 - 20 points: Half Earth
21 - 40 points: One Earth
41 - 60 points: One and One Half Earths
61 - 80 points: Two Earths
81 - 100 points: Two and One Half Earths
101 - 120 points: Three Earths
121 - 140 points: Three and One Half Earths
141-160 points: Four Earths
161 - 180 points: Four and One Half Earths
181 - 200 points: Five Earths

Basic Criteria

Scope/Content (15 points possible)

    • Audience for the site/ stated purpose/ mission statement
    • Comprehensiveness/ completeness/ does it do what it says it will
    • Adequate amount of material to allow users to see a number of viewpoints and draw their own conclusions?
    • Well written, free of grammatical errors, typos, etc.
    • NOTE: Scope and Content are covered in more depth below, under Public History Specific

Authority/Bias (15 points possible)

    • Who is sponsoring the site/ providing information?
    • Reliable contact information/ snail mail/ phone/ etc.
    • What is the point of view

Timeliness/Permanence (15 points possible)

    • Posting/revision dates
    • Do links work?
    • Do you think the site will still be there in a year?

Value Added Features (15 points possible)

    • Index/search/sitemaps
    • Summaries/abstracts
    • Annotated Links

Technical Aspects (15 points)

    • Accessibility/ text only alternative/ alt tags in images/ plug-ins/ browser compatibility/ ADA compliant
    • Navigation/ knowing where you are in the site/ knowing how to get back, go forward
    • Printability

Aesthetics/Visual Clarity & Appeal (15 points)

    • Consistent theme
    • Visual cohesiveness/ page layout
    • Readability/ colors/ font sizes/ background images

Overall impression of the site (10 points)

    • Favorable/unfavorable

Public History Specific Criteria

Interpretation of materials (40 points)

    • Original interpretive essays/exhibit available, written by an identified authority in the field?
    • Secondary/primary sources used & properly cited
    • Peer review of site/interpretations
    • Does the site point people to additional sources/ help them formulate their own view point of the subject/
    • Is the site accessible to non-academic publics/ jargon free/ visually appealing

Primary source documents (20 points possible)

    • Searchable database of materials?
    • Does the site include both scanned, verifiable, images and rekeyed, searchable information?
    • Explanations of how primary source items were described, digitized, etc.
    • Physical location of originals cited?
    • Scope of the collection? Reasoning behind the selection of particular documents?
    • Has the site taken into account copyright issues regarding the material they're making available?

Education (20 points possible)

    • Available curriculum enabling a teacher to use the material in a classroom?
    • Does the site contain interactive learning materials? For example, online games that promote critical thinking and learning of the material, tools that assist students in using the material to create school reports, and exercises designed for students to complete on their own, outside the supervision of an instructor?
    • Does the site showcase or link to student work utilizing its on/offline materials?
    • Any other special features that actively enhance the use of the site for the purposes of education? For example, an "Ask Doctor History" type of email address that students (or teachers) can pose questions to or opportunities to interact with experts through synchronous chat?

Promotion of a Community of Interest (20 points possible)

In his discussion of the New Deal Network, Tom Thurston writes: "I'd like to suggest a third category of websites, 'associative portals,' or aortals. An aortal is a site that enables the creation of social networks…. Its structure is designed to foster the creation of distributed networks of academic communities. If a website aims to be more than an archive, an online exhibit, a virtual textbook, it must consider how it can nurture similar dynamic relationships" ("Building Social Networks with Computer Networks: A New Deal for Teaching and Learning").

    • Evidence of community involvement with materials? For example, mechanisms for community members to contribute to the historical record presented on the site?
    • Evidence of active involvement in issues of importance to the communities whose stories are told on the site or to communities whose interests correspond to the site's topic? For example, the New Deal Network actively supported the preservation of Hugo Gellert's Seward Park Murals, using its electronic network and Web site to assist organizational efforts, publicize the issue, and further legitimize the preservation efforts.
    • Does the site sponsor any Listservs, bulletin boards, conferences, outreach, etc around the topic?
    • Interactive components outside of those covered under Education?

   

 
  

Back Home
© 1999 - 2010
Public History Resource Center
E-mail Contact editors@publichistory.org
Last updated on January 12, 2004
www.publichistory.org/reviews/rating_system.html