Evaluating source quality
All sources are not created equal. People write for many reasons – to inform, to entertain, to persuade, to describe, to mislead, to satirize – and the quality of information depends on this purpose. In an academic assignment, it is important to use evidence that is reliable, accurate, objective, and up-to-date.
Rauru Whakarare Evaluation Framework
The Rauru Whakarare Evaluation Framework can be used to evaluate source quality. To evaluate sources effectively, you need to consider a number of factors. Not every source will meet all of the quality indicators, so you need to think about what's important in your information search contexts.
The Rauru Whakarare Evaluation Framework provides you with a holistic Māori-informed view of information evaluation as you find and select information for a range of purposes.
It has five key concepts to consider as you evaluate an information source:
- Whakapapa is the central concept that asks you to consider the background of the source. All the other concepts connect to whakapapa.
- Orokohanga connects to the origins of the information source, considering where it has come from;
- Mana identifies source authority and credibility and considers how the source connects to other research through acknowledgement of others;
- Māramatanga relates to content and its relevance and whether the source informs your own knowledge and extends your learning;
- Aronga considers the purpose of the information and the author’s lens or perspective. It also relates to how you perceive the information as well.
You can apply this framework to any information sources you find online or from any context where information is kept. You can download a copy of the framework with detailed descriptors to help you unpack each of these concepts below.
- Rauru Whakarare Evaluation Framework
- Rauru Whakarare Evaluation Framework 2 sided print version
- Rauru Whakarare explanation video
Personal experience
Many first-time students rely only on their own personal experience when writing an assignment (your Aronga/Lens). While it is important to provide your own perspective and interpretations, there are some risks in depending solely on experience to further your argument.
- Overgeneralisation: we tend to think of our experiences as universal, but people in different demographics, countries, times, and situations may have significantly different perspectives.
- Hearsay: many pieces of ‘common knowledge’ are not based on fact, but on rumour (see, for example, the myth that humans use only 10 percent of their brains).
Outside sources, especially academic sources, consider multiple sources of information together to build a bigger picture of the topic. They rely on evidence, logic, and research to establish their points. For these reasons, outside sources are usually considered more valuable for the purposes of assignment writing (they have Mana/Authority to speak to this topic).
It is still possible (and, often, important) to present your own position on a topic, but that position should be supported by evidence from other sources wherever possible (this will enhance your Mana as a graduate).
Note: the amount of personal experience that can be used in an assignment varies from discipline to discipline, and assignment to assignment. Reflective writing, for example, emphasises personal experience. If unsure, ask your course coordinator.
Printed sources: books and articles
The quality of a printed source can vary widely. A popular magazine article does not have the same burden of evidence and thoroughness as an article published in an academic journal. There are a number of clues that can be used to determine the value of a printed source (carefully consider both Māramantanga/Content and Orokohanga/Origins).
- Sources that check their facts are better. Look for footnotes, a list of references, or other evidence that it has been well researched.
- Sources with systematic quality controls are better. Look for an editor, editorial board, or other evidence of peer review.
- Up-to-date sources are better. What could have changed about the topic since its publication?
- Consistency with existing knowledge is important. If a source contradicts what you already know or have already read, examine its evidence carefully.
In university assignments, academic sources are the most consistently useful. See identifying academic sources for more on this topic.
The process of evaluating the quality and usefulness of a piece of writing is known as critical reading: see critical reading for more on this topic.
Online sources
Most printed sources will have at least some editorial process: before a book or article is published, it is checked for factual accuracy. Online, anyone can ‘publish’, but only some sources have the same quality controls as printed sources. Therefore, it is essential to take greater care and apply criteria, like the Rauru Whakarare concepts, to determine the quality and relevance of online information.
There are many sources of the highest academic quality online. However, there are also personal pages, commercial websites, satires, and other sources that lack the quality to inform your knowledge for your university assignments. Online, you must take more responsibility to separate academic sources from other types of website.
Many of the same questions about printed sources can be applied to online sources:
- Does the page have footnotes, a list of references, or other evidence that it has been well researched? (Whakapapa/Background)
- Who reviews the page to ensure its accuracy? Look for evidence of peer review. (Whakapapa/Background)
- Is the source up-to-date? Look for a copyright date, or an indication of when the page was last updated. (Orokohanga/Origins)
- Does the page make unusual or extravagant claims? Extraordinary claims demand rigorous evidence to support them. (Aronga/Lens; Māramatanga/Content)
There are also some shortcuts that can help you to determine whether a website is likely to be useful or not:
- What type of website is this? Look at the URL address. (Orokohanga/Origins)
- .com or .co.nz: A commercial site. This may carry useful information, but be wary of bias. (Aronga/Lens)
- .org or .org.nz: An organisation, often a non-profit organisation or professional association. Again, there may be useful information, but bias towards the organisation's objectives is likely. (Mana/Authority; Aronga/Lens)
- .gov or .govt.nz: A government site. These sites are usually informative, and many are high quality, but they can also have limited scope. (Mana/Authority; Aronga/Lens; Māramatanga/Content)
- .edu or .ac.nz: An academic site. These sites are likely to be very useful and usually target an academic audience. Personal pages are sometimes hosted on academic sites, however, and their quality can vary. (Mana/Authority; Aronga/Lens)
- Does the website look professional? Websites with many advertisements and pop-up windows, and websites with only text on a plain background, are less likely to be sources of quality information. (Mana/Authority, accuracy)
- Does the website identify an author? Uncredited information is of dubious value. (Mana/Authority)
When searching online, don't restrict yourself to basic web searches. You are more likely to find high quality information through specialised searches, such as Google Scholar, Google Books, and Massey Library's article databases.