Evaluating the Reliability and Scientific Status of a Publication
Online sources
An internet search produces all kinds of results. Many of them are not scientifically reliable, even if they might look like it.
When evaluating the reliability of web-based sources, it is important to consider the following:
- Author: Is the author a respected researcher in the field, or is the organisation responsible for the publication considered to be of high scientific standard? What is the author’s affiliation?
- Publisher: What is the scientific standing of the publication channel, such as the journal in which the article was published?
- Publication type: Is the source a scientific article, a book published by an academic publisher, a report produced by an expert organisation, a newspaper article, a blog post, or something else?
- Quality: How accurate is the information presented? Is it supported by other sources, such as scientific articles or textbooks? Does the publication cite the sources on which its claims are based?
- Purpose: What is the motivation behind the publication (e.g., commercial, ideological, or other interests)? Is the information presented objectively, or is it intended to persuade or promote a particular viewpoint?
- Currency: Is the content up to date? Does the source provide a publication date or a date of the latest update? Are the links on the website functional?
Reliable internet sources include well-known expert organisations, reputable research institutions, and peer-reviewed publications, such as scientific online journals and their articles. Assessing the credibility of sources found through search engines can be challenging; therefore, the easiest way to find scientific information is often to use scientific databases.
Searches on the web often pick up theses and dissertations as well. Theses, such as Master’s theses and theses completed at universities of applied sciences, are generally not considered citable sources in scholarly publications, with the exception of doctoral dissertations. However, practices vary among disciplines. It is therefore advisable to verify the practises of your own discipline with your thesis supervisor, in research seminars, or through specific guidelines if such guidance is available.
Scientific journals and journal articles
The scientific quality and reliability of an article that appears scholarly can be assessed by finding out, for example:
- Is the article peer reviewed?
- Does the journal have a JUFO (Publication Forum) classification?
- Can the article’s bibliographic information be found in reliable databases?
- Has the article been cited by other publications?
- Does the journal have an Impact Factor?
Scientific articles are generally peer reviewed. Information about a journal’s and an article’s peer-review status can often be found on the journal’s website. In databases, searches can often be limited to scholarly publication types (e.g., article, review, randomised controlled trial). Some databases explicitly indicate whether a journal is ‘peer reviewed’ or ‘refereed’. Notes appearing in articles (such as ‘revised’) may also suggest that the article has undergone a peer-review process. For example, in UEF Primo, search results can be filtered to include only peer-reviewed materials.
It is worth remembering that scientific journals also publish content that has not been peer reviewed. Examples include editorials, news items, book reviews, and commentaries on published articles. In many databases, these publication types can be excluded from search results.
Publication Forum (JUFO) is a Finnish classification system covering all academic disciplines and evaluating different types of scholarly publication channels, including journals, conferences, and book publishers. A publication channel classified at level 1 meets the basic criteria of scholarly publishing.
JUFO Portal (Publication Forum)
Articles and journals are also evaluated based on citation counts. In general, the more frequently an article is cited in other research publications, the more impactful it is considered within the scientific community. Naturally, newer articles have had less time to accumulate citations, and some articles may never be cited.
A journal’s Impact Factor (IF) indicates the average number of citations received by articles published in that journal. A high Impact Factor generally suggests that the journal – and often the articles it publishes – is widely read and influential. However, citation-based metrics should always be interpreted critically.
Impact Factor (Guide to Information Retrieval > Library Terminology)
Research evaluation (UEF Library)
Scholarly books
Quality control of scholarly books is carried out by publishers, editors, and reviewers involved in the publication process. The Publication Forum (Julkaisufoorumi) can also be used to evaluate the scholarly status of publishers. Scholarly publishers include commercial academic publishers, universities, national and international academic societies, and certain research institutes.
Other sources
In many fields, a substantial amount of material is published for professionals, companies, and public authorities. Such publications often do not meet the criteria of scholarly publishing. Ministries, municipalities, and various organisations, associations, and professional bodies publish reports, surveys, and other forms of grey literature. Professional and trade journals also publish articles intended primarily for practitioners.
In academic writing, it is often recommended to stay critical when citing trade and professional magazines, publications issued by associations, newspapers, Master’s theses, and theses produced at universities of applied sciences.