Organization: Full Fact
Applicant: Phoebe Arnold
Assessor: Raymond Joseph
Conclusion and recommendations
Raymond Joseph wrote:
Full Fact is clearly a credible fact-checking organisation with all the necessary protocols and checks and balances and documentation in place. But I would recommend the following minor edits:
3) They state that “We link to all our sources” on their website. But, in fact, do far more, as is clear from the comprehensive and meticulous steps they follow, as set out in their application. (See details under Section 3 of this assessment).
Recommendation: Include an edited version of the details supplied with their application in the FAQs under a “How we source” (or something like that).
5a) Add an “Our fact-checking methodology” or “How we fact-check” section to the FAQs.
NOTE: It would be adequate to include sources and fact-checking methodology under a single entry in FAQs in order to avoid duplication of much of the info which, it seems, would be likely.
5b) Recommendation: On the contact page (and possibly also home page): Add something like: Got a claim for us to fact-check: Contact us via Facebook, Twitter or on team@fullfact.org. I would also recommend that they link to their FAQ page, pointing to the “About fact-checking section”, which makes it very clear what they can, and what they can’t, fact-check.
This would be adequate, but I also suggest pointing them - and others - to how AfricaCheck, which has an excellent method of inviting fact-checks, does it. On their home page they have a “Got a claim for us to fact-check”, which hyperlinks, to this page: https://africacheck.org/how-to-fact-check/submit-a-claim-to-check/
Full Fact pointed out in their application that their “Ask Full Fact” email is dormant because of staff restraints, but they might find the AfricaCheck method useful in the future.
Raymond Joseph recommended Accept with edits
Section 1: Organization
Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
Registered as a charity that does fact-checking and also offers training
done_all 1a marked as Fully compliant by Raymond Joseph.
Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
Recent fact-checks are published under a “most recent” column on the right hand side of the home page (https://fullfact.org/). Older fact-checks can be accessed via tabs along the top of the home page under different categories (Economy, Europe, Health, Crime, Education, Immigration and Law)
For example this is the Health page: https://fullfact.org/health/
Each claim and the conclusion of a fact-check is prominently presented, with the details and evidence below.
They also regularly publish useful guides and explainers on a wide variety of topical issues, like this one on how Family Courts work https://fullfact.org/law/family-court-england/
done_all 1b marked as Fully compliant by Raymond Joseph.
Section 2: Nonpartisanship and Fairness
Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.
Full Fact
21-Mar-2017 (7 years ago)
Full Fact is a registered charity (number 1158683) so we are legally required to be ‘objective, ‘balanced’, and to observe ‘strict political neutrality’. http://apps.charitycommission.gov.uk/Showcharity/RegisterOfCharities/CharityFramework.aspx?RegisteredCharityNumber=1158683&SubsidiaryNumber=0
We monitor media, politicians and pressure groups across the political spectrum.
Our policy is to attempt to contact the claimant about their evidence before writing.
We link to primary sources for our factual assertions.
We seek guidance from neutral external experts on particularly complex or specialist topics.
We operate a two-tier factchecker process so all articles go through two people. All factchecks and updates/corrections are reviewed by a senior researcher before publication and our director if the topic is politically sensitive.
Reviewing is restricted to experienced factcheckers. Our six week training programme covers statistics, impartiality, polling and surveys, making graphs, and good communication. Please see attached our reviewing checklist, which is not exhaustive.
Factchecks have a standardised claim and conclusion intended to guide not dictate readers’ conclusions.
Examples of our factchecking (which covers explainers, roundups, factchecks, videos and live factchecking)
- https://fullfact.org/europe/eu-facts-behind-claims-exports/
- https://fullfact.org/bbcqt/2017/Feb/23
- https://fullfact.org/pmqs/2017/Feb/22
- https://fullfact.org/law/uks-sharia-courts/
- https://fullfact.org/europe/our-eu-membership-fee-55-million/
- https://fullfact.org/immigration/eu-migration-and-uk/
- https://www.youtube.com/watch?v=kcgQJFsz_nA
- https://www.youtube.com/watch?v=5D5WUQBdVCU
- https://www.youtube.com/watch?v=thjKWePg2fk
- Livefactchecking: https://twitter.com/search?f=tweets&vertical=default&q=from%3Afullfact%20%23bbcqt&src=typd
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
From Full Fact’s application: “Full Fact is a registered charity (number 1158683) so we are legally required to be ‘objective, ‘balanced’, and to observe ‘strict political neutrality’. We monitor media, politicians and pressure groups across the political spectrum.
Our policy is to attempt to contact the claimant about their evidence before writing.
We link to primary sources for our factual assertions.
We seek guidance from neutral external experts on particularly complex or specialist topics.
We operate a two-tier fact-checker process so all articles go through two people. All fact-checks and updates/corrections are reviewed by a senior researcher before publication and our director if the topic is politically sensitive.
Reviewing is restricted to experienced fact-checkers. Our six week training programme covers statistics, impartiality, polling and surveys, making graphs, and good communication. Please see attached our reviewing checklist, which is not exhaustive. (Note: Word document attached)
Fact-checks have a standardised claim and conclusion intended to guide not dictate readers’ conclusions.
Examples of our fact-checking (which covers explainers, roundups, fact-checks, videos and live fact-checking)
https://fullfact.org/europe/eu-facts-behind-claims-exports/
https://fullfact.org/bbcqt/2017/Feb/23
https://fullfact.org/pmqs/2017/Feb/22
https://fullfact.org/law/uks-sharia-courts/
https://fullfact.org/europe/our-eu-membership-fee-55-million/
https://fullfact.org/immigration/eu-migration-and-uk/
https://www.youtube.com/watch?v=kcgQJFsz_nA
https://www.youtube.com/watch?v=5D5WUQBdVCU
I’ve also included these (Raymond Joseph, assessor)
BBC fact-checked https://fullfact.org/bbcqt/2017/Mar/02
Police officer numbers have fallen: https://fullfact.org/crime/police-officer-numbers-have-fallen/
Prime Minister’s Questions fact-checked https://fullfact.org/pmqs/2017/Mar/01
Do companies bidding for government contracts have to back Brexit? https://fullfact.org/europe/do-companies-bidding-government-contracts-have-back-brexit/
The number of NHS hospital beds has fallen https://fullfact.org/health/number-nhs-hospital-beds-has-fallen/
Germans are more productive than people in the UK and work less and earn more https://fullfact.org/economy/germans-work-less-earn-more-more-productive-than-uk/
Our prisons are “filling up with foreigners” https://fullfact.org/immigration/foreigners-prison/
Are there record numbers of young people going to university? https://fullfact.org/education/are-there-record-numbers-young-people-going-university/
Are judges deciding whether the UK will leave the EU? https://fullfact.org/law/are-judges-deciding-whether-uk-will-leave-eu/
Black people and justice: viral poster fact-checked https://fullfact.org/crime/black-people-and-justice-viral-poster-factchecked/
done_all 2a marked as Fully compliant by Raymond Joseph.
Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.
Full Fact
21-Mar-2017 (7 years ago)
We have rigorous safeguards in place to ensure our neutrality. These have been carefully constructed based on advice from our board and examples ranging from Amnesty International to the BBC.
We have a cross-party board of trustees, which has no involvement in day-to-day editorial decisions (these are the responsibility of the Director).
We have strict operating guidelines and because of our charity status we operate within a legal framework that obliges us to be non-partisan (see Charity Commission website: http://apps.charitycommission.gov.uk/Showcharity/RegisterOfCharities/CharityFramework.aspx?RegisteredCharityNumber=1158683&SubsidiaryNumber=0)
Our staff handbook states that staff may not reveal publicly how they vote or express support for any political party, or express a view for or against any policy which is currently a matter of party political debate, including on social media. Actively campaigning for a party or seeking nomination as a party candidate in local or national election, or as an independent, is incompatible with working at Full Fact. All staff must complete a declaration of personal interests before they start work here.
Full Fact does not express opinions on the topics we check. From time to time we express opinions on matters that directly affect our work (such as statistics policy, our experience of using the press regulator, or freedom of information policy).
Supporting attachments sent to factchecknet@poynter.org include our operating guidelines, charitable objects, conflicts of interest policy, staff declaration of personal interests form, and relevant parts of our staff handbook.
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
From Full Fact’s application: “We have rigorous safeguards in place to ensure our neutrality. These have been carefully constructed based on advice from our board and examples ranging from Amnesty International to the BBC.
We have a cross-party board of trustees, which has no involvement in day-to-day editorial decisions (these are the responsibility of the Director).
We have strict operating guidelines and because of our charity status we operate within a legal framework that obliges us to be non-partisan (see Charity Commission website: http://apps.charitycommission.gov.uk/Showcharity/RegisterOfCharities/CharityFramework.aspx?RegisteredCharityNumber=1158683&SubsidiaryNumber=0
Our staff handbook states that staff may not reveal publicly how they vote or express support for any political party, or express a view for or against any policy which is currently a matter of party political debate, including on social media. Actively campaigning for a party or seeking nomination as a party candidate in local or national election, or as an independent, is incompatible with working at Full Fact. All staff must complete a declaration of personal interests before they start work here.
Full Fact does not express opinions on the topics we check. From time to time we express opinions on matters that directly affect our work (such as statistics policy, our experience of using the press regulator, or freedom of information policy).
Statement on website on impartiality of Full Fact website: https://fullfact.org/about/impartiality/
Attached to email with assessment:
Operating Guidelines (PDF)
Conflicts of Interest Policy (PDF)
This is from their “Operating Guidelines (PDF attached)
Declaration of Personal Interest (PDF)
This is from their “Operating Guidelines
10. Political Balance
Full Fact’s concern is with issues and claims, not with the people making them. A balanced approach requires us to engage a broad and balanced range of contributors to public debate and not just politicians. However, many of our fact-checks engages with politicians and political parties. Where this is the case, our work will inevitably focus on the largest parties. Nonetheless we seek to maintain Political Balance by ensuring that:
1) No party is ignored by Full Fact which, in the context, has a significant role in a particular debate.
2) The coverage and manner of our work must as far as practical avoid being seen as giving support to any particular party or grouping, and should avoid being seen as unduly focusing on one government department.
3) To the extent that our work will naturally focus on government activity, this must only be because of its policy role rather than party political factors.
4) We treat bodies that are associated with parties with the same care as parties themselves.”
2b) Full Fact also has an “Operating Manual for all staff which covers a variety of issues, including conflicts of interest, declaring personal interests and memberships political parties and other organisations. It covers a wide variety of issues, is written in plain language and is a good example/template for any new fact-checking organisation.
See “Staff handbook” (attached as a Word document)
See Review Checklist (attached as a Word document), which are guidelines used to ensure consistency in Full Fact’s fact-checks.
done_all 2b marked as Fully compliant by Raymond Joseph.
Section 3: Transparency of Sources
Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.
Full Fact
21-Mar-2017 (7 years ago)
We draw on publicly available information, such as statistics or primary research, to assess the validity of claims. For some topics, such as foreign affairs or defence, there is a lack of independent or unclassified sources, so we are unable to factcheck those claims.
We link to primary sources for all factual assertions. We prefer to link to the data tables, legal document, or relevant page of a PDF report — rather than a press release or summary of a statistical release.
We prefer to link to a specific table rather than a directory of tables, where possible (the user interface of some public information websites does not always allow for as granular a link as we would like). For example, NHS Digital gives only the option of linking directly to a downloadable spreadsheet, or to a directory of spreadsheets, some of which are not relevant to a reader seeking to follow our analysis.
We also use anchor links when linking to parliamentary records, where possible.
Please see also https://fullfact.org/finder/. The finder tool empowers people to do independent research round a topic rather than just following links we provide in a factcheck. It provides a guide to the key sources of information and a brief description of some of the variables each data source would be able to provide, so readers don’t have to navigate blind.
Occasionally we seek information /advice from external experts. We will name them and their organisation (if applicable). We normally quote them for their informed judgement and where relevant provide a primary source.
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
From Full Fact’s application: “We draw on publicly available information, such as statistics or primary research, to assess the validity of claims. For some topics, such as foreign affairs or defence, there is a lack of independent or unclassified sources, so we are unable to fact-check those claims.
We link to primary sources for all factual assertions. We prefer to link to the data tables, legal document, or relevant page of a PDF report — rather than a press release or summary of a statistical release.
We prefer to link to a specific table rather than a directory of tables, where possible (the user interface of some public information websites does not always allow for as granular a link as we would like). For example, NHS Digital gives only the option of linking directly to a downloadable spreadsheet, or to a directory of spreadsheets, some of which are not relevant to a reader seeking to follow our analysis.
We also use anchor links when linking to parliamentary records, where possible.
Please see also https://fullfact.org/finder/. The finder tool empowers people to do independent research round a topic rather than just following links we provide in a fact-check. It provides a guide to the key sources of information and a brief description of some of the variables each data source would be able to provide, so readers don’t have to navigate blind.
Occasionally we seek information /advice from We draw on publicly available information, such as statistics or primary research, to assess the validity of claims. For some topics, such as foreign affairs or defence, there is a lack of independent or unclassified sources, so we are unable to fact-check those claims.
We link to primary sources for all factual assertions. We prefer to link to the data tables, legal document, or relevant page of a PDF report — rather than a press release or summary of a statistical release.
We prefer to link to a specific table rather than a directory of tables, where possible (the user interface of some public information websites does not always allow for as granular a link as we would like). For example, NHS Digital gives only the option of linking directly to a downloadable spreadsheet, or to a directory of spreadsheets, some of which are not relevant to a reader seeking to follow our analysis.
We also use anchor links when linking to parliamentary records, where possible.
Please see also https://fullfact.org/finder/. The finder tool empowers people to do independent research round a topic rather than just following links we provide in a fact-check. It provides a guide to the key sources of information and a brief description of some of the variables each data source would be able to provide, so readers don’t have to navigate blind.
Occasionally we seek information /advice from external experts. We will name them and their organisation (if applicable). We normally quote them for their informed judgment and where relevant provide a primary source. We draw on publicly available information, such as statistics or primary research, to assess the validity of claims. For some topics, such as foreign affairs or defence, there is a lack of independent or unclassified sources, so we are unable to fact-check those claims.”
done_all 3a marked as Fully compliant by Raymond Joseph.
Section 4: Transparency of Funding & Organization
Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
Funding: Full Fact is completely transparent in its sources of funding and listing all donations over £5000 It also seeks public donations:
https://fullfact.org/about/funding/
Financial accounts
Annual Report and Financial Statements for the Year Ended 31 December
2015
http://apps.charitycommission.gov.uk/Accounts/Ends83/0001158683_AC_20151231_E_C.pdf
done_all 4a marked as Fully compliant by Raymond Joseph.
Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
The team: https://fullfact.org/about/our-team/
done_all 4b marked as Fully compliant by Raymond Joseph.
Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
Contact details: https://fullfact.org/about/contact/
done_all 4c marked as Fully compliant by Raymond Joseph.
Section 5: Transparency of Methodology
Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.
Full Fact
21-Mar-2017 (7 years ago)
Please see
- "What kind of thing do you check" and "How do you live-factcheck" in FAQs: https://fullfact.org/about/frequently-asked-questions/
- and our blog: https://fullfact.org/blog/2016/oct/how-find-fact/
- and https://fullfact.org/blog/2016/oct/how-find-fact-disappearing-asylum-seekers/
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
They state on their website: "We provide links to all sources so that you can check what we’ve said for yourself." (on this page: https://fullfact.org/about/impartiality/)
See recommendation in “Conclusion and recommendations”
"What kind of thing do you check" and “"How do you live fact-check" in FAQs: https://fullfact.org/about/frequently-asked-questions/
And these two blog
2. https://fullfact.org/blog/2016/oct/how-find-fact/
3. https://fullfact.org/blog/2016/oct/how-find-fact-disappearing-asylum-seekers/
done_all 5a marked as Fully compliant by Raymond Joseph.
Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.
Full Fact
21-Mar-2017 (7 years ago)
People can contact us on Twitter or Facebook and by email (team@fullfact.org). See also https://fullfact.org/ask/ (currently dormant as we do not have enough staff to service demand fairly)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
From Full Fact application: “People can contact us on Twitter or Facebook and by email (team@fullfact.org). See also https://fullfact.org/ask/ (currently dormant as we do not have enough staff to service demand fairly)” See also contact page (https://fullfact.org/about/contact/) (See recommendation)
done 5b marked as Partially compliant by Raymond Joseph.
Section 6: Open & Honest Corrections Policy
Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
In their FAQs (https://fullfact.org/about/frequently-asked-questions/) they have a “How can I make a complaint to Full Fact” section, which links to”
done_all 6a marked as Fully compliant by Raymond Joseph.
Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.
Full Fact
21-Mar-2017 (7 years ago)
Raymond Joseph Assessor
21-Mar-2017 (7 years ago)
Examples correction:
Corrections, below, published on Twitter. And, following the link from Twitter leads to the original fact-check with details of the correction at the end.