Fbae Logo
Home | | Support Us | Contact Us
Goals & Objectives Our Position False Propaganda Special Topics Important Publications Important Links Events news Biosafety
Fbae Header Home

NEWS

 

 

 
Peer Review Under the Spotlight


What matters most is the lack of public scrutiny rather than the lack of peer review in times of corporate corruption of science Prof. Peter Saunders

You've probably come across the expression "peer-reviewed" a lot recently, especially in discussions on GM food, mobile phones, or organic farming. It's almost always used as part of a sentence that begins "There is no peer-reviewed evidence for …" or "There is nothing about this in any peer-reviewed journal …"

What you're meant to understand by that is: "there is no credible evidence for whatever it is, and you can safely ignore anything you've heard about it." When the question is about safety, as it often is, it means the regulatory authorities are not going to look into it.

A lot of people take peer review very seriously, or at least they say they do. When Sir David King, then the Chief Scientific Adviser to the British government, put forward a code of ethics for scientists, one of his chief examples of unethical behaviour, right up there with plagiarism, was "disseminating work before it has been peer reviewed". You may remember Arpad Pusztai who spoke for 150 seconds in a television programme on unpublished results indicating that genetically modified (GM) potatoes were harmful to rats, because he saw it his duty to warn the public. He was subjected to fierce attacks from the scientific establishment (led by the Royal Society) that continue to the present day.

The scientific establishment's double standard

The scientific establishment may claim to oppose disseminating results that have not been peer-reviewed, but there is a blatant double standard being applied, and all too often. Just recently, a UK government research funding agency, the Economic and Social Research Council (ESRC) put out a press release that was not only highly misleading about farmers being upbeat about GM crops ("UK Farmers Upbeat about GM Crops" Debunked and Marketing Masquerading as Scientific Survey , SiS 38); but was also based on research that had not been peer-reviewed, according to the ESRC's own web site.

Another recent example came from the top mainstream journal Nature Biotechnology. In an editorial, it criticised the Italian National Research Institution for Food and Nutrition (INRAN) for not publishing some results that were allegedly favourable to GM crops.

The director of the project in question, Giovanni Monasatra, wrote to the journal to put the story straight, and his letter was published along with a response from the editor, Andrew Marshall. In his letter,  Monastra dealt with the points raised in the editorial and expressed his surprise that Marshall, far from criticising Salute, Agricolura, Ricerca (SAGRI) for organising a press conference to publicise data that were, according to Marshall, "too preliminary for peer-reviewed publication," instead complained that the Italian media did not give it even more coverage than they did.

Marshall's response is that the data had [his italics] to be press released by SAGRI because they were of interest to the public and political debate. Yet, in 1999, Marshall had written in Nature Biotechnology that Arpad Pusztai's work should be submitted for peer review before it could be considered, even when safety was at stake.  The difference is that he was then writing about results that were against the interests of the biotech industry.

It is not at all unusual for scientific bodies and lobby groups to issue press releases and hold press conferences on non-reviewed material.  The people who set themselves up as the guardians of sound science either say nothing or even join in, except when it is a matter of things the corporations don’t want the public to hear. In that case, they suddenly rediscover their strong objection to the practice.

Peer review is a useful part of the scientific process, but it is not as effective, as important, or as universal as some would have us believe, and it needs to be put in perspective.

What is peer review?

One of the distinguishing features of science is that when you discover something, you don’t expect other people just to take your word for it. You’re expected to describe exactly what you have done and why this justifies what you claim, and the usual way of doing this is to publish a paper in a scientific journal.

When you submit your paper to a journal, the editor will generally send it to be reviewed by experts in the field. The referees, usually two or three, are supposed to read the manuscript carefully and assure themselves as best they can that the work was done using appropriate techniques, that it takes into account and properly acknowledges earlier relevant work, and that the conclusions are properly derived from the data, or, in the case of a theoretical paper, that the arguments are sound. They advise on whether the work is interesting enough and contains enough that is new to be worth publishing, and, even if it is, whether they consider it is suitable for the particular journal. They may also suggest ways in which the paper could be improved.

This peer review system is important in science. It prevents many very poor papers from being published and it improves many others. Above all, it helps maintain a consensus of what is expected in a scientific paper; what you find in a scientific journal is very different not only from the popular press but even from most papers in the humanities or social sciences.

But peer review is very limited in what it can do. Referees, who are not paid, vary considerably in the time and effort they devote to the task. They are all too likely to nod a paper through if it looks plausible and comes from a lab or a group that they know and trust, or to reject one because they disagree with it or don’t understand it and haven’t the time or inclination to go through it carefully. They may reject a paper as “not interesting” when what they mean is that it’s not the sort of thing they and their friends are interested in.

Referees do not go into the laboratory to watch the experiments being carried out. They do not have access to the authors’ notes and raw data. They make their decisions on the basis of nothing more than what the reader will see if the paper is accepted. Even the most conscientious are simply not in a position to guarantee that the results are correct or even that the work was done properly.

Peer review could certainly be improved, and there are a number of ways in which this might be done. For example, research has confirmed the suspicion of many scientists that there is often bias, whether conscious or not, and it has been suggested that referees should not be told the authors’ names or institutions, or even their gender. But while the system could be improved, it is hard to see how it could do much more, even if we really want it to. The real test of a paper comes after it is published and is open for comment by the whole of the scientific community, and that can be a far more stringent test. Indeed, many poor or outright fraudulent works have been exposed after they were in print. One recent example is a paper published in the British Food Journal and given an Award for Excellence, which provoked 40 scientists and two MPs to sign an open letter demanding its retraction (Wormy Corn Paper Must be Retracted, SiS 37). We shouldn’t expect peer review to do more than it actually can, and by the same token we shouldn’t claim to the public that it does.

Not all science is peer reviewed

If all real science were peer reviewed before it was made public or used in decision making, then you might want to say that peer review, rightly or wrongly, defines real science. In fact, there are a number of ways in which science often gets into circulation without peer review. For example, the work can be published in the proceedings of scientific meetings or, especially in rapidly moving subjects like theoretical physics, circulated as a preprint or posted on a web server.

On the whole, exceptions like those don’t matter much. Most of the work will eventually be refereed or else just forgotten. In any case, it is out in the open for anyone to see and criticise.

The absence of peer-review matters in regulation but not as much as the absence of public scrutiny There is, however, an area in which the absence of peer review matters a great deal, and that is in regulation. Many products, including pharmaceuticals and GM foods, have to be licensed. The manufacturers are required to carry out trials, safety tests and risk assessments and submit the results to the regulators.

Much of this work is never peer reviewed. What is more, much of it is never published, and worse, concealed from the public and often from the regulators as well under claims of “commercial confidentiality”, so that other scientists are never able to comment on it.

Companies use commercial confidentiality in much the same way that the UK government uses the Official Secrets Act, less as a means of keeping sensitive information from a possible competitor than to ensure that nothing embarrassing reaches the public. And as with the Official Secrets Act, the claim that something must be kept confidential on commercial grounds is seldom challenged. Just to quote one example, even after the TGN1412 trial went so disastrously wrong (see Post Mortem on the TGN1412 Disaster, SiS 30), and it was abundantly clear that the drug would never be developed further, the Medicines and Healthcare Products Regulatory Agency (MHRA) still refused to release some details of the test protocol on the grounds of commercial confidentiality. It is hard to imagine how a competitor of either Te Genero, the company that developed the drug, or Parexel, the company that ran the trials, could have gained any unfair advantage from the information, though it might have been of use to anyone trying to improve the safety of trials, and more importantly, to provide effective remedy for the victims.

The absence of peer review is nowhere near as important as the absence of public scrutiny. To make claims on the basis of research that you will not reveal to the scientific community is to go against one of the basic principles of science: that we provide evidence for the claims we make. It is all the more serious when these claims can affect health and the environment. Such data should never be held secret on grounds of commercial confidentiality.

Conclusion

Peer review is a useful part of science but it is not and cannot be the dividing line between good science and bad, between what can be relied upon and what must be dismissed out of hand. In particular, when we are told that there is no peer-reviewed evidence,  that does not mean that there is no evidence, nor does it mean there is no credible evidence. The scientific establishment has been deliberately applying a double standard to exclude evidence unfavourable to industry. What is worse, our regulators have accepted all kinds of evidence in approving new products and processes not just without peer review, but without the possibility of scrutiny by the public or even by the regulators themselves.

A fully referenced version of this article is posted on ISIS members' website.

An electronic version of this report, or any other ISIS report, with full references, can be sent to you via e-mail for a donation of GBP3.50.

Please e-mail the title of the report to:
report@i-sis.org.uk

 

Related News Articles
 

Bt-corn does not harm biodiversity

Countering insect resistance with designer Bt toxins

ICGEB receives grant from the Bill & Melinda Gates Foundation to strengthen and expand biosafety systems in sub-Saharan Africa

Policy on the transfer of Genetically Improved Farmed Tilapia (GIFT) from Asia to Africa by the WorldFish Center

Rules on marketing GM produce face review

EU ministers to debate Bayer's GM cotton, soybeans

EU's legal labyrinth of GMO legislation

BRAZIL APPROVES SYNGENTA'S INSECT-RESISTANT Bt11

The latest issue of Plant Physiology (July 2008; Volume 147, Issue 3) has a special section on next generation of biotech crops especially on nutritional improvement.  These papers can
be downloaded free!

Influence of Transgenosis on the Plant-Insect- Relationships, in Particular on Chemically       Mediated Interactions

Effect of Transgenes Conferring Enhanced Pathogen Resistance on the Interaction with Symbiotic        Fungi in Rice

Impact on the Soil Ecosystem through Natural and Genetically Engineered Organisms:
      Effects, Methods and Definition of Damage as Contribution to Risk Assessment

The Decomposition of Bt-Corn on the Fields and its Impact on Earthworms and on other        Macroorganisms in the Soil

Environmental Post-market Monitoring of Bt-maize:
       Approaches to Detect Potential Effects on Butterflies and Natural Enemies

Columns by Dan Gardner

Against the Grains: 'The Terminator Hoax '

Decisions taken in the 84th Meeting of the Genetic Engineering Approval Committee

Brazilian Health Biotech: Fostering Crosstalk Between Public and Private Sectors

Biotechnology Related Article Appeared on 'Samyukta Karnataka' ( Regional Language )
S.B.Sulia
June 12, 2008.

Nothing Left to the Imagination

The Politics of GM Food
Kirit S Javali

Hi-tech seed factories: Sowing Seeds of Success

"Indian Seed Industry is Well Placed to Serve Both Domestic and International Markets"
Dr MK Sharma,
Managing Director,
Mahyco Monsanto

"If we Facilitate Seed Industry, we Facilitate Growth in Agriculture"
Dr Govind Garg,
Director,
R&D,
Krishidhan Seeds

Metagenomics: Window to the Microbial Universe

Few Checks to Prevent Entry of GM Food

Gene Campaign Criticises India’s ‘Silence’ at Global Bio-Safety Meet

An Enforceable International Compact for Infectious Diseases

"Indian Science in Genomics has been Able to Place Itself on the Global Map"

Indian Gene Decoded

The Development of RNAi as a Therapeutic Strategy

FAO E-Conference on Biotechnologies and Water Scarcity

Genetic Landscape

Biotechnology in Food and Agriculture

RH Nature Reviews Genetics 08- Opposition to Transgenic Technologies


Germany: Discussion Paper of German Ag-Industry about EU Biotech Policy Implications


Bt maize performance in Spain

Arsenic speciation varies with type of rice

Why I Am Bothered by Neo-Colonialist NGOs

China experts identify gene for yield, height in rice

The French government has called for a debate on the review of the EU
The World Trade Organisation (WTO) has also repeatedly criticised the EU for "undue delays" in the authorisation of GMOs. See the latest WTO ruling:

The legal bans are in France, Austria, Poland, Hungary and Greece.

EU delays decision on approving more GM crops

UCR Geneticist Plays Scientific Advisor to Movie about “Love, Adventure and ... Genetically Modified Rice”

Gujrat worst-hit by illegal Bt cotton production

Farmers seek ban on GM crops

Call for policing
Ijaz Ahmed Rao discusses the virtues of a bio-safety framework for genetically modified crops, now that they have become farmers’ favourite


Stem cells: The 3-billion-dollar question


Genes as the solution

Food crisis spurs research spending


Global Food Crisis / UN / Bilingual Transcript of Statements by Secretary-General, Heads of Concerned Agencies, and Response to Questions at Press Conference on Global Food CrisisGM Crops, A World View

Mass Protests against GM Crops in IndiaInterference at the EPA

Open letter to Robert B. Zoellick, President, World BankNew BT variety may push short staple cotton output.

The future of agricultural biotechnology: Creative, destruction, adoption, or irrelevance? ICABR Conference 2008

Soaring food prices and global grain shortages are bringing new pressures on governments, food companies and consumers to relax their longstanding resistance to genetically engineered crops.


Prof. Kameswara Rao and Dr. T.M. Manjunath's Participation in 2008 Biotech Activities

Scrutinizing Industry-Funded Science: The Crusade Against Conflicts of Interest

LEADER: Nurturing nanotech


Center for Indigenous Knowledge for Agriculture and Rural Development


Scientists find potential schistosomiasis treatment

Islamic conference boosts S&T with new resolutions

Mexico publishes GM approval guidelines


Uganda 'close to stamping out Hib meningitis'


New method 'prevents spread of GM plants'


Social factors 'help women with post-tsunami stress'


Women scientists celebrated in new charter


Sub-Saharan Africa news in brief: 13–25 March

Brazil creates US$18 million fund for young scientists


Health weeks 'powerful tools' for deworming children


Rotavirus vaccine, not treatment, 'cheaper for Panama'