Sadly, it appears that in some cases the answer is "no".
With this I don't mean that the peer review process itself is flawed, but that the process is not being applied rigorously enough, before drawing conclusions (and, for example, making sociopolitical and legal decisions).
Science is supposed to be reliable, and one of the main reasons for this is the peer review process: No claim or conclusion is accepted unless it has gone through the rigorous test of scientific peer review, where numerous independent parties verify that the scientific paper in question does not contain flaws, errors, omissions or wrong conclusions using wrong logic, and that any experiments and results are repeatable, and have been repeated numerous times.
Unfortunately, sometimes this process is skipped, especially when there are sociopolitical forces and ideologies pushing for a certain idea or agenda, and very flawed papers are too often taken too seriously, ignoring their flaws. Moreover, when other scientists do engage in the peer review process and point out the flaws in the paper, they tend to be ignored.
This skipping-the-peer-review-process in order to push some kind of agenda is most often seen when it comes to hot topics related to products related to food, agriculture, farming, people's health, and so on. Examples of such topics include genetic modification of plants, certain pesticides, certain medical drugs, and so on. When a published paper seems to fuel the controversy and support a certain agenda, it's often taken as proof without even waiting for the peer review process.
Most typical flaws in such papers include things like:
With this I don't mean that the peer review process itself is flawed, but that the process is not being applied rigorously enough, before drawing conclusions (and, for example, making sociopolitical and legal decisions).
Science is supposed to be reliable, and one of the main reasons for this is the peer review process: No claim or conclusion is accepted unless it has gone through the rigorous test of scientific peer review, where numerous independent parties verify that the scientific paper in question does not contain flaws, errors, omissions or wrong conclusions using wrong logic, and that any experiments and results are repeatable, and have been repeated numerous times.
Unfortunately, sometimes this process is skipped, especially when there are sociopolitical forces and ideologies pushing for a certain idea or agenda, and very flawed papers are too often taken too seriously, ignoring their flaws. Moreover, when other scientists do engage in the peer review process and point out the flaws in the paper, they tend to be ignored.
This skipping-the-peer-review-process in order to push some kind of agenda is most often seen when it comes to hot topics related to products related to food, agriculture, farming, people's health, and so on. Examples of such topics include genetic modification of plants, certain pesticides, certain medical drugs, and so on. When a published paper seems to fuel the controversy and support a certain agenda, it's often taken as proof without even waiting for the peer review process.
Most typical flaws in such papers include things like:
- Too small of a sample size.
- Too few tests.
- No controls, or poor controls. (In the worst examples that exist out there, controls showed the experiment as flawed, but the result of the control samples were discarded and ignored willy-nilly with a handwave!)
- Testing the wrong thing, or other errors in the testing procedure.
- Selection and confirmation bias.
- Citing other papers that are likewise flawed and haven't passed peer review either.
- Making conclusions that cannot be drawn from the results (ie. the numbers don't actually back up the conclusion being made).
Comments
Post a Comment