Dr. Marcus Jäger, editor-in-chief of Orthopedic Reviews, which has failed to retract a fatally flawed paper.
Part of the science publishing community seems to have developed a casual relationship with fraud and false information. Take the case of Orthopedic Reviews, an online-only, peer-reviewed journal.
Orthopedic Reviews has refused to correct or retract a false paper it published on CrossFit trainers last year. We contacted their editorial board by email and through certified mail. Our letter informed them of the many problems with their study. They did not respond.
The false Orthopedic Reviews paper is “CrossFit® Instructor Demographics and Practice Trends” by Gregory R. Waryasz et al. It misrepresents cited research and conflates the varying levels of CrossFit credentials. The paper is ultimately so ambiguous as to be truly meaningless.
Here is the full text we sent Orthopedic Reviews on Feb. 9, 2017:
“Hello,
I am writing to report major errors in an article recently published by your journal, “CrossFit® Instructor Demographics and Practice Trends,” by Gregory R. Waryasz, Vladimir Suric, Alan H. Daniels, Joseph A. Gil and Craig P. Eberson.
First, I would like to request that your journal retract the article and issue a correction given the severity of the following errors. For example, it states,
“There is evidence that fatigue can lead to suboptimal changes in technique seen in high repetition sets. These changes lead to a smaller knee flexion angle and a larger hip flexion angle during a barbell back squat,11 but this has not been shown in the literature to result in increased injury.”
This is nearly the opposite of what “Effects of fatigue from resistance training on barbell back squat biomechanics” by Hooper et al., the source cited in endnote 11, found:
“At the early stages of the protocol, knee angle was significantly lower in men and in women demonstrating less knee flexion. Also, hip angle was significantly lower early in the program in men and in women, demonstrating a greater forward lean.”
Obviously fatigue was higher towards the end of the workout than at the beginning, yet the subjects bent their knees more and leaned forward less at the end of the workout. As Hooper’s study notes,
“This study was also able to demonstrate that changes in proprioception are not solely dependent on the level of fatigue but that altered movement patterns are the result of multiple factors, and the negative effects of fatigue have the potential to be overcome. For example, the changes in movement that occurred at the hip and knee did not significantly correlate with any indicator of fatigue (Tables 2 and 3). Further, as the subjects moved toward the end of the protocol, both men and women increased knee flexion during the course of the workout. This is in contrast to findings seen during box lifting (8,17) and a body weight squat performed after an identical resistance training protocol (9). This favorable change in technique occurred despite an immediate postblood draw, demonstrating highly elevated blood lactate concentrations…”
Here is the graphic that illustrates this fact:
Back to Waryasz’s paper. It states,
“There were no statistically significant differences in other forms of education or certifications. CrossFit® instructors with a bachelor’s degree made up a higher percentage of non-CrossFit® certified (P=0.04) compared to instructors without a bachelor’s degree (Supplementary Table S1).”
Unfortunately, it is not possible to verify this claim because Supplementary Table S1 is missing (from both the PDF and HTML versions available on PubMed and your journal’s website: 1,2,3,4).
Waryasz also claims,
“We did not differentiate between CrossFit® Level 1 to 4 training certification but rather just asked if the instructor was certified or not. There could be differences between these credentials, but we opted not to investigate these in this study.”
CrossFit’s L1 is not a “certification” but a certificate course. In contrast, the CrossFit L3 is a “certification,” according to the ISO 17024 standard as verified by the American National Standards Institute that accredits it. Nor is CrossFit’s L2 course a “certification.” It is highly misleading and inaccurate to label the L1 and L2 as certifications. In fact, the authors respect the distinction themselves elsewhere, as they note:
“Currently, there is a CrossFit® Level 1 Trainer Certificate Course, Level 2 Certificate Course, Certified CrossFit® Level 3 Trainer Certification (CF-L3), CF-L4, and Certified CrossFit® Trainer.”
Every attendant to CrossFit’s L1 course is notified of the distinction between a certificate and a certification (pp. 124-125, CrossFit Level 1 Training Guide). This distinction is included in the test as well. Hence, it is reasonable to expect that CrossFit L1 certificate holders are aware that they are not in fact “certified” by CrossFit.
Waryasz’s conflation between certifications and a certificate course critically impacts another claim in the paper:
“A CrossFit® instructor certification was held by 86.6% of survey participants, the survey did not specify what level of certification or what CrossFit® certification the participant had obtained [sic].”
This is very confusing given the subject of the article: “CrossFit ® instructors.” Did the 13.3 percent of “CrossFit instructors” who said they were not certified by CrossFit say so given that they were CrossFit L1 certificate holders but not certified through the CrossFit L3? In this case, if we for the sake of argument accept the article’s false premise that the L1 is a certification, then these subjects were indeed “certified” by CrossFit.
Or were these subjects misled by the authors’ false portrayal of the L1 as a certification? Did these subjects in fact mean that they had not even acquired the L1 certificate? In that case, if they were neither L1 certificate holders nor certified by CrossFit, then they did not qualify as “CrossFit instructors” at all since the CrossFit L1 certificate is the minimum prerequisite to legally instruct CrossFit classes. And that would render the study’s title and topic both false.
Either way, the study makes a major disqualifying error in failing to accurately or clearly distinguish between what it describes as “certified” and “non-certified” instructors. It’s not possible to verify what the respondents thought it meant to be CrossFit certified, nor to verify whether their answers were accurate.
And this error has catastrophic implications for the remainder of the article. The abstract describes the major findings in the following terms:
“Instructors with a CrossFit® certification have less bachelor’s (P=0.04) or master’s (P=0.0001) degrees compared to those without a CrossFit® certification, more utilization of Olympic weightlifting (P=0.03), one-on-one teaching (P=0.0001), 1-RM max on snatch (P=0.004), 1-RM on clean and jerk or hang clean (P=0.0003), kettlebell use (P=0.0001) and one-on-one training (P=0.0001).”
Given the article’s failure to accurately and clearly distinguish between certified and non-certified CrossFit instructors, it’s not possible to make any sense of this data.
Thus, the article’s main conclusions, the ones highlighted in the abstract, are entirely invalid.
I would again urge you to inspect the article’s major errors, highlighted here, retract the article, and issue a public correction. Thank you for your consideration.
Russ Greene
CrossFit HQ”
We still await their response.
