Supercourse Newsletter, August 10, 2004
---------------------------------------
End of Summer
We are nearing the end of summer, pulling out, and dusting off
our lectures for the up coming year. We look forward to you all
contributing a lecture or two to the Supercourse.
Quality Control:
A question we are always asked about the Supercourse is one of
peer review/Quality control. Over, and over, and over we are
asked this question.
We just completed a paper, which was submitted to the BMJ. If
you would like a copy of this, we are happy to send it to you.
The title of the paper is: Ho: Scientific Journals are Faith
Based. Obviously it is a rather strange title.
The major points of the paper are that journals have survived
relatively intact for 300 years, whereas all other forms of sci-
entific technology have radically changed. There have been over
500,000,000 articles published during this time. There have been
only 13 articles scientifically evaluating the Introduction,
Methods, Results and Discussion model, 19 articles on Peer re-
view, and 14 articles on editorial decision system. Tom Jeffer-
son wrote a wonderful article on evidenced based peer review in
JAMA, and he could find little evidence that peer review worked.
(Tom was the first international contributor to the Super-
course). Therefore because of the paltry number of research ar-
ticle on the journal process itself, journals have not evolved.
We argue that the journal process is based upon the scientific
method, but rather based upon "faith" with no facts.
With web lectures it is even worse. There are 5.5 million .ppt
files on the web, with no standardized approach for quality con-
trol.
Our approach is one of continuous quality improvement (CQI) us-
ing a system established in industry by Deming and others. The
idea being is that at every stage of the quality control process
we need to test the process to find continuing better systems.
We have 5 methods of evaluation. The main approach is one based
upon Amazon.com, Consumer reports, and Slashdot.com, where at
the end of the lecture the readers rate the lecture according to
various parameters. Faina Linkov just submitted a paper on our
approaches, if you would like a copy, please write to
mailto:fyl1@Pitt.edu
Faina will be completing her dissertation on QC and Web lectures
in a fascinating project. She will be having 3 experts review
100 lectures each. We want to compare the expert review with the
Amazon type review as well as other measures of quality. We
would love to have you as reviewers so we can examine the within
compared to between person variability of the quality control
process, as well as the interrelationships of ratings. To our
knowledge this is the first time that QC systems have been used
with web lectures. If you are interested in participating in
this exciting project as an expert or being kept posted about
its progress, please contact Faina <fyl1@pitt.edu>. We really
look forward to your participation!
Best regards from Pittsburgh. We are rapidly moving to the 2,000
barrier.
Ron, Faina, Mita, Soni, Eugene, Samar, Monica, Julia, Abed, Deb,
Tom, Akira, Sherine
mailto:super1+@pitt.edu