The influence of order effect and reviewer experience in the grants peer review process
The review of the literature of the grants peer review process provides evidence that a variety of factors beyond the funding agencies criteria and merit of the proposed project may influence the outcome and ultimate funding decisions in competitive grants. Among these factors are the demographics of the applicant, the accusation of cronyism present in peer review panels, and the presence of corruption and bias in the process. Additionally, it has been suggested that there is an element of arbitrariness and luck involved in the ultimate score a proposal receives. In most cases, the difference of one point can determine if a proposal is funded. When two proposals are basically equal in merit, the taste, biases, and mood of the reviewer can make the difference. The primary purpose of this study was to determine if the score a proposal receives is influenced by the order in which it was read and the experience in grants peer review of the reader. The competition which provided data for the study was conducted by the U.S. Department of Housing and Urban Development (HUD), Office of University Partnerships. Thirty-seven persons participated as readers for this study. These readers served on 13 proposal panels. Each panel was assigned 6 proposals to read and score. In all, 438 scores were generated from this proposal review process. After the last proposal was read and scored, each reviewer completed a nine item questionnaire which was designed to provide supplemental information about the reviewers and to provide descriptive data related to the rationale used to select the order in which the proposals were read. The primary finding of this study is that there is a statistically significant difference between the initial mean scores for proposals that are read first and the initial mean scores for proposals that are read second through sixth. The initial mean score for proposals that were read first was 51.00 while the initial mean score for proposals read second through sixth was 62.67. The results of the ANOVA found this difference to be well below the.05 level of significance
Thesis, Dissertation, English, 1998
University of Missouri-Columbia
Academic Dissertation
ix, 120 leaves ; 29 cm
43977217
Typescript
Vita
Full text PDF download free to MU campus, to others for purchase