JP2%NIHCU.BITNET@CUNYVM.CUNY.EDU (07/09/88)
Attached is an NIH announcement of a change in the way it calculates
and uses percentiles in the evaluation of applications. Many
study sections in the current round (June/July '88) have spread
their priority scores over the full range of scores (100-500)
in response to this change of policy. Applicants should check
with their perspective Institute program administrator as to
the meaning of individual percentiles and prioty scores.
Jane Peterson, NIGMS, NIH.
ANNOUNCEMENT: NEW BASIS FOR PRIORITY SCORE PERCENTILES AT NIH
Research grant applications reviewed at the June/July, 1988 round
of initial review groups (IRGs) will be presented to the October
meetings of National Advisory Councils and Boards. At those Council
and Board meetings, percentile values, rather than priority scores,
will be utilized as one important factor in making funding
decisions. Thus, starting in October, all of the NIH funding
components will be utilizing percentile values. This action will
emphasize the importance of relative rank and provide compensation
for widely differeing scoring practices that have occurred among
IRGs in recent years.
Until now, percentile values have been calculated against a
reference base of applications favorably recommended by a chartered
review group at three consecutive meetings. This has served to
smooth out some of the variations that may exist from meeting to
meeting. Beginning with this June/July round of meetings, the
basis for calculating percentiles will be altered. For this round,
the percentile base will be the applications favorably recommended
at this round only. When the IRGs meet again in October/November
of this year, percentile values will be calculated against a
reference base of that meeting and the preceding June/July meeting.
In other words, there will be a phase-in process whereby percentiles
are calculated on the bases of one round, and then for the next
cycle of reviews they will be based on two rounds of scores, and
for the one after that (February/March 1989 IRG meetings) we will
be back to the standard usage of three rounds for the DRG Study
Sections.
This phase-in of new bases for the calculation of percentiles is
designed primarily to encourage reviewers to adjust their
evaluation scales and ratings and to utilize fully the entire
range of 1.0 to 5.0. All NIH IRGs are being asked to utilize the
following adjectival scale in terms of scores that are assigned:
Outstanding 1.0 - 1.5
Excellent 1.5 - 2.0
Good 2.0 - 2.5
Satisfactory 2.5 - 3.0
Adequate 3.0 - 3.5
Fair 3.5 - 4.0
Acceptable 4.0 - 5.0
It should be understood that the percentile rank of applications
reviewed at this June/July round will be wholly independent from
the priority score distribution in any prior review meeting.
Therefore, IRGs have a window of opportunity to adjust
evaluation scales and to make the distribution of priority scores
more symmetrical and more useful. This may be done at this time
without penalizing any applicants because of the change. Thus,
members now have an opportunity to recalibrate their voting scales
without the concern of disadvantaging any application.
It is important to note that this assurance can be given only if
such voting changes occur now and are not postponed to some future
round.
The bases for percentile score calculations are summarized below:
June/July IRG Oct/Nov IRG Feb/March '89
Meeting Meeting IRG Meeting
(Oct '88 Council) (Jan '89 Council) (May Council)
____________________________________________________
Chartered DRG June/July '88 June/July '88 and June/July '88 and
Study Sections only Oct./Nov '88 Oct/Nov '88 and
Feb/March '89
Other Initial June/July '88 June/July '88 June/July '88 and
Review Groups (partial) (total) Oct/Nov '88