Questions for the grant proposal reader

In the last newsletter, we compiled a list of funding resources of interest for graduate students and postdocs. Writing for grants can be time-consuming and sometimes difficult, especially when one wonders what happens in the “black box” of the proposal committee process. We wanted to follow up our last newsletter article with a conversation with a grant proposal reader in the hope of providing insight into the minds of the proposal readers. We hope that this conversation will be beneficial to those of us who will be studiously writing for academic and financial support for research projects in the coming months. One of the most important exercises in our nascent careers may be figuring out what gets our manuscript or application from being discarded (and on top of the pile, for that matter), especially considering that the competition for funding can be daunting. However, strong competition should not dissuade a researcher from applying. Perhaps a well-formatted and conscientious application is the key to a successful proposal and, hopefully, you’ll get the notice that your research funding is approved.

What is the format during the time all the readers converge to go over grant proposals? What occurs during this time period?

This depends on the agency and the competition you are reading for, but there are common elements: all readers must be qualified, so each of us has had to obtain a DUNS number so we are independent contractors, and each must register with the CCR Central Contractor Registry to work for the federal government. Then, for each agency separately in their particular system, each of us will have had to fill out a demographic and professional profile, upload a CV, and provide a statement of the competitions we are qualified to read for, and why. Then months to years later, we receive an emailed invitation to read for a specific competition 6-8 weeks in advance to check our availability and give the block of time of the reading window. If we are available, we go into the system and check the box and say yes. Then, a week later, if we are selected as a possible reader, we get an email saying we are selected and providing passcodes to enter into the competition area of the database. We download the proposal RFT application instructions the applicants had, the rubric and instructions for readers, and read them all. We are given directions for how to participate in an orientation phone call (although some agencies are moving to webinars now). We do the orientation, which lasts about an hour and provides all readers the same instructions. After the orientations are complete and the agency sees the list, they select the readers they will use. If not selected, you get $100 for doing the orientation and are placed into a list of alternates that might be called in at the last moment. If selected, how the reading happens is variable. Sometimes we all start by reading and ranking a sample proposal and participating in a calibration exercise so that we know what to look for, but in other competitions, we just read without the calibration.

Some agencies deliver proposals in physical form to your home; for others, you download the applications to your computer. Similarly sometimes we have the ranking discussions on the telephone in group calls moderated by a monitor from the agency, but in others you will read, rank, enter and then be flown to Washington DC to have the discussions while yet in other competitions, you will be flown to Washington DC to participate in reading panels where you are reading and ranking in real time and entering your scores in a computer or reading and ranking on paper sheets in the room with all the readers and in those cases the pressure to read quickly and rank precisely is enormous as we have to get them all read and ranked in the time period available, even if we have to work late into the night.

You read the proposals in numbered order, and no matter who you read for you are instructed not to do comparisons, but rather to make sure that you are only considering the information and data in the one you are reading. Personally I work with a printed copy so that I can make notations on the first read through right on the paper, which is helpful in the discussions later. We complete the scoring rubric and generally input the information and your comments into a form in the online database provided by the agency, and then after all 3 or 4 committee members in your panel subset have read and ranked and placed comments in the panel area, we are allowed to enter into a group conversation about them. We look for wide variation in scoring first as we have to be within a specified range of one another If we cannot come within the range because one of us highly disagrees with the assessment of the others (a situation highly discouraged), then the proposal cannot be completed and has to be sent to a different panel for reading. Interestingly, in most cases, the proposal in the batch that received the highest score from me will have also received high marks from the others and vice versa for the poor ones. The differences are generally in number of points we awarded, but overall excellence rises in the collective assessment process. The panel of the proposals and their scores are sent forward to the agency. Agency personnel read all the comments and look for any contradictions; we are asked to adjust our comments, then the panel is cleared to send forward the scores to the agency as official. The agency then distributes funding based on the internal directives, generally on numeric order, but not always as geographic or disciplinary distribution might also be a factor, but that will be spelled out in the application packet. We generally will read and rank 10 proposals in a panel in a two-week time period, but may have to do more. Compensation for all of this is generally $100 per proposal read. Most readers have “day jobs” as teaching faculty or researchers or administrators, so this work is an add-on to our regular work schedule and requires us to use our vacation time to participate.

In rare cases, we are asked to read and evaluate a single research proposal that has been sent to us from a program officer and is being considered independently for a targeted or solicited research project, in which case we do that and make a written report.

What are you looking for in a proposal?

Within the conditions of panel readings, a reader is trying to read quickly to find the information and data that must be in that section for either awarding or deducting points. Once located, then the quality of that piece is ranked. Clearly from the reader’s perspective,

we hope to find the proposal arranged in the order dictated in the application with headings to guide us. If we have to search the proposal to find the information, it makes the task more difficult and can add an element of confusion in ranking and in the discussions later. We hope to find excellence. A solid idea for a systematic investigation that is well planned and articulated, that will yield findings of value to the agency in that competition and that has qualified staff and budget that balances and provides adequate resources to accomplish the proposed project in the given time frame. We are looking for evidence that if funded, the project has a high likelihood of being completed.

With proposals for fellowships for pre-doctoral awards/graduate fellowships, we are funding the person not exactly the proposal idea; so we will be looking for evidence of potential, such as previously completed research, transcripts of not only grades but courses taken, presentations at conferences, ability to write well, and strong letters of recommendation.

How do you decide if the proposal you are reading fits the criteria for the funding source?

Once it is scored, you know. If it is not a good fit, it does not receive a high score. If it seems to be placed in the wrong disciplinary group, for instance sent to the social-psychology panel but it is for neurobiology research, you alert the program officer and it will be relocated to the proper panel to read.

Do proposals that don’t exactly match the statement of the grant provider get a break if they are good?

The readers are required to read and rank and score using the rubric, so it would be rare to be off the mark and receive a good score. However at funding sources, the program officers are often also looking for innovation, and readers might flag a proposal for consideration for a different sort of funding, but cannot score it highly, for example, just because it is my favorite topic or is proposing an innovation I personally think would work really well.

Readers must see lots of applications during the review period. What makes a proposal easier to read?

Follow the format to the letter, and provide internal headings to guide us. It is a subjective process, and we are under enormous time pressure to complete the task quickly, so searching the entire proposal to verify whether or not one specific element/data item is somewhere in there takes time, can frankly annoy the reader, and put the reader in a negative space for the rest of the proposal, which might result in a lower score for the same project than if it were presented in a coherent way following the guidelines exactly.

How important is it that the budgeting match the aims of the proposal exactly?

Agencies seek to provide funding to projects they think are going to be successfully operated. If the budget is too low, the readers (who are experienced with budgets and research) will know that. If the budget is too high, or if a lot of less-than-specific consultants are to be used, and the narrative does not explain the reasons for  this, that is cause for notation too. But the most frequent problem with budgets is that either they do not add up due to mathematical errors or they do not match the figures presented in the narrative budget justification statement.

How do you approach proposals that are not from your field of expertise?

We are rarely selected to read outside of our areas, but when that has happened with me, for example being called in at the last moment as an alternate, I have done my best in the reading and ranking and then listen carefully in the discussions and ask for the others to explain to me their logic if I am far out of alignment with their assessments, and I may adjust my comments and scoring a bit.

What makes it easier for you to understand the importance of a study that is not specifically in your field?

It is helpful if the proposal is not filled with acronyms (or they are defined in the first use) and if the language is not overly filled with jargon. A good abstract and a solid budget justification narrative with details of how amounts were calculated are also very helpful.

Where do you expect to see the goals of the proposal outlined in a way that answers the question, “So What?”? In other words, how can a writer explain the importance of their study in a way that grabs the reader?

Assume that your readers are in your discipline, and often in your sub-topic area of knowledge, so they “have a clue” regarding the field. I know that proposals that can set the project in the discipline and then tell how the results of such an investigation might be applied “in the real world” are useful to the reader, even if the proposal is not addressing the applications itself. To tell someone how the knowledge can have practical applications is helpful.

What do you do with a proposal that doesn’t have a realistic budget?

The proposal receives a low score or a zero score on that piece and will not be funded. This is true also of proposals that request an amount that exceeds the level stated in the application; however, if the proposal is otherwise sound, the program officer might encourage the applicant to adjust the budget and submit again in the next competition one to five years from then. What are the things about a proposal that gets it talked about? What sets a proposal apart from the others?

I was raised in an oral tradition so I personally like a proposal that tells me a “story” or provides an “anecdote” as a very short example of either the problem or why the research should be conducted. But more importantly is that we like the excellent ones. They are easy to read and rank, the applicant took care in construction and, if presenting a novel or new idea, they tell us that it is such.

Are there any other comments you would like to add?

My advice for graduate students and people starting their careers in research or the academy is to engage in the process yourself. You will quickly become a better proposal writer by engaging in the funding process through participation as a reader. Then, when you write for funding, you will put on your “reader hat” and critically evaluate your own proposals.

Readers seldom engage in this process “for the money” because, in the end, the amount of compensation for the time expended results in minimum wage or less. However, sometimes when the reading transports you to a different location, you meet new, interesting, and engaged peers and are given the opportunity to meet the program officer in person. If you do a good job, the program officer will call on your services again and/or they will remember your name when you are calling them about a call for proposals and will give you time to vet your idea and give you constructive feedback before you write and submit a proposal, which can save you significant time and energy and enhance the odds that your project, if well-articulated, will be funded.

Readers engage in the process to learn more about a topic, keep their own knowledge of the contemporary field current, develop their proposal writing skills and/or to be of service to their country, their disciplines, and the common “good”.