Proposal Review Table of Contents

ALMA Proposal Review

Dual-anonymous Guidelines

Overview of Distributed Peer Review

Guidelines for Reviewers

Reviewer Tool and User's Manual

Frequently Asked Questions

Overview

ALMA is dedicated to ensuring a fair and impartial proposal review process. To support this commitment, each review cycle is analyzed for potential biases, and findings are periodically published. The most recent analysis can be found in Carpenter et al. (2022).

To mitigate potential biases, ALMA employs a dual-anonymous review process, in which the identities of the proposal team and reviewers are concealed from each other. This approach encourages reviewers to focus on the scientific merit of the proposals, independent of the proposing team. 

Although proposers must enter their names and affiliations in the ALMA Observing Tool (OT), this information is omitted from the proposal cover sheet and the tools used by reviewers. Proposers are responsible for maintaining anonymity throughout their proposals, including the cover sheet, Scientific Justification, and Technical Justification.

Guidelines for preparing anonymous proposals are provided below. These guidelines primarily focus on writing style and include best practices for handling references. 

General Guidelines pertaining to all proposals

To ensure anonymity and fairness in the review process, please adhere to the following guidelines when preparing your proposal:

 

1. Do not identify the investigators

• Do not identify the PI, co-PIs, or co-Is anywhere in the proposal, including the proposal cover sheet, the Scientific Justification, and the Technical Justification.
• Discussing the proposer team’s expertise is strongly discouraged, even if it is done in an anonymous fashion.
• Proposal teams must delete any comments added to their proposals’ Scientific Justification using PDF editors, as these comments could inadvertently reveal their identities.
• Do not include personal acknowledgments or the source of grant funding.

 

2. Citations to papers, software, and datasets

In general, PIs must avoid claiming ownership of past work. Citations to papers, software, and data must be used in third person neutral wording. You can use any standard format for references, including using numbers for citations.

• Self-referencing

Use third-person or neutral wording when referencing your own work. For example, “We show in Smith et al. (2023)…” must be replaced with “Smith et al. (2023) show…”. In general, only use the first person possessive when talking about future work by the proposal team.

• Public software and datasets

Software or datasets that are published or publicly accessible (e.g., through repositories like GitHub) should be referenced as per normal practices in the third person or neutral wording. It is permissible to list the project code for datasets.

• Non-public software and datasets

Reference software or datasets that are unpublished or not publicly available as “obtained via private communications” or similar language, but a name should not be specified since it could strongly imply who may be an investigator on the proposal. It is permissible to list the project code for datasets.

• Submitted and “in press” papers

References to papers that have been submitted or in press are only permitted if they are available on public archives (e.g., arXiv), in which case the archive paper can be referenced. Otherwise, the paper must be referenced as “private communication” with no names indicated.

• Papers in preparation

Papers that are in preparation should be cited as "private communication” with no names indicated.

• Private communications

To preserve anonymity, do not mention the name of the Principal Investigator (PI) or project teams when referencing private communications, including unpublished papers, non-public software, or datasets.

• Personal or Team web pages

Do not provide links to personal web pages or to the web pages of project teams.

 

3. Proposal resubmissions

While proposers may note on the proposal cover sheet (in the “Duplicate Observation Justification” box) if they are resubmitting an ongoing proposal from the previous cycle, the proposal code, investigator names, priority grade, or ranking of the previous proposal must not be listed.

 

Here are examples of how to anonymize the proposal text.

Guideline	Non-anonymized text	Anonymized version
Self-referencing a paper	“In Smith et al. (2024), we demonstrated…”	“As demonstrated in Smith et al. (2024),....” or “As demonstrated in [1],...” (where [1] corresponds to the full citation in a reference list.)
Citing your previous work	“The data from our pilot program…”	“The data from program 2024.1.02045.S …”
Citing your previous work	“Our previous disk survey…”	“The survey from Smith et al. (2018) ...”
Citing your ongoing program	“The proposed ALMA observations will be combined with our HST data…”	“The proposed ALMA observations will be combined with available HST data (private communication) ...”
Citing a program from another observatory	“We will combine these ALMA observations with the HST program led by Smith et al.”	"We will combine these ALMA observations with ongoing HST observations (private communication).”
Citing published software	“We will use our software package XXXX (Smith et al. 2024).”	“We will use the software package XXXX (Smith et al. 2024).”
Citing unpublished software	“We will use our group’s line identification package XXXX…”	“We will use a line identification package (obtained via private communication) ...”
Citing unpublished data	“Figure 1 shows the image from ALMA program 2024.1.02045.S (PI Smith).”	“Figure 1 shows the image from ALMA program 2024.1.02045.S.”
Citing papers in preparation	“Figure 1 shows the CO image from Smith et al. (in preparation).”	“Figure 1 shows the CO image (private communication).”
Resubmissions	“This is a resubmission of our ongoing grade B program 2024.1.02045.S (PI: Smith). Half of our targets have been observed, and we are resubmitting the proposal to obtain the remaining half.”	“This is a resubmission of our ongoing program. Half of the targets have been observed, and we are resubmitting the proposal to observe the remaining half.”
Private communication with individuals or project teams	“The sources all have prior ALMA observations (Smith, private communication).” or “The sources all have prior ALMA observations (private communication from the XXXX Large Program).”	“The sources all have prior ALMA observations (private communication).”

 

Guidelines pertaining only to Large Programs

As described in the Proposer's Guide, proposals for Large Programs consists of two parts:

1. The Scientific Justification, which includes a Management Plan, must be prepared following the dual-anonymous guidelines described above. 
2. A Team Expertise that describes the team's expertise and background. This document can include the names and affiliations of at least the key members of the proposal team.

Further information on submitting Large Programs and the expected contents of the proposal can be found in the Proposer's Guide.

Compliance

PIs are required to anonymize their proposals. The Proposal Handling Team (PHT) will follow up on any reported violations of these guidelines. Proposal with significant violation may be disqualified, especially those that reveal directly the identity of the proposing team. PIs will be provided with feedback regarding any detected violations to avoid similar violations in future cycles. If this is the case, such information will be included as a Note from the JAO in the PI notification letter and also in SnooPI.

In some cases, a proposal may be very specialized, and the identity of the proposal team may seem obvious to the reviewers even after the text is anonymized. As long as the guidelines to anonymize the proposals are followed, the proposal will not be considered in violation.

Guidelines for Reviewers

Proposals will be reviewed through either the distributed peer review process or the APRC. To keep the process focused on the science, we provide guidelines to follow during the proposal review:

• Reviewers should evaluate the proposals based on scientific merit.
• Reviewers should not try to guess the identities of the PI or the proposing team.
• In the APRC meeting, the APRC chair is responsible for refocusing the discussion on the science merit of the proposal if the panel begins to discuss the identity of the proposal team.
• Even if a reviewer feels that a proposal has not followed the guidelines for anonymity, they should continue to review the proposal based on scientific merit and notify the PHT as follows:
• In distributed peer review, reviewers should use the box “Comment to JAO” via the Reviewer Tool.
• APRC members may also do so by adding a comment to the PHT in the ARP Meeting Tool during the panel meeting.