All proposals submitted to the Main Call that request less than 50 h on the 12-m Array or less than 150 h on the 7-m Array in standalone mode will be peer reviewed using a distributed system, in which a designee from each proposal team participates as a reviewer in the review process.

Basic rules

1. All participants in the review process must behave in an ethical manner. If it is found that a reviewer has not behaved in an ethical manner or did not complete their reviews in good faith, the proposal(s) on which the reviewer is acting as the designated reviewer may be rejected.
2. Each proposal must designate one reviewer to participate in the review process. The designated reviewer may be the PI of the proposal or one of the co-Is.   
3. To keep the workload to a manageable level, it is recommended that reviewers review at most three Proposal Sets. Thus, PIs who are planning to submit multiple proposals are encouraged to designate one of their co-Is as the reviewer. The maximum number of Proposals Sets that a reviewer can be assigned is FIVE. This maximum number is not yet enforced by the OT but will be checked by the PHT after the proposal deadline. If a reviewer has been selected to receive more than five Proposal Sets, the reviewer will be contacted by the PHT to designate another reviewer among the proposal co-Is. If the reviewer does not identify alternative reviewers by 30 April 2024, 15:00 UTC, the PHT will reject the reviewer's proposal/s with the highest proposal code/s until the maximum allowed number of Proposal Sets to review is reached.
4. The reviewer must be specified in the Observing Tool (OT) at the time of proposal submission. The reviewer can be changed after the review process has started only in exceptional circumstances (e.g., medical emergency, urgent care for family member). A PI can request to change the reviewer through the ALMA helpdesk by opening a ticket to the department called "Proposal Review Support". If the PHT approves the request, the new reviewer will be given access to the assigned proposals and will assume responsibility for completing the review. The Stage 1 deadline for the new reviewer will remain the same because the Stage 2 process starts shortly after Stage 1 is completed.
5. PIs who do not have a PhD may be selected as the designated reviewer. In such cases, a mentor must be specified who will assist the PI in the review process. The mentor does not need to be part of the proposal team, but must have a PhD in astronomy or a related field, and must have an ALMA account since the mentor must be specified in the OT at the time of proposal submission. Co-Is who do not have a PhD are not eligible to be selected as reviewers.
6. Mentors will be able to access the assignments and reviews of their mentees through the Reviewer Tool in read-only mode.
7. Proposals will be assigned to reviewers based on the expertise of the reviewer as specified on the reviewer's user preferences. If a reviewer has not registered their expertise, then the assignment algorithm will use the scientific topics of their submitted proposals. More information on how the reviewer expertise is used to assign proposals is here.
8. Reviewers have the option to provide a list of investigators with which they are conflicted based on the conflict criteria. If this list is provided, reviewers will not be assigned proposals in which any investigator of their list is the PI, a co-PI, or a co-I. The list of conflicts of interest must be provided by 30 April 2024, 15:00 UTC. Additionally, when reviewers receive their proposal assignments, they must declare any major conflicts of interest by 15 May 2024, 15:00 UTC. Any proposals with a major conflict of interest will be replaced by another proposal.
9. During Stage 1, each designated reviewer will be assigned to review ten proposals. The reviewer must rank the proposals relative to each other in order of scientific priority from 1 (strongest) to 10 (weakest), and write a review for each one of them. If a person is the designated reviewer on multiple proposals, they will receive ten unique review assignments per submitted proposal.
10. Stage 1 is mandatory. If a reviewer does not submit their reviews and ranks by the Stage 1 deadline (05 June 2024, 15:00 UT), the proposal for which they were identified as the designated reviewer will be rejected.
11. Participation in Stage 2 is strongly encouraged. During Stage 2, the Stage 1 reviews will be shown anonymously to the other (9) reviewers of each proposal. The reviewers can then re-rank their assigned proposals and edit their reviews.
12. At the conclusion of the process, reviews and ranks will be sent anonymously to PIs without any editing by the PHT. If reviewers participate in Stage 2, then their edited reviews and modified ranks will be sent to the PIs, otherwise the Stage 1 reviews and ranks will be sent.
13. All participants in the review process agree to keep the materials confidential and will not use the materials for any other means other than the proposal review. Participants will delete any proposals and any other review materials after they have completed their assessments.
14. All communications between the PHT and the reviewers will be done by email. The PHT will use the email address associated with the reviewers’ ALMA user account. Please make sure to keep your user preferences updated so you do not lose important information.  
15. Reviewers can contact the PHT through the ALMA helpdesk by opening a ticket to the department called "Proposal Review Support".

Proposal review assignments

After the proposal deadline, the PHT will assign ten proposals to the designated reviewer of each submitted proposal. The PHT will attempt to assign proposals within the area of expertise of the reviewer. The expertise will be assessed based on the reviewer’s proposal history using machine learning algorithms and the expertise specified in their user preferences accessible from the ALMA Science Portal. If the reviewer does not specify their expertise, they will be assumed to be an expert in the scientific topic of the submitted proposal.

The PHT cannot guarantee that all proposal assignments will match  the reviewer's expertise. Reviewers may receive proposals in different topics within the scientific category of their expertise, or even “similar” categories; e.g., a reviewer who is an expert in Category 1 (Cosmology and the high redshift universe) may receive proposals from Category 2 (Galaxies and galactic nuclei) and vice versa. The PHT will avoid assigning proposals far outside the reviewer’s expertise; e.g. the PHT will not assign a stellar evolution proposal to a cosmology expert unless the reviewer indicates their expertise is in both categories.

If a reviewer receives a proposal that does not match their expertise, they are nonetheless asked to review it. PIs are instructed to write their proposals for a non-expert. Reviewers will also have the opportunity to view comments from the other reviewers in Stage 2.

Declaring conflicts of interest

Proposals are assigned to reviewers to avoid major conflicts of interest. Because of the dual anonymous review process, reviewers can provide a list of investigators with whom they have a conflict of interest through their user preferences on the ALMA Science Portal. If no major conflicts of interest are specified in the user preferences, then an algorithm is used to determine the conflicts based on the PI and co-Is of the proposals from the reviewer. Further information on the procedure and what constitutes a conflict of interest is provided here.

Stage 1

The PHT will assign ten proposals per each submitted proposal; this set of ten proposals is called a “Proposal Set”. During Stage 1, reviewers will rank the proposals in their Proposal Set relative to each other in order of scientific priority (from 1 to 10, strongest to weakest), and will write a review for each one of them. Reviewers access their Proposal Set through the Reviewer Tool. Usage of this tool is explained in the document: How to use the Reviewer Tool. All reviewers should refer to that documentation for a detailed description of the tool functionalities; here a broad outline of the main actions is provided.

At Stage1, all reviewers should carry out the following sequence of steps:

1. Access the Reviewer Tool.
2. Browse through the abstract of the proposals in their Proposal Set and declare in the Reviewer Tool any possible conflicts of interest that the assignment algorithm did not identify. Reviewers should note that the proposal PDF files are not available until conflicts of interest are submitted.
3. Submit the conflict decisions before starting to review the proposals. Reviewers should submit their decisions by 15 May 2024, 15:00 UT, at which time the PHT will review and replace any conflicted proposals with other proposal assignments when the conflict is confirmed. If a person is the designated reviewer on more than one proposal (i.e., they have more than one Proposal Set), they will need to identify conflicts of interest on all their Proposal Sets before being able to submit them. Even if no conflicts are detected, the reviewer should submit the decisions by the deadline. Conflict decisions submitted after the mentioned date will have lower chances of getting replacement proposals that are aligned with the reviewer's expertise
4. Download the proposal package. For every Proposal Set, the corresponding proposal package will contain the PDF files of the proposals that were declared free of conflict.
5. Review the proposals to provide a rank from 1 to 10 (strongest to weakest) and write a review that summarizes the strengths and weaknesses of the proposal.
6. Submit the ranks and reviews. Stage 1 is mandatory. If a reviewer does not submit the Stage 1 ranks and reviews by the Stage 1 deadline (05 June 2024, 15:00 UT), then the proposal for which they were identified as the designated reviewer will be rejected. If a person is acting as designated reviewer on more than one proposal, they will be able to submit the reviews for each Proposal Set independently.
7. Submit the optional reviewer survey, accessible through the Reviewer Tool.

Stage 2

In Stage 2, reviewers who submitted their Stage 1 ranks and reviews can read the anonymized reviews from the other (9) reviewers who were assigned each proposal. This will allow reviewers to determine if they overlooked critical strengths or weaknesses in their original review that were identified by other reviewers. If desired, reviewers can then modify their original ranks and reviews. If a reviewer participates in Stage 2 but does not submit their ranks and reviews in Stage 2, the Stage 1 ranks and reviews will be considered final. Note that the purpose of Stage 2 is not to start a conversation among reviewers to drive to a consensus among them, but to help them identify important issues in the proposal that may have been overlooked initially in their own reviews.

Stage 2 consists of the following steps::

1. Access the Reviewer Tool.
2. Access the anonymized reviews from the other (9) reviewers following the instructions in Section 7 of How to use the Reviewer Tool.
3. Edit the Stage 1 reviews as needed, and/or re-rank the proposals in their Proposal Set.
4. Submit the updated reviews and ranks. The deadline to complete the Stage 2 review is 20 June 2024, 15:00 UT. Stage 2 is optional. No proposals will be subject to disqualification if their designated reviewer does not submit the Stage 2 ranks and reviews by the deadline.

Creating the ranked list of proposals

After the completion of Stage 2, the ranks of the individual reviewers are combined to form  an overall ranked list of proposals. For a given proposal, the highest and lowest ranks from the assigned ranks will be dropped in order to remove possible outliers. The remaining ranks will be averaged. The average ranks for all proposals will then be sorted to determine the overall ranked list.

Frequently asked questions

Answers to frequently asked questions are available on the distributed peer review FAQ page.

Return to the main ALMA Proposal Review page