🔍

Science Highlight

 

 

More...
news!

ALMA Proposal Review

The ALMA proposal review process is organized by the Proposal Handling Team (PHT) at the Joint ALMA Observatory (JAO). ALMA proposals are selected by competitive peer review through either the distributed peer review process or the ALMA Proposal Review Committee (APRC).

Proposals that request less than 50 h on the 12-m Array or less than 150 h on the 7-m Array in standalone mode are reviewed using the distributed peer review system, in which the proposal team designates one member of the proposal team to participate in the review process. The outcomes of the distributed peer review process are:

  • A scientifically ranked list of proposals
  • Individual comments for each proposal written by the reviewers that are sent to the Principal Investigators (PIs).

Large Programs, i.e., proposals that request more than 50 h on the 12-m Array or more than 150 h on the 7-m Array in standalone mode, are reviewed by the APRC, a panel composed of experts selected from the international astronomical community. To gain further expert assessment, external Science Assessors will provide reviews on Large Programs, that will be considered by the APRC. The outcomes of the APRC review process are:

  • A list of recommended Large Programs
  • A consensus report for each Large Program that summarizes the strengths and weaknesses of the proposal.

All proposals are reviewed in a dual anonymous fashion in which the proposers do not know the identity of the reviewers and the reviewers do not know the identity of the proposers. All proposals need to be prepared in accordance with the dual-anonymous guidelines.

 

Distributed peer review plenary session

The PHT will be hosting a series of live Webinars, during which the PHT will present a video that describes distributed peer review and will also be available to answer your questions. The video will be posted here before the Webinars. Reviewers and mentors are strongly encouraged to view the video or attend one of the sessions. The Webinars will be held between 25 May and 31 May at different times to accommodate reviewers and mentors in each of the ALMA regions. Reviewers and mentors can attend any one of the three scheduled Webinars that is most convenient to them. The schedule for the three sessions is below:

Session 1: Thursday May 25, 17:00 UTC

Session 2: Friday May 26, 13:00 UTC

Session 3: Wednesday May 31, 2:00 UTC 

Prior to the Webinars, the PHT will send an email to all reviewers and mentors with more detailed information.

 

Slides can be downloaded as a PDF file.

 

Available documents for the proposal review process

 

Document

Description

Dual-anonymous Guidelines

Provides guidelines for PIs on how to write their proposals in a dual-anonymous fashion

Distributed Peer Review

Detailed description of the distributed peer review process

Guidelines for Reviewers

Guidelines for reviewing proposals and writing comments to the PIs

How to use the Reviewer Tool

Reviewer Tool manual for the distributed peer review process

Frequently asked questions

Contains answers to numerous questions about the dual-anonymous format and distributed peer review

 

New in Cycle 10

Distributed peer review

  • A reviewer can be assigned a maximum of FIVE Proposal Sets; i.e., 50 proposals. More information is provided on the web page that describes Distributed Peer Review.
  • Reviewers who require support from the PHT during the proposal review process should open an ALMA helpdeskticket to the department called "Proposal Review Support".

Proposal review timeline

Table 1 summarizes the timeline of the Cycle 10 proposal review process. A brief description of each step is given below. 

Table 1: Timeline of the Cycle 10 Proposal Review Process

Date

Milestone

16 May 2023

Deadline for reviewers to provide their scientific expertise and their list of conflicts of interest before proposals are assigned.

24 May 2023

Proposals are released to all reviewers. Stage 1 begins for both distributed peer review and the APRC.

01 June 2023

Deadline to provide conflicts of interest

28 June 2023

Stage 1 deadline for all reviewers.

29 June 2023

Distributed peer review Stage 2 begins.

10-14 July 2023

APRC meeting.

13 July 2023

Distributed peer review Stage 2 deadline.

August 2023

Results sent to PIs.

 

Distributed peer review

Here is a quick guide to the basic steps in distributed peer review, from proposal submission to the end of the review process. A complete description of the process can be found here.

  1. Submit a proposal through the ALMA Observing Tool. When submitting the proposal, the PI must designate one member of the proposal team to be a reviewer.  
  2. Reviewers should indicate their scientific expertise through their user preferences on the ALMA Science Portal by 16 May 2023, 15:00 UTC. All efforts will be made by the PHT to assign proposals to reviewers that match their field/s of expertise. Click here for more information on how the keywords are used by the PHT to assign proposals.
  3. Reviewers have the option to provide a list of investigators for their conflicts of interest through their user preferences on the ALMA Science Portal. Reviewers need to provide this list by 16 May 2023, 15:00 UTC, at which point the PHT will begin the proposal assignment process. The PHT will not assign proposals to a reviewer in which a PI, co-PI, or co-I is in this list. Click here for more information on conflicts of interest.
  4. The PHT will assign 10 proposals (a “Proposal Set”) to the reviewer on behalf of each proposal in which they are the designated reviewer.
  5. In Stage 1, the reviewers will rank the proposals within a Proposal Set from 1 (strongest) to 10 (weakest) and write a review for each proposal that indicates the strengths and weaknesses. Click here for guidance on writing helpful reviews. Stage 1 is mandatory. Reviewers must submit their reviews by the Stage 1 deadline (28 June 2023, 15:00 UTC) or the corresponding proposal for which they were identified as the designated reviewer will be rejected.
  6. In Stage 2, reviewers can read the anonymized comments from the other reviewers and revise their own ranks and reviews as needed. Stage 2 is optional; no proposals will be disqualified if their designated reviewer does not submit Stage 2 ranks and reviews by the Stage 2 deadline (13 July 2023, 15:00 UTC). If a reviewer does not submit the Stage 2 ranks and reviews, then the Stage 1 ranks and reviews will be considered final.

 

ALMA Proposal Review Committee

Large Programs will be sent for review to the APRC. The general steps for the assessment of these proposals are outlined below.

Stage 1

The APRC members review their assigned proposals, provide a numerical score for the proposals, and write comments summarizing the strengths and weaknesses of the proposal. The PHT assigns each Large Program a Primary Assessor, who is responsible for leading the proposal discussing in Stage 2 and writing the consensus report that will be sent to the PI.

External Science Assessors review a small number (1-3) of Large Programs in their area of expertise and provide a written report with their reviews.

Stage 2

The APRC  read the reports of the external Science Assessors in preparation for the meeting. The APRC discusses the proposals and recommends which Large Programs should be scheduled. The Primary Assessors write consensus reports to reflect the final scientific assessment.

 

Queue building

The main outcome of the proposal review process is a scientifically ranked list of proposals. The inputs used to generate the ranked list are 1) the proposal rankings of Proposal Sets from individual reviewers in distributed peer review, and 2) the ranked list of Large Programs recommended by the APRC.

In distributed peer review, each reviewer ranks their assigned proposals from 1 (strongest) to 10 (weakest). The rankings from individual reviewers are combined to form an overall ranked list. For a given proposal, the highest and lowest ranks are dropped to remove possible outliers. The remaining ranks are averaged.  The sorted list of average ranks determines the overall scientific ranked list of proposals.

The ranked list of proposals only establishes the priority order to build the observing queue.  The Large Programs recommended by the APRC have first priority in building the queue, but they are limited to how much time they can fill in a configuration/LST as described in the Proposer’s Guide. Creating the observing queue also takes into consideration the available time in a configuration and LST, the amount of time allocated to each executive, and the amount of time available based on historical weather conditions for a given observing band and observing frequency. It is during the queue-building stage that the priority grades (A, B, C) are assigned to the accepted proposals, and at this stage a proposal may be declined depending on the amount of telescope time available. Because numerous factors are considered in building the queue, it is possible for a high ranked proposal to be declined while a lower ranked proposal is accepted.

After the ranked list is generated and the observing queue is created, the ranked list and the priority grades are sent to the ALMA Director for approval. After the Director’s approval, the results are sent to representatives from East Asia, Europe, North America, and Chile for final approval. Principal Investigators are then notified of the results.