You are here: Home / Processing / Overview and Pipeline

Overview and Pipeline

ALMA calibrates and images PI data following the "Quality Assurance level 2" or QA2 process.  Most (>95%) of ALMA PI data is now processed by the pipeline through calibration and imaging, with the remainder processed manually by ALMA staff (the fraction processed manually was larger in earlier cycles).  Pipeline and manual processing both use CASA.  All data are reviewed by a data analyst before delivery, to ensure quality and that the PI-requested angular resolution and sensitivity are achieved.

The processed data are packaged for distribution through the archive, and as described in the ALMA QA2 Data Products Document, products include images, but not calibrated visibilities.

For pipeline-processed data, the most useful resource is the pipeline weblog, included in the qa/ directory of a product package, which contains a very detailed log, with heuristics, quality assurance, and plots of the processing steps and outcomes.  The Pipeline User's Guide (links below) describes in detail how the pipeline works, and the User's Guide version corresponding to the pipeline version listed in the weblog is the most accurate for that version.

Users can recover calibrated visibilities by several methods:

  • by running scriptForPI.py contained in the archived QA2 package.  For manually processed data, one must run that script in the same version of CASA used to originally calibrate the data (CASA task parameters and inputs may have changed).  For pipeline processed data, one can use the latest accepted version of CASA, with some exceptions for older data as noted in the Table below.  See the ALMA QA2 Data Products Document for more details on running scriptForPI.py and its options.
  • by request to the helpdesk, staff at your ARC will restore the visibilities and provide them to you for download. (knowledgebase instructions)

For all data analysis other than restoring older calibrated visibilities, ALMA strongly recommends using the most recent version of CASA that has been accepted by ALMA for data processing (currently 6.1.1-15; see Table below). These accepted versions of CASA also contain the data processing pipeline, which the user can rerun themselves, as described in the Pipeline User's Guide.   Additionally, a CASAguide describing interferometric pipeline re-imaging examples is available at: https://casaguides.nrao.edu/index.php/ALMA_Imaging_Pipeline_Reprocessing

In general, pipeline tasks from any pipeline release should run successfully on data from that cycle or earlier.  Using the most recent pipeline version will include improved calibration, flagging, and imaging heuristics that should improve the products, resulting in differences from previous versions (from modest to significant, depending on how early the original version was). To get the identical results as an archived pipeline product, one should use the version of CASA+PL that was used to produce the archived product.

The pipeline is not officially validated on MacOS, nevertheless there are no known bugs when restoring data or performing data analysis on OSX.

PIs are strongly requested to note the versions of both CASA and of the Pipeline in their publications, or that the data were calibrated manually by ALMA staff, to provide a clear record of what heuristics were used to process the data. 

CASA versions accepted for ALMA data processing:

CASA versionPipeline branch and versionPipeline DocumentationDescriptionused in operations

tarball for most modern OS available for each

versions that can be used to restore these data
6.1.1-15 2020.1.0-40

Users Guide 2020.1

Reference Manual 2020.1

Pipeline2020 known issues

Cycle 7 reprise 2021-05-10 ~

casa-6.1.1-15-pipeline-2020.1.0.40.tar.xz

casa-6.1.1.15-pipeline-2020.1.0.40-10.15.dmg

most recent
5.6.1-8 Pipeline-CASA56-P1-B, r42866

Users Guide 5.6.1

Cycle 7 2019-10-01 ~ 2021-05-10

casa-pipeline-release-5.6.1-8.el7.tar.gz

casa-pipeline-release-5.6.1-8-10.14.dmg

most recent
5.4.0-70 Pipeline-CASA54-P1-B, r42254 and r42866

Users Guide 5.4

Reference Manual 5.4

Cycle 6 2018-10-01 ~ 2019-10-01

casa-release-5.4.0-70.el7.tar.gz

most recent
5.1.1-5 r40896 Pipeline-Cycle5-R2-B, r40896 Users Guide 5.1.1 Cycle 5

2017-10-01 ~ 2018-10-01

casa-release-5.1.1-5.el7.tar.gz

casa-release-5.1.1-5-10.12.dmg

casa-release-5.1.1
4.7.2 r39762 Pipeline-Cycle4-R2-B, r39732

Users Guide 4.7.2

Reference Manual 4.7.2

Cycle 4 patch

2017-04-17 ~ 2017-10-01

casa-release-4.7.2-el7.tar.gz

casa-release-4.7.2-10.11.dmg

casa-release-4.7.0 or 4.7.2
4.7.0 r38355 Pipeline-Cycle4-R2-B, r38377

Users Guide 4.7.0

Reference Manual 4.7.0

Cycle 4 2016-10-25 ~ 2017-04-17

casa-release-4.7.0-1-el7.tar.gz

casa-release-4.7.0-1-10.11.dmg

casa-release-4.7.0 or 4.7.2
4.5.3 Pipeline-Cycle3-R4-B, r36660

QuickStart 4.5.X

Reference Manual 4.5.X

third  release 2016-02-10 ~ 2016-04-01

casa-release-4.5.3-el6.tar.gz

casa-release-4.5.3-10.10.dmg

casa-release-4.5.X

 

4.5.2-REL r36115 Pipeline-Cycle3-R4-B, r36091 patch 1 2016-04-01 ~ 2016-04-08

casa-release-4.5.2-el6.tar.gz

casa-release-4.5.2-10.10.dmg

4.5.1-REL-6 r35996 Pipeline-Cycle3-R4-B, r35932 patch 2 2016-04-08~ 2016-10-06

casa-release-4.5.1-el6.tar.gz

casa-release-4.5.1-10.10.dmg

4.3.1-REL r34291

Pipeline-Cycle3-R1-B, r34044

QuickStart 4.3.1

Reference Manual 4.3.1

second release, Cycle 3 2015-08-07 ~ 2016-02-10

casa-release-4.3.1-pipe-1-el6.tar.gz

casa-release-4.3.1-10.8-pipe-1.dmg

casa-release-4.3.1-pipe-1
4.2.2 r30721,r30986

Pipeline-Cycle2-R1-B, r30594~31667

QuickStart 4.2.2

Reference Manual 4.2.2

first releases, Cycle 2 2014-09-22 ~ 2015-08-07

casapy-42.2.30986-pipe-1-64b.tar.gz ; compiled on RHEL5, works on RHEL6, not on RHEL7

CASA-intel64b-10.8-30986-pipe.dmg

casapy-42.2.30986-pipe-1