Pyxis

Basic Scripting (Pyxis 101)

Interferometry Scripting (Pyxis 102)

In this section we will write a pipeline that performs the steps described in the last section of Calibration 101. This section also serves as an introduction to Pyxis.

Pyxis is a set of Python-based scripting services that is meant to simplify the process of making data reduction and simulation scripts, using MeqTrees and other diverse bits of software. Pyxis scripts are essentially Python scripts, with extra convenience features provided by the Pyxis framework\footnote{Pyxis borrows interface ideas from, among others, the Perl language, casapy, pybdsm, etc.}. Pyxis scripts can be run from the command line, or interactively via ipython. Skeem through the "documentation" before you continue.

Lock and Load

First thing to do is import Pyxis and the modules we will need for our pipeline

import Pyxis
# once Pyxis is loaded, ms,mqt, im, lsm, std,stefcal become available
import ms # ms module
import im # imager module 
import mqt # meqtree-pipeliner wrap
import stefcal # self calibration module 
from Pyxis.ModSupport import * # I will make a note whenever I use something from here

Lets set up how we want our output to be structured

DESTDIR_Template = 'plots-${OUTDIR>/}${MS:BASE}' # images will be sent to this directory
OUTFILE_Template = '${DESTDIR>/}${MS:BASE}${_s<STEP}${_<LABEL}' # prefix for output from pipeline
LOG_Template = lambda : II('log-${DESTDIR>/}${MS:BASE}_pyxis.txt') if OUTDIR!='.' else II('log-${MS:BASE}_pyxis.txt') 

define some globals

LSM = 'bright2deg.lsm.html'
MS = 'kat7_8h30s.MS'

I want to add documentation for the globals I'll define bellow, so I use the define funtion from Pyxis.ModSupport

define("MS_REDO",True,"Remake MS if it already exists")
define("OBSERVATORY","kat-7","Observatory")
define("ANTENNAS","KAT7_ANTENNAS","Antenna table") 

# Lets also define some MeqTrees related variables

define("CATTERY",os.environ['MEQTREES_CATTERY_PATH'],"Cattery path")
TURBO_SIM = II('$CATTERY/Siamese/turbo-sim.py')
STEFCAL = II('$CATTERY/Calico/calico-stefcal.py')
define("TDLCONF","tdlconf.profiles","TDL configuration file")
define("SIMSEC","sim","Simulation section in TDL profile")
define("CALSEC","stefcal","Calibration section in TDL profile") 

Finally, lets set up the the imaging parameters

import im.lwimager # we will use LWIMAGER for imaging. 
# the casa imager (clean task) and wsclean are also available via im.{casa,wsclean}
im.npix = 1024
im.cellsize = '30arcsec'
im.stokes = 'I'
im.niter = 1000
im.weight = 'uniform' 

Now, we are ready to define functions that will simulate, calibrate and image data. I will only discuss the function that simulates visibilities into a measurement set, given a sky model. The rest of the pipeline can be found here.

Lets define a function that will simulate visibilities into an MS given an sky model, and a TDL script.

def simsky(msname="$MS",lsmname="LSM",column="$COLUMN",
           tdlconf="$TDLCONF",tdlesec="SIMSEC",
           noise=0,args=[],**kw):
    """ 
    Simulates visibilities into a MS.
    msname : MS name
    lsmname : LSM name
    column : Column to simulate visibilities into
    tdlconf : Meqtrees TDL configuration profiles file (required to run MeqTrees pipeliner) 
    tdlsec : Section to execute in tdlconf
    noise : Visibility noise to add to simulation.
    args, kw : extra arguments to pass to the MeqTrees pipeliner
    """

The interpolate_locals() function is also part of ModSupport. It interpolates the variable names (from the local context) given by its argument(s), and returns new values in the order given

    msname,lsmname,column,tdlsec,tdlconf = interpolate_locals('msname lsmname column tdlsec tdlconf')
# Now update LSM and MS   
    v.MS = msname
    v.LSM = LSM
# Overwrite/set MS and LSM names in the TDL profile
    args = ["${ms.MS_TDL} ${lsm.LSM_TDL}"] + list(args)

Next, we want to update the output column and noise paremeters in the tdl profile. To do this lets create an options dictionary, which we will use to update parameters in the tdl profile.

    options = {}
    options['ms_sel.output_column'] = column
    if noise:
        options['noise_stddev'] = noise
    options.update(kw) # extra keyword args get preference

We are finaly ready to run the MeqTrees pipeliner!

    mqt.run(TURBO_SIM,job='_tdl_job_1_simulate_MS',config=tdlconf,section=tdlsec,options=options,args=args)

One more thing. To add the globals MS, LSM, COLUMN, TDLCONF, TDLSEC to the documentation of this function, use the function document_globals (from ModSupport). The documentation given via define will be displayed. Note: LSM and MS are already by the function.

document_globals(simsky,"MS LSM COLUMN SIMSEC TDLCONF TURBO_SIM")