CALIM 2011

25 Jul 2011, 07:30 → 29 Jul 2011, 13:30 UTC

Lovell seminar room 3.225 (SPDO, University of Manchester)

Andreas Wicenec (ICRAR), Anna Scaife (Dublin institute for advanced studies), Brian Jeffs (Brigham Young University), Duncan Hall (SPDO), Gerry Harp (SETI), Ludwig Schwardt (SKAZA), Ronald Nijboer (ASTRON), Sanjay Bhatnagar (NRAO), Tim Cornwell (CSIRO)

This workshop focuses on progress in algorithms, software and computing aimed at addressing the challenges of calibration and imaging of the Square Kilometre Array (SKA), its pathfinders and other major new radio telescopes such as EVLA, ALMA, LOFAR, ASKAP, MeerKAT, ATA, FAST, MWA, LWA, PAPER/HERA, eMERLIN, GMRT, WSRT/APERTIF, etc.

The workshop is the 6th in a series, following the successful previous meetings held in Dwingeloo (2005), Cape Town (2006), Perth (2008), Socorro (2009), and Dwingeloo (2010).

We welcome presentations on topics from the previous meetings.  In addition, we welcome relatively new areas of investigation such as all sky imaging, wide field polarisation, and end to end pipeline processing.

We prefer attendees to give either a talk or present a poster.  At future meetings, invitations will be first sent to those who have done so.

The formal meeting will run 4 days from Monday July 25 to Thursday July 28.  As at previous workshops, on Friday morning there will be informal discussions of selected topics arising from the workshop.

01: Link to CALIM 2005 CALIM 2005

02: Link to CALIM 2006 2: Link to CALIM 2006

03: Link to CALIM 2008 3: Link to CALIM 2008

04: Link to CALIM 2009 4: Link to CALIM 2009

05: Link to CALIM 2010 5: Link to CALIM 2010

06: Link to Novotel Hotel Manchester 60 rooms have been block booked

07: Map of walk from Novotel to Alan Turing Walking map from Hotel Novotel to Alan Turing Bldg

08: 2011-07-21 Abstracts book with details 08: 2011-07-21 Abstracts book with details

Monday, 25 July

07:30 → 08:00 Registration: Including tea and coffee

08:00 → 08:05 Welcome

Speaker: Kobus Cloete on behalf of Richard Schilizzi

08:05 → 08:10 Logistics

Speaker: Duncan Hall

08:10 → 12:00 Status updates

08:10 Introductions

Speaker: Dr Tim Cornwell (CSIRO/CASS)

video 2011-07-25 09:10 Status Updates and Introductions Session Chair Tim Cornwell (CSIRO/CASS)

08:15 LOFAR Status Update

Status and plans of the LOFAR radio telescope

Speaker: Dr Ronald Nijboer (ASTRON)

slides Nijboer_calim2011.pdf

video 2011-07-25 09:15 LOFAR Status Update Speaker Ronald Nijboer (ASTRON)

08:50 Status of ASKAP

I will discuss the status of ASKAP, including the latest measurements on the Phased Array Feeds, and the many other steps towards the goal of testing on 6 antennas by the end of 2011.

Speaker: Dr Tim Cornwell (CSIRO/CASS)

slides Cornwell_CALIM2011.pdf

video 2011-07-25 09:50 Status of ASKAP Speaker Tim Cornwell (CSIRO/CASS)

09:30 MeerKAT update

MeerKAT update

Speaker: Dr Ludwig Schwardt (SKA ZA)

slides MeerKAT_Status_Update_Calim2011_schwardt.pdf

video 2011-07-25 10:30 MeerKAT update Speaker Ludwig Schwardt (SKA ZA)

10:10 Tea and Coffee

10:35 MWA update

MWA update

Speaker: Dr Daniel Mitchell (SAO)

slides 20110725_Mitchell_MWA.pdf

video 2011-07-25 11:35 MWA update Speaker Daniel Mitchell (SAO)

11:15 EVLA status and update

EVLA status and update including development of CASA: algorithms and for use on High Performance Computers.

Speaker: Dr Sanjay Bhatnagar (NRAO)

slides Bhatnagar_CALIM2011.pdf

video 2011-07-25 14:00 EVLA status and update Speaker Sanjay Bhatnagar (NRAO)

12:00 → 13:00 Lunch

13:00 → 16:00 Calibration 1of2

13:00 Beam quality and stability of PAF systems

Phased array feed (PAF) systems provide a much larger field-of-view than single pixel feeds thereby considerably improving the survey speeds achievable with reflector dishes. PAF systems also provide the flexibility to adaptively change the primary beams of the telescope, for example for interference suppression. One of the main worries from a self-calibration and imaging perspective is that this flexibility reduces the quality and stability of the beam pattern. In this presentation, I will present a framework to assess these issues. Using EM-simulations of the APERTIF system, I show that PAF systems can achieve similar performance as classical feed systems.

Speaker: Dr Stefan Wijnholds (ASTRON)

slides 20110725_CalIm_Wijnholds.pdf

video 2011-07-25 12:20 Beam quality and stability of PAF systems Speaker Stefan Wijnholds (ASTRON)

13:40 Main beam representation in non-regular arrays

The main differences between patterns in non-regular arrays and in continuous apertures will be reviewed. Mathematical tools generally used for radiation from apertures will be applied to non-regular arrays and the convergence versus number of terms will be illustrated. Comments will be provided on the applicability of the method in the presence of mutual coupling.

Speaker: Prof. Christophe Craeye (Université Catholique Louvain)

slides calim_craeye_public.pdf

video 2011-07-25 14:40 Main beam representation in non-regular arrays Speaker Christophe Craeye (Université Catholique Louvain)

14:20 Tea and Coffee

14:40 Fundamental Limit of Polarimetric Calibratability

There is a widespread myth in radio astronomy that says that complete knowledge of antenna beam gains will lead to perfect calibration and hence, perfect images. I show that this is almost never true in practice. I will present a fundamental bound on the error of fully calibrated data. This "error bar" on the final image shows that even for completely known gains, there is, in addition to thermal noise, polarimetric noise due to the ill-conditioning of the Jones matrix of the telescope. This polarimetric noise affects not only polarized sources, but also unpolarized sources, thus affects all imaging. This effect can be seen as the fundamental limit of calibration.

Speaker: Dr Tobia Carozzi (Onsala Space Observatory, Chalmers University, Sweden)

slides Carozzi_CALIM2011.pdf

video 2011-07-25 15:40 Fundamental Limit of Polarimetric Calibratability Speaker Tobia Carozzi (Onsala Space Observatory)

15:20 Calibration pipelines for ASKAP

The Australian Square Kilometre Array Pathfinder (ASKAP) is fundamentally a real-time telescope given the data rate from the correlator to the central processor of 2.5 GB/s. The calibration pipelines have to keep up with observations and produce a solution on time scales shorter than the time required for calibration data acquisition. We have previously prototyped the calibration pipelines within the ASKAPsoft's master-worker framework. The prototype implements a straight forward least square solver with an option to have distributed calculation of the normal equations. It is also easily configurable for different types of measurement equation (e.g. to get gain or polarisation leakage pipelines). However, the performance of the first prototype was found inadequate for real-time operations. Here we present a new approach based on pre-computing the sums in the expression for the elements of normal matrix during the first iteration. This approach led to a five times higher performance of the gain calibration while the overall structure of the code had been retained.

Speaker: Dr Maxim Voronkov (CSIRO Astronomy and Space Science)

slides MVoronkovASKAPCalib.pdf

video 2011-07-25 16:20 Calibration pipelines for ASKAP Speaker Maxim Voronkov (CSIRO Astronomy and Space Science)

Tuesday, 26 July

07:30 → 08:00 Tea and Coffee

08:00 → 11:40 Calibration 2of2

08:00 Single dish polarization calibration results using KAT-7

This presentation will show early single dish polarization calibration results using KAT-7. Two methods of observation, namely radial and raster scans, are compared. Direction dependent beam pattern results as well as on-axis calibration solutions for leakages will be shown.

Speaker: Dr Mattieu de Villiers (SKA SA)

slides devilliers_calim2011.pdf

video 2011-07-26 09:00 Single dish polarization calibration results Speaker Mattieu de Villiers (SKA SA)

08:40 Fringe Fitting in Casa

In general, VLBI networks need to calibrate with additional terms over complex gain, namely, delay and rate. This type of calibration, known as fringe fitting has also been found beneficial and in some cases necessary with interferometers other then traditional VLBI networks. The use of AIPS for fringe fitting is well established however many of the new or upgraded telescopes use the Measurement Set as their native data format and employ Casa and other software built on top of Casacore for processing. Under the ALBiUS project, a project has begun to explore and develop a Casa implementation of fringe fitting. This talk will report on its progress.

Speaker: Mr Stephen Bourke (JIVE)

slides Calim2011_FringeFitting_sbourke.pdf

video 2011-07-26 10:00 Fringe Fitting in Casa Speaker Stephen Bourke (JIVE)

09:20 Next-generation calibration: John Q. Astronomer's perspective

The reduction, calibration and imaging of data from the SKA and its pathfinders will necessarily be automated on dedicated high-performance computing facilities instead of manual operations performed on the desktop PC of the lucky astronomer, as has been the story so far. This is already partially realised by the LOFAR project which offers observers and analysts access to a dedicated data processing cluster. Until this paradigm shift is complete however, mildly adventurous radio astronomers may immediately reap the benefits of powerful next-generation algorithm and software developments, the installation, use and scripted operation of which are easier than ever. I will present here something of an 'uninvited review', detailing my experiences in expanding my radio astronomy software toolkit beyond the stalwart AIPS package. This expansion is driven by necessity as well as curiosity, and I will frame this talk in the context of my efforts to deal with EVLA observations of a particularly troublesome target field.

Speaker: Dr Heywood Ian (University of Oxford)

slides Heywood_EVLA_DDE_CALIM2011.pdf

video 2011-07-26 09:40 Next-generation calibration: John Q. Astronomer's perspective Speaker Ian Heywood (University of Oxford)

10:00 Tea and Coffee

10:20 Solving for primary beams, pointing errors, and The Westerbork Wobble

I will present recent results on dealing with beam-related DDEs and the associated calibration artefacts in WSRT data. An algorithm conceptually similar to pointing selfcal, but amenable to any kind of beamshape (including empirical beamshapes specified as e.g. FITS files), and able to solve for parameters of the primary beam, has been implemented in MeqTrees. This has been applied to a range of test fields observed with and without deliberate pointing error. The conclusion is that mis-pointing can be reliably recovered, but the improvements to the resulting maps are marginal at best, which is probably due to an inadequate beam model. Better beam models are urgently needed. In addition, my results suggest some rather puzzling low-level pointing variations at the WSRT.

Speaker: Dr Oleg Smirnov (ASTRON)

slides Smirnov-CALIM2011-WesterborkWobble.pdf

video 2011-07-26 11:20 Solving for primary beams, pointing errors, and The Westerbork Wobble Speaker Oleg Smirnov (ASTRON)

11:00 Complex Factor Analysis

Many techniques for array processing assume either that the system has a calibrated array or that the noise covariance matrix is known. If the noise covariance matrix is unknown, training or other calibration techniques are used to find it. Here another approach to the problem of unknown noise covariance is presented. The factor analysis (FA) model is used to model the data. In order to make the theory applicable in telecommunication and radio astronomy, the model is extended to the case of complex numbers. The necessary mathematical tools for estimation, detection and performance analysis are derived. The maximum likelihood estimator for the FA model in the case of proper complex Gaussian distributed noise and signals is given. Two different iterative algorithms for finding the MLE of the model parameters are presented. The necessary iteration steps for an alternating least squares algorithm are also presented. This is an initial attempt for developing the factor analysis as a tool for signal processing. The multivariate nature of the theory makes it a good candidate for solving many problems in telecommunication and radio astronomy.

Speaker: Mr Ahmad Mouri Sardarabadi (Delft University of Technology)

slides Complex_Factor_Analysis.pdf

video 2011-07-26 12:00 Complex Factor Analysis Speaker Ahmad Mouri Sardarabadi (Delft University of Technology)

11:40 → 12:40 Lunch

12:40 → 15:50 Imaging 1of2

12:40 LOFAR Imaging Pipeline Status

In this contribution I will summarize the LOFAR imaging pipeline and give a status update. I will illustrate the performance of the various stages of the pipeline and discuss prospects for improvements and future developments.

Speaker: Dr George Heald (ASTRON)

slides heald_calim.pdf

video 2011-07-26 13:40 LOFAR Imaging Pipeline Status Speaker George Heald (ASTRON)

13:20 Rotation Measure Synthesis

I will describe the technique of Rotation Measure (RM) Synthesis for recovering polarisation images in Faraday Depth from multi-frequency radio data, and the parallels that exist with aperture synthesis. RM Synthesis will be a powerful tool for all pathfinder telescopes of the Square Kilometre Array (SKA), where it will be vital for magnetism science. In this talk I will describe specifically the current development of the RM Synthesis pipeline of the LOFAR telescope with reference to current early results. Advanced imaging and deconvolution algorithms for recovery of Faraday Depth structure are a natural extension to such pipelines and I will highlight the applicability of advanced techniques, particularly that of compressed sensing, for the recovery of features such as the so-called Faraday Caustics caused by rapid reversals of the magnetic field along the line of sight.

Speaker: Dr Anna Scaife (DIAS)

slides CALIM_Scaife.pdf

video 2011-07-26 14:20 Rotation Measure Synthesis Speaker Anna Scaife (DIAS)

14:00 Tea and Coffee

14:30 Algorithm scaling for ASKAP and SKA

Algorithms for data processing for ASKAP have been forced to change as the scale of processing has increased. Since 2007, we have implemented 4 substantial changes to our wide field imaging algorithms. We expect to require one or two more changes for full ASKAP processing. Without these changes, we could not expect to meet the resource constraints of 10000 cores each with 2GB memory. The same will be true for SKA but only more so - memory per core will continue to shrink and the programming model with change, probably substantially over the pre-construction and construction phases. I will describe our ASKAP experience and speculate over changes required for SKA-scale processing.

Speaker: Dr Tim Cornwell (CSIRO/CASS)

slides skacalim2011.pdf

video 2011-07-26 15:30 Algorithm scaling for ASKAP and SKA Speaker Tim Cornwell (CSIRO/CASS)

15:10 Recent imaging results with wide-band EVLA data, and lessons learnt so far

The EVLA has been producing wide-band data for science observations since Fall 2010. In this talk, I will present imaging results that demonstrate our current wide-band and wide-field imaging capability. The increased dynamic-range due to wide-band imaging (using ms-mfs) and the accuracy at which both spatial and spectral reconstructions have been made will be demonstrated via the 3C286 field as well as several Galactic supernova fields with plenty of compact and extended emission across a wide field-of-view. The effect of the wide-band primary beam and its implications will be discussed, with an example that shows the near absence of a first null in the wide-band sensitivity pattern, and non-trivial sensitivity out to very wide field of view. Performance bottlenecks and factors currently limiting the dynamic range in complicated fields with extended emission will also be discussed.

Speaker: Dr Urvashi Rau (National Radio Astronomy Observatory)

slides URV_EVLAWideBand.pdf

video 2011-07-26 16:10 Recent imaging results with wide-band EVLA data Speaker Urvashi Rau (National Radio Astronomy Observatory)

Wednesday, 27 July

07:30 → 08:00 Tea and Coffee

08:00 → 11:10 Imaging 2of2

08:00 Warped snapshot imaging for low-frequency dipole arrays

I will discuss a strategy for adapting the CUDA-based, real-time, MWA calibration and imaging approach for use in an off-line iterative deconvolution system (codenamed CUWARP). This approach is attractive for compact, low frequency arrays, since the image re-sampling required to deal with time-dependent ionospheric distortions is also used to correct for the wide-field w-term effects. The system is being considered for the proposed Large Aperture Experiment to Detect the Dark Age (LEDA), which involves full broadband correlation of the 256 element LWA1 station. I will also discuss potential convolutional gridding options for the system.

Speaker: Dr Daniel Mitchell (Harvard-Smithsonian Center for Astrophysics)

slides 20110727_Mitchell_warpedsnapshots.pdf

video 2011-07-27 09:00 Warped snapshot imaging for low-frequency dipole arrays Speaker Daniel Mitchell (Harvard-Smithsonian CfA)

08:40 Fast W-Projection Gridding on GPUs

In this talk, I will report on a novel way to implement W-projection gridding on GPUs. This method is several times faster than existing GPU gridders, because it needs significantly fewer accesses to the relatively slow GPU memory. I will show how gridding on an eight-GPU system is typically 250 times faster than on a regular dual-CPU server, and over 20 times more energy-efficient. The algorithm is programmed in both CUDA and OpenCL, which I will briefly compare. I will also compare and explain the enormous performance differences between GPUs from Nvidia and AMD.

Speaker: Dr John Romein (Stichting ASTRON)

slides romein.pdf

video 2011-07-27 09:40 Fast W-Projection Gridding on GPUs Speaker John Romein (Stichting ASTRON)

09:20 Tea and Coffee

09:50 Imaging using the GPU

Techniques like the Minimum Variance Distortionless Response (MVDR) beamformer enable us to minimize the undesired power entering a sensor array via its sidelobes, but have been deemed computationally expensive to use. We present the results of an implementation of the MVDR algorithm on the GPUS as well as the modifications we made to the classic MVDR algorithm to work with interferometric data.

Speaker: Mr Vamsi Krishna Veligatla (Kapteyn Institute (University Of Groningen))

slides calim2011_pres1.pdf

video 2011-07-27 10:50 Imaging using the GPU Speaker Vamsi Krishna Veligatla (Kapteyn Institute (University Of Groningen))

10:30 Deep observations with LOFAR

During this talk, I will present the progress and current status of LOFAR Epoch of Reionization experiment data reduction pipeline. I will discuss the techniques and algorithms used and show some early results of deep observations for the time being for LOFAR. The image quality, however, is still not perfect and significant improvements can be expected in the months ahead using improved knowledge of the effects of the LOFAR station beams.

Speaker: Dr Panos Labropoulos (ASTRON & Kapteyn Institute)

slides calim2011_panos.pdf

video 2011-07-27 11:30 Deep observations with LOFAR Speaker Panos Labropoulos (ASTRON & Kapteyn Institute)

11:10 → 12:30 Lunch

12:30 → 16:30 Field trip to Jodrell Bank

Thursday, 28 July

07:30 → 08:00 Tea and Coffee

08:00 → 11:30 Data

08:00 Looking beyond the LOFAR central processor.

The LOFAR Phase II cluster is finished, marking perhaps the final chapter in the roll-out of LOFAR. Over the last 6 years or so we've built four incarnations of the LOFAR central processor (and we have become exceedingly efficient at it). During this time the design of the central processor evolved from a loosly coupled collection of processing boxes to a tightly knit and highly integrated system. Looking back there are several important lessons to learn. Some of these may significantly impact algorithm development. I'll give a short overview of the lessons learned and give a quick look into future technologies that may impact SKA processing.

Speaker: Mr Chris Broekema (ASTRON)

slides calim_2011_broekema.pdf

video 2011-07-28 09:00 Looking beyond the LOFAR central processor Speaker Chris Broekema (ASTRON)

08:40 GALFACTS - dealing with large data volumes

The Galactic ALFA Continuum Survey (GALFACTS) is a large-area spectro-polarimetric survey on the Arecibo Radio telescope. It uses the seven-beam focal plane feed array receiver system (ALFA) to carry out an imaging survey project of the 12,700 square degrees of sky visible from Arecibo centred at 1.4 GHz. The raw data produced by the spectrometer creates 56 digital data streams (seven beams, four polarization states and two frequency bands) each with 4096 spectral channels sampled at 1 millisecond. This produced terabyte sized data sets, the data processing pipeline raises considerable challenges. Here we discuss some of the aspects of the computation and calibration of such a dataset.

Speaker: Dr Samuel George (MRAO)

slides galfacts_sgeorge.pdf

video 2011-07-28 09:40 GALFACTS - dealing with large data volumes Speaker Samuel George (MRAO)

09:20 Tea and Coffee

09:50 Status of the SDMv2

SDMv2 is an instance of a data model intended to represent the data produced for a broad variety of types of radio-telescopes (ref. presentation given at CALIM 2010). The model is still under development on several fronts, amongst them a formalization of its theory. Using the use-case of EMBRACE I'll explain why it has been necessary to go a step forward beyond the standard relational data model. The data model involves the definition of data-types. I will describe how some of these are elaborated to implement their algebras. Building logical structures and their associated algebraic structures leads, de facto, to the definition of a domain specific language. For our purpose this language is dedicated at the meta-model level to experimental physics through the concepts of physical quantities and measurements. Profiling the concepts that we need in radio astronomy leads to the theory, an assembly of data models. The net result with this approach is: a) to offer the possibility to develop highly expressive codes using a concise terminology b) to make these codes type-safe and thus providing very robust implementations, and c) to achieve good efficiency at run-time. These different aspects are essential to make easier the maintainability of these codes. The formalization highlights the properties of the structures and the different ways they can be glued together. This should help when optimizing these codes to specific computer architectures in order to maximize their performances in different infrastructure environments with problems of different sizes. This work is done within the context of the ALMA CIPT and taking the EMBRACE prototype as a use-case for experimentation.

Speaker: Dr Francois Viallefond (OBSPM/CNRS)

slides fvcalim2011.pdf

video 2011-07-28 10:50 Status of the SDMv2 Speaker Francois Viallefond (OBSPM/CNRS)

10:30 How small can you get?

e-MERLIN and ALMA are currently operating with 5-15 antennas, up to about 10,000 spectral channels and integration times around one second. The high resolution is useful for debugging during commissioning but produces data sets up to a TB which take hours to process on a desktop at the high end of what is currently commonly available. These data volumes are, of course, but a drop in the ocean compared with SKA data or even the full operation of ALMA. Conventional expressions for time- and bandwidth-smearing (including phase-rate and spectral index effects) can be used to estimate how much averaging is possible in imaging. I will present progress towards guidelines for how much data averaging is worthwhile without degradation, for various kinds of data, during calibration, imaging and continuum subtraction.

Speaker: Dr Anita Richards (UK ARC Node, JBCA)

slides AMSR_CALIM_FoV.pdf

video 2011-07-28 11:30 How small can you get? Speaker Anita Richards (UK ARC Node, JBCA)

11:00 ALMA user software tools - other than CASA

Demo of user software tools for ALMA other than CASA, i.e. any or all of Portal and Helpdesk system OT including sensitivity calculator and line list OST

Speakers: Dr Anita Richards (UK ARC Node, JBCA), Dr Ian Heywood (University of Oxford)

video 2011-07-28 12:00 ALMA user software tool Speaker Ian Heywood (University of Oxford)

11:30 → 12:30 Lunch

12:30 → 16:00 Discussions

12:30 "More research required": a dialogue

This is proposed as an interactive dialogue. I'll describe the imaging requirements for SKA Phase 1. I'll present a suggested framework to address the research and development activities required to work towards delivering the SKA Phase 1 requirements. I'll seek a group consensus as to the priorities of these research and development activities.

Speaker: Mr Duncan Hall (SPDO)

slides Includes summary of workshop discussion

video 2011-07-28 13:30 "More research required": a dialogue Speaker Duncan Hall (SPDO)

14:00 Tea and Coffee

14:30 Lowering the Threshold

The purpose of MeqTrees is "to increase the rate of evolution of data processing software", e.g. for 3rd generation calibration (3GC) of the new radio telescopes. The idea is to reduce the implementation time for new ideas by several orders of magnitude, from years to hours. Moreover, this should be within the reach of a large group of people around the world, who would also be able (and motivated) to exchange results and processing scripts with a minimum of hassle. The first capability, i.e. implementing new ideas, has been amply demonstrated by people like Smirnov, Willis, Yatawatta, Heywood and Labropoulos. However, despite the spectacular quality of their results, and the clear urgency of 3GC, the threshold to joining in the fun has proved a little high for less determined individuals. This talk describes the MeqWizard tool, which endeavours to address the problem.

Speaker: Jan Noordam (ASTRON)

slides CALIM_2011_Noordam.pdf

video 2011-07-28 15:30 Lowering the Threshold Speaker Jan Noordam (ASTRON)

16:00 → 16:20 Workshop Summary

Speaker: Dr Anna Scaife (DIAS)

slides CALIM2011_summary.pdf

video 2011-07-28 16:30 Workshop Summary Speaker Anna Scaife (DIAS)