SRC Data Storage Pilot

28 Nov 2019, 14:04
19m
Chicago

Chicago

Talk SKA Regional Centres SKA Regional Centres (session)

Speaker

Markus Dolensky (ICRAR)

Description

Provisioning long-term storage capacity will be a core function of SKA regional centres (SRCs). Commercial Cloud storage accumulates significant costs over time and typical HPC processing scenarios require moving the data products elsewhere for preservation. The same applies to the storage of large test and simulation data sets, which is of immediate interest prior to SKA construction. Later, during the operational phase, Observatory Data Products (ODPs) and derived Advanced Data Products (ADPs) will be made available through SRCs only.

The new compute cluster and storage pilot at SHAO provides an opportunity to work with a prospective SRC facility already during the SKA design phase. We will report on an example workflow implemented by ICRAR and originally deployed on the currently most powerful supercomputer Summit at ORNL. It is now redeployed at SHAO for more general use and features a large-scale imaging pipeline, starting from the generation of simulated visibilities all the way to the analysis of an image cube.

Suggested duration 20 min

Primary author

Co-authors

Tao An (Shanghai Astronomical Observatory) Baoqiang Lao (Shanghai Astronomical Observatory, Chinese Academy of Sciences) Rodrigo Tobar (International Centre for Radio Astronomy Research) Mr Dave Pallot (ICRAR) Andreas Wicenec (ICRAR)

Presentation materials