With modern radio astronomy already recognised as a data-intensive science, the SKA has been widely identified as one of the major “Big Data” challenges for the next decade.
An as exa-scale computing challenge, the practicalities of deploying a research infrastructure capable of science with the SKA in terms of computing, storage, networking and analytics have already attracted the attention of major commercial computing hardware and platform providers. However, the associated challenges of automated processing, workflow management and machine learning applications at scale that are required at a software level have wider applications for a broader community of users, both academic and commercial.
A distributed and federated European SKA Regional Centre, therefore, can provide a platform for a European and nationally focused partnership with industry for the continued development of these core technologies and hence a clear route to delivering impact and return. While large scale storage, networking and computing facilities play to the strengths of larger companies, the European Science Data Centre (ESDC) network can act as a catalyst for scientific excellence in Big Data and High Performance Computing, with ESDC centres having the potential to enhance the business models of small and medium-sized enterprises (SMEs) that provide their services on smaller scales and in specialised areas, especially as Big Data expands beyond the Scientific Domain and into our daily lives.