Kavli Affiliate: Feng Wang
| First 5 Authors: Ying Mei, Shoulin Wei, Feng Wang, Chen Wu, Rodrigo Tobar
| Summary:
The Square Kilometre Array (SKA) project is an international cooperation
project to build the largest radio telescope worldwide. Data processing is one
of the biggest challenges of building the SKA telescope. As a distributed
execution framework, the Data Activated Liu Graph Engine (DALiuGE) was proposed
to be one of the candidates for addressing the massive data of the SKA. DALiuGE
has many distinctive features, but its actual ability to handle scientific data
is still not evident. In this paper, we perform an objective evaluation of the
usability of DALiuGE concerning the execution performance, developer workload,
and implementation difficulty of porting the SAGECal to DALiuGE. The evaluation
results showed that the DALiuGE enables fast integration of astronomical
software, but there are significant differences in the efficiency of different
parallel granularities. Even with the deep optimization of the program, there
is still a gap between the current DALiuGE and the traditional MPI in execution
performance. Therefore, we come to a preliminary conclusion that the DALiuGE
has no performance advantage in batch processing of massive data, while it may
be more suitable for application scenarios with more customized computational
tasks, such as SKA science regional centers.
| Search Query: ArXiv Query: search_query=au:”Feng Wang”&id_list=&start=0&max_results=10