Kavli Affiliate: Richard Dubois
| First 5 Authors: Edward Karavakis, Wen Guan, Zhaoyu Yang, Tadashi Maeno, Torre Wenaus
| Summary:
The Vera C. Rubin Observatory will produce an unprecedented astronomical data
set for studies of the deep and dynamic universe. Its Legacy Survey of Space
and Time (LSST) will image the entire southern sky every three to four days and
produce tens of petabytes of raw image data and associated calibration data
over the course of the experiment’s run. More than 20 terabytes of data must be
stored every night, and annual campaigns to reprocess the entire dataset since
the beginning of the survey will be conducted over ten years. The Production
and Distributed Analysis (PanDA) system was evaluated by the Rubin Observatory
Data Management team and selected to serve the Observatory’s needs due to its
demonstrated scalability and flexibility over the years, for its Directed
Acyclic Graph (DAG) support, its support for multi-site processing, and its
highly scalable complex workflows via the intelligent Data Delivery Service
(iDDS). PanDA is also being evaluated for prompt processing where data must be
processed within 60 seconds after image capture. This paper will briefly
describe the Rubin Data Management system and its Data Facilities (DFs).
Finally, it will describe in depth the work performed in order to integrate the
PanDA system with the Rubin Observatory to be able to run the Rubin Science
Pipelines using PanDA.
| Search Query: ArXiv Query: search_query=au:”Richard Dubois”&id_list=&start=0&max_results=3