Introduction
e-VLBI (e-Very Long Baseline Interferometry) provides a new astronomical observation method that combines long baseline interferometry technology with high-speed network. And in 2007, Europe, Australia, and China jointly conducted the first e-VLBI test demonstration spanning three continents at a rate of 256Mbit/s and obtained real-time stripes through JIVE related processors. The Chinese VLBI Network (CVN) (see Figure 1) has successfully applied e-VLBI technology to the Chang'e series satellite VLBI orbit measurement and positioning system in the first and second phases of China's lunar exploration project.
Fig.1 Chinese VLBI NETWORK (CHEN Z et al, 2015)
As one of the largest, the Square Kilometer Array Radio Telescope (SKA) data collection will cover over 1 square kilometer, and is dedicated to the exploration of the universe in astronomy, radio, information science, and system science. SKA official member states include China, and another ten observer states. The SKA1 (2021-2028) will cover the first 10% of the entire plan with a second phase (SKA2) finishing the remaining 90% work. At present, Chinese scientists have successfully developed SKA's first antenna prototype and the first regional center prototype. According to the scale of SKA data flow, the SKA1 data generation and dissemination could reach 300PB per year and up to EB level at the end of the SKA2 stage.
Challenges
Through the life cycle of massive data management, the main challenges within the e-VLBI observation and the SKA program include:
(1) Cross-regional, long-distance, high-speed real-time transmission
The e-VLBI experiment carries out real-time data transmission and real-time related processing for observation. In the SKA program, there’re many regional centers on several continents around the world (Figure 2). And for data collection, high network link bandwidth together with excellent network stability are urgently needed.
Figure 2 SKA regional center distribution and network connection
(2) Enhanced computing capability challenges
It is estimated that the computing power required for the SKA1 data processing may reach 500 Pflops (AN et al, 2018). Thus, the in-depth analysis and processing of SKA data will be completed in regional centers distributed on several continents with great challenges to the computing capability of regional centers for online data aggregation, arrangement, scheduling, and application.
(3) Backup, disaster recovery and long-term storage of massive data demands
For example, the MWA (Murchison Wide Field Array (MWA) is a low-frequency radio telescope located in the Murchison Radio Astronomy Observatory (MRO) in Western Australia with 24PB data archived. In the first phase of the sky survey, archive data volume has reached more than 1PB with a second phase of estimated data capture up to 6.5 PB. And this is only 1% of the SKA program. Obviously, the demand for massive data curation is super huge, and no-single data center can manage it without international cooperation in the long run.
CSTCloud engagement
Facing the needs of the major international scientific plans and projects, China Science and Technology Cloud (CSTCloud, www.cstcloud.net) keeps on providing a full range of services and supports based on high-quality international bandwidth and computing and storage resources for many years. A professional 7×24-hour operation and maintenance team ensure the stability and security of the network services. According to the statistics from the Ministry of Industry and Information Technology, P.R.C. (2020), CSTCloud's international network performance has ranked the top from Jan. to Sep. 2020, among all the domestic network backbone operators, with lower data transfer delay and less data loss.
In order to better meet the needs of large-scale real-time data transmission for high-quality and long-distance transmission in the international science plan, CSTCloud has launched the "Wide Area Network (WAN) Data Transmission Optimization Service", a self-developed wide area network transmission performance optimization system. To improve the WAN data transmission performance, technics deployed includes SD-WAN technology and multiple network optimization technologies (i.e., transmission protocol optimization, network coding, and data compression). The system can significantly increase the download rate of the wide area network, and reduce the end-to-end transmission delay, with improved data transmission performance.
Progress and Prospect
Based on this typical use case, CSTCloud has successively supported the first real-time VLBI experiment on China, Europe and Australia led by the Shanghai Astronomical Observatory of the Chinese Academy of Sciences, and provided strong technical support for the "Chang'e-1" lunar exploration satellite VLBI orbit measurement system. Based on the joint efforts, the continental data transfer can reach upload and download peak value of 2.49Gbps and 4.43Gbps. Facing the on-going demand for massive data transfer and data management, the cooperation between CSTCloud and e-VLBI China together with her international partners will continue in the long run.
Bibliography
[1] AN T,WU X,HONG X,et al. Science Applications and Challenges of SKA Big Data. In Bulletin of Chinese Academy of Sciences, 2018,33(8), pp.871-876. (In Chinese)
[2] CHEN Z, ZHENG WM, CHEN X. Construction and application of China e-VLBI Network. In Annals of Shanghai Astronomical Observatory Chinese Academy of Sciences, 2015, pp.136-147. (In Chinese)
[3]About Chinese VLBI Network. Available at: http://astro.sci.yamaguchi-u.ac.jp/eavn/aboutcvn.html
[4] Li, J. L., Guo, L., and Zhang, B., The Chinese VLBI network and its astrometric role. Giant Step: from Milli- to Micro-arcsecond Astrometry, 2008IAUS.248, pp. 182–185. doi:10.1017/S174392130801898X.
[5] Zheng W, Zhang J, Wang G, et al. Technical Progress of the Chinese VLBI Network, IVS 2018 General Meeting Proceedings, available at: https://ivscc.gsfc.nasa.gov/publications/gm2018/02_zheng_etal.pdf