[1]杨阳,张红梅,王爽.基于科学工作流的通用卫星数据处理调度系统[J].计算机技术与发展,2024,34(06):164-170.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0075]
　YANG Yang,ZHANG Hong-mei,WANG Shuang.A Universal Satellite Data Processing and Scheduling System Based on Scientific Workflow[J].,2024,34(06):164-170.[doi:10.20165/j.cnki.ISSN1673-629X.2024.0075]
点击复制

基于科学工作流的通用卫星数据处理调度系统()

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
34
期数:
2024年06期
页码:
164-170
栏目:
新型计算应用系统
出版日期:
2024-06-10

文章信息/Info

Title:
A Universal Satellite Data Processing and Scheduling System Based on Scientific Workflow
文章编号:
1673-629X(2024)06-0164-07
作者:
杨阳1; 2; 张红梅1; 王爽1
1. 中国科学院 高能物理研究所,北京 100049;2. 中国科学院大学,北京 100049
Author(s):
YANG Yang1; 2; ZHANG Hong-mei1; WANG Shuang1
1. Institute of High Energy Physics,Chinese Academy of Sciences,Beijing 100049,China;2. University of Chinese Academy of Sciences,Beijing 100049,China
关键词:
天文卫星数据处理; 数据产品; 科学工作流; 任务调度; 自动化
Keywords:
astronomical satellite data processing; data products; scientific workflow; task scheduling; automation
分类号:
TP391
DOI:
10.20165/j.cnki.ISSN1673-629X.2024.0075
摘要:
天文卫星产生的海量观测数据下传至地面后,需要经过一系列的处理流程生成可供科学研究人员的数据产品。传统数据处理方式需要耗费大量的时间和人力成本,同时存在缺乏灵活性、任务管理困难及执行效率低下等缺陷。 基于上述问题,为实现卫星数据产品生成的自动化和规范化,该文研究并设计了一种基于科学工作流的通用卫星数据处理调度系统。 通过封装卫星数据处理算法,结合科学工作流、可视化编辑等相关技术,实现卫星数据处理的自动化和可配置,用户更新需求时无需修改代码,只需重新配置参数;利用可拔插组件设计机制实现系统的通用性,满足不断扩增的卫星的不同数据处理需求。 根据已有卫星的数据处理需求进行应用测试,应用实例表明,该系统能够实现卫星数据的自动化处理,并且能够极大地提高处理流程的灵活性、可维护性和执行效率,有利于科学研究人员根据自己的需求获取卫星数据产品。
Abstract:
After the massive observation data generated by astronomical satellites is transmitted to the ground,it needs to go through a series of processing processes to generate data products that can be used by scientific researchers. Traditional data processing methods require a lot of time and labor costs,and there are shortcomings such as lack of flexibility,difficult task management,and inefficient exe-cution. Based on the above problems,in order to realize the automation and standardization of satellite data product generation,we study and design a general satellite data processing and scheduling system based on scientific workflow technology. By encapsulating satellite data processing algorithms,combined with scientific workflow,visual editing and other related technologies,to realize the automation and configurability of satellite data processing,users do not need to modify the code when updating requirements,only need to reconfigure pa-rameters. The pluggable component design mechanism is used to achieve a universal type of system to meet the different data processing needs of the expanding number of satellites. Application tests are carried out according to the data processing requirements of existing sat-ellites,and the application examples show that the system can realize the automatic processing of satellite data,and can greatly improve the flexibility,maintainability and execution efficiency of the data processing process,which is conducive to scientific researchers to obtain satellite data products according to their own needs.

相似文献/References:

[1]谢孔树 黄晓冬 王存仁[] 温玮.基于产品机制的环境数据重用方法研究[J].计算机技术与发展,2012,(04):210.
　XIE Kong-shu,HUANG Xiao-dong,WANG Cun-ren,et al.Research on Environment Data Reuse Method Based on Production Mechanism[J].,2012,(06):210.

更新日期/Last Update: 2024-06-10