基于大数据架构的地球物理观测数据管理系统

Geophysical Observation Data Management System Based on Big Data Architecture

摘要:

中国地震地球物理观测网络的核心业务软件“十五”地震前兆数据管理系统已运行十多年,随着观测数据量的增长,数据库的读取和统计查询的性能逐渐降低。此外,管理系统存在SQL注入和跨站脚本攻击等漏洞的安全问题,系统架构难以满足业务发展的需求。中国地震台网中心基于分布式组件的大数据架构,使用消息中间件和时序数据库等技术对管理系统进行重构,将原来存储于Oracle数据库中的地球物理观测数据、基础信息、产品数据等按照各自特点在时序数据库中建表,并将历史数据和增量数据都迁移到新的时序数据库,显著提升了系统读写和统计性能、数据的安全性和可用性(多副本技术)。分布式和容器化等技术的引入,增强了系统的高可用性,并可通过横向扩展进一步增加系统的处理能力。

Abstract:

The "Tenth Five-Year Plan" earthquake precursor data management system,as the core business software of the China Earthquake Geophysical Observation Network,has been in operation for more than ten years. With the increase in the amount of observation data,the performance quality of the database has gradually decreased. In addition,since there exist many security issues in the management system,the system architecture is outdated,which makes it difficult to meet the needs of business development. Based on the big data architecture,the China Earthquake Networks Center uses technologies such as message middleware and time-series database to reconstruct the management system. In this system tables are built in the database,and historical data and incremental data are migrated to the new time-series database,which significantly improves system performance,data security and availability. The introduction of technologies such as distributed and containerization has enhanced the high availability of the system,and can further increase the processing capacity of the system through horizontal expansion.