[1]黄向平,彭明田,杨永凯.基于内存映射文件的复杂对象快速读取方法[J].计算机技术与发展,2020,30(03):82-87.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 016]
　HUANG Xiang-ping,PENG Ming-tian,YANG Yong-kai.Fast Reading Method of Complex Objects Based on Memory Mapping Files[J].Computer Technology and Development,2020,30(03):82-87.[doi:10. 3969 / j. issn. 1673-629X. 2020. 03. 016]
点击复制

基于内存映射文件的复杂对象快速读取方法()
分享到：

《计算机技术与发展》[ISSN:1006-6977/CN:61-1281/TN]

卷:
30
期数:
2020年03期
页码:
82-87
栏目:
智能、算法、系统工程
出版日期:
2020-03-10

文章信息/Info

Title:
Fast Reading Method of Complex Objects Based on Memory Mapping Files
文章编号:
1673-629X(2020)03-0082-06
作者:
黄向平1; 2; 彭明田1; 2; 杨永凯1; 2
1. 中国民航信息网络股份有限公司,北京 101318; 2. 民航旅客服务智能化应用技术重点实验室,北京 101318
Author(s):
HUANG Xiang-ping1; 2; PENG Ming-tian1; 2; YANG Yong-kai1; 2
1.TravelSky Technology Limited,Beijing 101318,China; 2.Key Laboratory of Intelligent Passenger Service of Civil Aviation,Beijing 101318,China
关键词:
内存数据库; 嵌入式数据库; 复杂对象共享读取; 内存映射; 内存分配器
Keywords:
NoSQL database; embedded database; shared reading of complex objects; memory mapping; memory allocator
分类号:
TP301
DOI:
10. 3969 / j. issn. 1673-629X. 2020. 03. 016
摘要:
当前内存数据库(NoSQL)、嵌入式数据库技术在高并发高性能系统中得到了广泛的应用,但对于复杂对象数据的 读取效率仍然低下,研究发现主要性能瓶颈有两个: 一是内核态与用户态间的内存拷贝,拷贝消耗时间与复杂对象的数据 量成线性增长;二是从数据库数据到运行时数据对象的格式转化操作,不但需要开辟新的内存空间存储运行时数据对象, 而且还需要解析原始数据并拷贝至新对象之中。为此,提出了一种基于内存映射(memory mapping)文件的复杂对象共享 读取方法。 借助内存映射文件与自定义内存分配器,实现了结构复杂的 C++ 标准模板库容器对象跨进程无拷贝、无格式 转化的共享,有效降低了数据读取延时。 通过性能的分析比较表明,与 NoSQL 内存数据库、嵌入式数据库比,读取性能效 率提升 10 倍以上。 再加上底层技术成熟稳定,复用了标准模板库,具有开发成本低、可维护性强、实用性高等优点,因此, 适用于高并发高性能的高可用后台服务系统。
Abstract:
The current NoSQL database and embedded database technology have been widely used in high concurrent and high performance systems,? ? ? ? but? the efficiency of reading complex object data is still low. It is found that there are two main performance bottlenecks. One is the memory copy between kernel mode and user mode,which consumes time linearly with the amount of data of complex objects. The other is format conversion operation from database data to run time data object,which not only needs to allocate new memory space to store runtime data objects,but also needs to parse the original data and copy them to new objects. Therefore,we propose a method of shared reading of complex objects based on memory mapping file. By means of memory mapping file and custom memory allocator,the sharing of container objects of C++ standard template library with complex structure without copy and format conversion across processes is realized to effectively reduce the data reading delay. The performance analysis and comparison showthat the efficiency of data reading is more than 10 times higher than that of NoSQL template library is reused,with the advantages of low development cost,strong maintainability and high practicability. Therefore,it is suitable for high concurrent and high performance high-availability back-end service system.

相似文献/References:

[1]石英伟 姜浩.实用内存数据库核心及其在VLR系统中的应用[J].计算机技术与发展,2008,(01):216.
　SHI Ying-wei,JIANG Hao.Core of Main Memory Database and Application in VLR System[J].Computer Technology and Development,2008,(03):216.
[2]鲁艳 马旭东 问治国 黄艳.基于Linux—MiniGUI嵌入式系统监控管理软件开发[J].计算机技术与发展,2008,(06):179.
　LU Yan,MA Xu-dong,WEN Zhi-guo,et al.Development of Embedded System Management Software Based on Linux- MiniGUI[J].Computer Technology and Development,2008,(03):179.
[3]夏铭 陆阳 盛业兴 李大勇.嵌入式数据库中利用Lex，Yacc设计SQL编译器[J].计算机技术与发展,2007,(11):121.
　XIA Ming,LU Yang,SHENG Ye-xing,et al.Design of SQL Compiler with Lex and Yacc Based on Embedded Database[J].Computer Technology and Development,2007,(03):121.
[4]范毅华 王璐.嵌入式油井供水压力流量监控系统[J].计算机技术与发展,2012,(09):193.
　FAN Yi-hua,WANG Lu.Monitoring and Control System of Pressure and Flow in Water Supply of Embedded Oil Well[J].Computer Technology and Development,2012,(03):193.
[5]王宏伟,方群,陈伟.基于内存数据库的OPC监测系统的设计与实现[J].计算机技术与发展,2013,(07):242.
　WANG Hong-wei,FANG Qun,CHEN Wei.Design and Realization of OPC Real-time Monitor System Based on Main Memory Database[J].Computer Technology and Development,2013,(03):242.
[6]时亚南. B+树算法的Java实现方法研究[J].计算机技术与发展,2015,25(01):111.
　SHI Ya-nan. Study on Java Implementation Method of B+ Tree Algorithm[J].Computer Technology and Development,2015,25(03):111.
[7]王寅峰[][][],王龙翔[]. 一种基于V3模型的内存数据库性能分析研究[J].计算机技术与发展,2015,25(06):77.
　WANG Yin-feng[][][],WANG Long-xiang[]. Research on Performance Survey in Memory Database Based on V3-model[J].Computer Technology and Development,2015,25(03):77.

更新日期/Last Update: 2020-03-10