Hadoop分布式存储架构的性能分析Performance analysis of Hadoop distributed storage architecture
王凌晖,解云月,周美华
摘要(Abstract)：
近几十年,随着互联网的发展,数据正在呈爆炸式增长。为了保存和处理这些数据,传统的IT架构已经满足不了,云计算应运而生。Hadoop是云计算下的一种分布式处理的软件架构,能够对大数据进行存储和处理,而它最核心的两大组件就是HDFS和Map Reduce,分别实现了分布式文件存储和分布式并行任务处理。文中,将基于PEPA对HDFS进行建模分析,并对HDFS的性能进行评估。
关键词(KeyWords)： Hadoop;云计算;分布式存储;HDFS;PEPA;性能评估
基金项目(Foundation): 国家自然科学基金资助项目(61472343)~~
作者(Author): 王凌晖,解云月,周美华
DOI: 10.16652/j.issn.1004-373x.2018.18.021
参考文献(References)：
- [1]董新华,李瑞轩,周湾湾,等.Hadoop系统性能优化与功能增强综述[J].计算机研究与发展,2013,50(z2):1-15.DONG Xinhua,LI Ruixuan,ZHOU Wanwan,et al.Performance optimization and feature enhancements of Hadoop system[J].Journal of computer research and development,2013,50(S2):1-15.
- [2]李成华,张新访,金海,等.Map Reduce:新型的分布式并行计算编程模型[J].计算机工程与科学,2011,33(3):129-135.LI Chenghua,ZHANG Xinfang,JIN Hai,et al.Map Reduce:a new programming model for distributed parallel computing[J].Computer engineering&science,2011,33(3):129-135.
- [3]郝树魁.Hadoop HDFS和Map Reduce架构浅析[J].邮电设计技术,2012(7):37-42.HAO Shukui.Brief analysis of the architecture of Hadoop HDFS and Map Reduce[J].Designing techniques of posts and telecommunications,2012(7):37-42.
- [4]DING J.A comparison of fluid approximation and stochastic simulation for evaluating content adaptation systems[J].Wireless personal communications,2015,84(1):231-250.
- [5]DING J,HILLSTON J,LAURENSON D I.Evaluating the response time of large scale content adaptation systems using performance evaluation process algebra[C]//Proceedings of IEEE International Conference on Communications.Cape town:IEEE,2010:1-5.
- [6]WANG H,LAURENSON D I,HILLSTON J.Evaluation of RSVP and mobility-aware RSVP using performance evaluation process algebra[C]//Proceedings of IEEE International Conference on Communications.Beijing:IEEE,2008:192-197.
- [7]潘耘,王励成,余镇危,等.基于随机进程代数的Overlay网络上的Qo S性能评价[J].北京邮电大学学报,2009,32(z1):129-133.PAN Yun,WANG Licheng,YU Zhenwei,et al.Based on stochastic process algebra overlay network Qo S performance evaluation[J].Journal of Beijing University of Posts and Telecommunications,2009,32(S1):129-133.
- [8]黄金才,陈洒然,程光权.舰艇编队防空过程建模及分析[J].国防科技大学学报,2014,36(3):184-190.HUANG Jincai,CHEN Saran,CHENG Guangquan.Modeling and analysis of air defense process for warship formation[J].Journal of National University of Defense Technology,2014,36(3):184-190.
- [9]李宽.基于HDFS的分布式Namenode节点模型的研究[D].广州:华南理工大学,2011.LI Kuan.Research of the model of distributed Namenodes in HDFS[D].Guangzhou:South China University of Technology,2011.
- [10]HILLSTON J.Fluid flow approximation of PEPA models[C]//Proceedings of Second International Conference on the Quantitative Evaluation of Systems.Torino:IEEE,2005:1-10.