微信服务号

微信订阅号

2025-4-6- 5
Home > Archive>Volume 24, Issue 2, 2013 >331-342. DOI:10.3724/SP.J.1001.2013.04177
PDF HTML XML Export Cite reminder
Adaptive Metadata Load Balancing for Object Storage Systems
Author:
Affiliation:

  • Article
    • | |
  • Metrics
    • |
  • Reference [20]
    • |
  • Related [20]
    • |
  • Cited by [1]
    • | |
  • Comments
    Abstract:

    Object-Based storage is a good choice for large scale storage systems. Load balancing of metadata is important to improve the performance of I/O. The existing load balancing schemas cannot evenly distribute the accesses of metadata in a dynamic way. Moreover, the adaptability and fault-tolerance ability need to be improved. This paper presents an adaptable distributed load balancing of metadata (ADMLB) which is composed of basic load balancing algorithm (BBLA) and distributed incremental load balancing algorithm (IBLA). Specially, ADMLB first uses BBLA to distribute metadata loads according to the performances of the metadata servers and then uses IBLA to incrementally reorganize loads on each metadata server. ADMLB can evenly distribute loads between metadata servers and adapts well to the changes of loads. It also has good fault-tolerance ability, and locates metadata servers very quickly.

    Reference
    [1] Sun Microsystems, Inc. Lustre file system: High-Performance storage architecture and scalable cluster file system. 2007. https://www.sun.com/offers/docs/LustreFileSystem.pdf
    [2] Pawlowski B, Juszczak C, Staubach P, Smith C, Lebel D, Hitz D. NFS version 3: Design and implementation. In: Proc. of theUSENIX Summer 1994 Technical Conf. San Jose: USENIX, 1994. 137-152.
    [3] Morris JH, Satyanarayanan M, Conner MH, Howard JH, Rosenthal DS, Smith FD. Andrew: A distributed personal computingenvironment. Communications of the ACM, 1986,29(3):184-201. [doi: 10.1145/5666.5671]
    [4] Satyanarayanan M, Kistler JJ, Kumar P, Okasaki ME, Siegel EH, Steere DC. Coda: A highly available file system for a distributedworkstation environment. IEEE Trans. on Computers, 1990,39(4):447-459. [doi: 10.1109/12.54838]
    [5] Weil SA, Pollack KT, Brandt SA, Miller EL. Dynamic metadata management for petabyte-scale file systems. In: Proc. of the 2004ACM/IEEE Conf. on Supercomputing. Washington: IEEE Computer Society, 2004. [doi: 10.1109/SC.2004.22]
    [6] Wu JJ, Liu PF, Chung YC. Metadata partitioning for large-scale distributed storage systems. In: Proc. of the 3rd Int’l Conf. onCloud Computing. Washington: IEEE Computer Society, 2010. 212-219. [doi: 10.1109/CLOUD.2010.24]
    [7] Xiong J, Hu Y, Li G, Tang R, Fan Z. Metadata distribution and consistency techniques for large-scale cluster file systems. IEEETrans. on Parallel and Distributed Systems, 2011,22(5):803-816. [doi: 10.1109/TPDS.2010.154]
    [8] Brandt SA, Miller EL, Long DDE, Xue L. Efficient metadata management in large distributed storage systems. In: Proc. of the 20thIEEE/the 11th NASA Goddard Conf. on Mass Storage Systems and Technologies. Washington: IEEE Computer Society, 2003.290-298. [doi: 10.1109/MASS.2003.1194865]
    [9] Liu Z, Zhou XM. A metadata management method based on directory path. Ruanjian Xuebao/Journal of Software, 2007,18(2):236-245 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/18/236.htm [doi: 10.1360/jos180236]
    [10] Li WJ, Xue W, Shu JW, Zheng WM. Dynamic hashing: Adaptive metadata management for petabyte-scale file systems. In: Proc.of the 23rd IEEE/the 14th NASA Goddard Conf. on Mass Storage System and Technologies (MSST 2006). Washington: IEEEComputer Society, 2006.
    [11] Zhu Y, Jiang H, Wang J. HBA: Distributed metadata management for large cluster-based storage systems. IEEE Trans. on Paralleland Distributed Systems, 2008,19(4):1-14. [doi: 10.1109/TPDS.2008.33]
    [12] Hua Y, Zhu YF, Jiang H, Feng D. Scalable and adaptive metadata management in ultra large-scale file systems. In: Proc. of the28th IEEE Int’l Conf. on Distributed Computing Systems (ICDCS 2008). Washington: IEEE Computer Society, 2008. 401-408.
    [doi: 10.1109/ICDCS.2008.32]
    [13] Guo CC, Yan PL. A dynamic load balancing algorithm for heterogeneous Web server cluster. Chinese Journal of Computers, 2005,28(2):179-184 (in Chinese with English abstract).
    [14] Ni YZ, Lu GH, Huang YH. The solution of disk load balancing based on disk striping with genetic algorithm. Chinese Journal ofComputers, 2006,29(11):1995-2002 (in Chinese with English abstract).
    [15] Chen T, Xiao N, Liu F, Fu CS. Clustering-Based and consistent hashing-aware data placement algorithm. Ruanjian Xuebao/Journalof Software, 2010,21(12):3175-3185 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/3706.htm [doi: 10.3724/SP.J.1001.2010.03706]
    [16] Shan ZG, Dai QH, Lin C, Yang Y. Integrated schemes of Web request dispatching and selecting and their performance analysis.Ruanjian Xuebao/Journal of Software, 2001,12(3):355-366 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/12/355.htm
    [17] Jin C, Wei D, Low S. FAST TCP: Motivation, architecture, algorithms, performance. In: Proc. of the 23rd Conf. of the IEEECommunications Society (INFOCOM 2004). Washington: IEEE Computer Society, 2004. [doi: 10.1109%2fINFCOM.2004.1354670]
    [18] Schindelhauer C, Schomaker G. Weighted distributed hash tables. In: Paul S, ed. Proc. of the 17th ACM Symp. on Parallelism inAlgorithms and Architectures (SPAA 2005). Las Vegas: ACM Press, 2005. 218-227. [doi: 10.1145/1073970.1074008]
    [19] Wu CX, Burns R. Handling heterogeneity in shared-disk file systems. In: Proc. of the 2003 ACM/IEEE Conf. on Super Computing(SC 2003). Washington: IEEE Computer Society, 2003. [doi: 10.1145/1048935.1050158]
    Comments
    Comments
    分享到微博
    Submit
Get Citation

陈涛,肖侬,刘芳.对象存储系统中自适应的元数据负载均衡机制.软件学报,2013,24(2):331-342

Copy
Share
Article Metrics
  • Abstract:4527
  • PDF: 6134
  • HTML: 0
  • Cited by: 0
History
  • Received:September 20,2010
  • Revised:September 01,2011
  • Online: February 02,2013
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the first2033294Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063