一种海量数据流应用并行优化模型
作者:
基金项目:

Supported by the National High-Tech Research and Development Plan of China under Grant No.2006AA01A102 (国家高技术研究发展计划(863))


A Parallel Optimization Model for Massive Data Stream Application
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [14]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    计算进入了多核时代,处理器的发展不再由更快的主频带动,而是依靠增加片上的多个核心.但是,对于高性能应用来说,多核平台的并行处理由于缺少适合的并行程序开发工具还处于初始阶段,对应用的优化需要对底层线程结构的深入了解和正确使用.从海量数据流应用的特点出发,提出了三级流水多线程模型,它的线程同步机制没有竞争,并且实现了不同特征数据流的差别服务.然后,在遥感图像处理和骨干网网络入侵检测系设计中,应用了海量数据流应用模型,并在多个多核平台下对骨干网网络入侵检测系统进行了性能评价.对SPARC T1平台上的线程映射方法进行研究,测试了不同映射方法的性能,并归纳出应用在体系结构方面的特征;采用Sun SPARC T1架构8核32线程服务器和曙光x86架构8处理器16核服务器对系统吞吐率进行了测试,实验结果都表现了良好的可扩展性;使用真实骨干网络流量记录文件回放产生的模拟流量,对比测试了模型应用前后数据流的服务时间,改进系统的响应时间获得了显著的提高;针对系统连接数大、负载重和处理多样性的特点,采用基于探针流的采样算法准确测试了在精确预测IP网段策略下系统的服务质量,同时也测试了增加服务质量优化后系统的延迟开销,实验结果表明,系统在引入较少延迟下提高了数据流的服务质量.

    Abstract:

    While computing is entering a new phase in which CPU improvements are driven by the addition of multiple cores on a single chip, rather than higher frequencies. Parallel processing on these systems is in a primitive stage, and requires the explicit use and knowledge of underlying thread architecture. Based on the features of massive data stream application, this paper proposes three-level pipelining programming model of multithreading system, which realizes the new synchronization mechanism with no contention of shared structures and is capable to provide differential service for data streams. Then the paper applies the new model to remote sensing information processing system and backbone network intrusion detection system, and evaluates the improved system on several multicore platforms. In performance analysis, the optimized effects of backbone network intrusion detection system are evaluated in several aspects of throughput scalability on both SPARC T1 and x86 platforms, the impacts of different multithreading mapping methods on throughput, and the comparison of response time and service quality before and after optimization. The experimental results show that the system throughput has good scalability on both platforms, the values of response time are greatly improved and the prioritized streams achieve better response time with the differential service mechanism.

    参考文献
    [1] Sun Microsystem, Inc. Sun Fire T1000 and T2000 Servers Benchmarks. http://www.sun.com/servers/coolthreads/t1000/ benchmarks.jsp
    [2] Sun XJ, Sun NH, Chen MY. Optimization of B-NIDS for multicore. Journal of Computer Research and Development, 2007,44(10):1733?1740 (in Chinese with English abstract).
    [3] Kongetira P, Aingaran K, Olukotun K. Niagara: A 32-way multithreaded SPARC processor. IEEE Micro, 2005,25(2):22?29.
    [4] Sun Microsystem, Inc. OpenSPARC T1 Microarchitecture Specification. 2006. http://opensparc-t1.sunsource.net/specs/UA2005- current-draft-P-EXT.pdf
    [5] Sun Microsystem, Inc. Sun Fire T2000 Server. 2007. http://www.sun.com/servers/coolthreads/t2000/ [6] Craig Z. Master/Slave speculative parallelization and approximate code [Ph.D. Thesis]. Madison: University of Wisconsin-Madison, 2002.
    [7] Alarm SR, Barrett RF, Kuehn JA, Roth PC, Vetter JS. Characterization of scientific workloads on systems with multi-core processors. In: Proc. of the IEEE Int’l Symp. Workload Characterization. San Jose, 2006. 225?236.
    [8] Hongzhang S, Erich S, Ji Q, David HB, Kathy Y. Performance modeling and optimization of a high energy colliding beam simulation code. In: Proc. of the ACM/IEEE SC 2006 Conf. Tampa, 2006.
    [9] Berekovic M, Stolberg HJ, Pirsch P. Multicore system-on-chip architecture for mpeg-4 streaming video. IEEE Trans. on CSVT, 2002,12(8):688?699.
    [10] David DC. The structuring of systems using upcalls. In: Proc. of the 10th ACM Symp. on Operating Systems Principles. Washington, 1985. 171?180.
    [11] Mats B., Per G. Locking effects in multiprocessor implementations of protocols. In: Proc. of the ACM SIGCOMM Symp. on Communications Architectures and Protocols. San Francisco, 1993. 74?83.
    [12] David JY, Erich MN, James FK, Don T. Networking support for large scale multiprocessor servers. In: Proc. of the SIGMETRICS. 1996. 116?125.
    [13] Jack LL, Susan JE, Joel SE, Henry ML, Rebecca LS, Dean MT. Converting thread-level parallelism into instruction-level parallelism via simultaneous multithreading. ACM Trans. on Computer Systems, 1997,15(2).
    [14] Susan JE, Joel SE, Henry ML, Jack LL, Rebecca LS, Dean MT. Simultaneous multithreading: A foundation for next generation processors. IEEE Micro, 1997,17(5).
    附中文参考文献: [2] 孙小涓,孙凝晖,陈明宇.多核平台上B-NIDS的优化.计算机研究与发展,2007,44(10):1733?1740.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

孙小涓,孙凝晖,雷 斌.一种海量数据流应用并行优化模型.软件学报,2009,20(zk):23-33

复制
分享
文章指标
  • 点击次数:4911
  • 下载次数: 6481
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2008-07-01
  • 最后修改日期:2009-04-02
文章二维码
您是第19792042位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号