An Efficient Valid Page Crawling Approach for Websites with Dynamic Scripts

微信服务号

微信订阅号

Home > Archive>Volume 20, Issue zk, 2009 >176-183

An Efficient Valid Page Crawling Approach for Websites with Dynamic Scripts
DOI:
                        
Author:
                        
Affiliation:
Clc Number:
Fund Project:

Article

Figures

Metrics

Reference

Cited by

Materials

Comments

Abstract:

In times of Web 2.0, more and more websites adopt dynamic scripts for user interaction, and the switches between pages are no longer all based on the “” tags and the URL is no longer the unique identification of a Web page. Traditional Web crawlers can’t deal with Web pages containing dynamic scripts, as a result, search engines, such as Google, give up these Web pages. The research on crawling website with dynamic scripts is still in the early stage. This paper proposes an efficient valid page crawling approach for websites with dynamic scripts. Firstly, by training the paper can get the events and the Web elements that triggered the events, which would lead the people to desired Web pages. Then, the paper generates the XPath patterns of these elements and record the events the people need to trigger. During crawling, the paper only considers these event and element combinations for accelerating the crawling. Additionally, the paper demonstrates the efficiency and the effectiveness of the approach by extensive experimental evaluation.

Reference

Cited by

Get Citation

夏冰,高军,王腾蛟,杨冬青.一种高效的动态脚本网站有效页面获取方法.软件学报,2009,20(zk):176-183

Copy

Article Metrics

Abstract:
PDF:
HTML:
Cited by:

History

Received:May 01,2009
Revised:July 20,2009
Adopted:
Online:
Published:

You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063

微信服务号

微信订阅号

Get Citation

Share

Article Metrics

History