An enhancement of depth first-search algorithm applied in full-text website search An enhancement of depth first-search algorithm applied in full-text website search

ABSTRACT: This search, An Enhancement of Depth-First Search Algorithm applied in Full-Text Website Search, aimed to study and improve the process of Depth-First Search Algorithm and be used for the future of Full-Text website search. According to Mabayoje and Adebayo in their study, A Full-text Website Search Engine Powered by Lucene and The Depth First Search Algorithm, published in March 2013; The Web is seen as a large graph with page at its nodes and hyperlinks as its edges and with the amount of available text data on the web growing rapidly, the need for users to search such information is dramatically increasing. Full text systems are better for quickly searching high volumes of unstructured text for the presence of any word or combining of words. This research also addressed the following problems that can be found in Depth-First Search Algorithm: (1) The Algorithm goes through unnecessary depths; (2) The algorithm does not perform a pruning technique to cut down unwanted nodes; (3) There is no guarantee in finding minimal solution exists. Prior to these problems, the proponents have conducted thorough simulations in the Depth-First Search Algorithm in order to test the validity of the stated problems. Moreover, the proponents also gathered and reviewed different researches, in form of graph traversal algorithm to support the present problems in this research. This research also contains the following objectives to be done in order to formulate solutions for the said problems. (1) To prevent the web crawling algorithm from digging unto unnecessary depths. (2) To enhance the algorithm by applying the pruning technique that will reduce the crawling time as well as the memory consumption. (3) To enhance the algorithm so that it produces less but more relevant search results. Once these following objectives were accomplished, the Depth-First Search Algorithm was enhanced. The researchers through and brainstorming solved the problems sated above accordingly. The first problem indicating that the algorithm goes through unnecessary depths was solved by checking if the URL is already visited, if it is already a visited URL it is no longer revisited and the crawler fetches a new URL from the list of pages to be visited, but if it is a URL that is yet to be visited the crawler proceeds to visiting it. The second problem that states that the algorithm does not perform a pruning technique to cut down unwanted was solved by taking into consideration the webpages metadata (a set of data that describes and gives information about the other data), it served as a criteria that allowed the cutting down of unwanted nodes and retaining the wanted ones. And lastly the third problem that states that there is no guarantee in finding minimal solution, if more than one solution exists; it was solved through the use of Fuzzy Query processing (it returns a list of results based on likely relevance and highly relevant matches usually appear near / at the top of the list), this process enabled the system to produce less but more relevant search results. In applying those changes to the existing system, the researchers were able to achieve the objectives of this study and enhance the system. The enhanced system no longer goes through unnecessary depth, it is able cut down unwanted nodes which results to lower crawling time and memory consumption, and lastly it is able to produce less search result but more relevant ones. The researchers also saw that there is still room for improving the Depth-First Search Algorithm applied in Full-Text Website Search by implementing the system in a different programing language which will produce a standalone User Interface, formulate a more efficient pruning technique to incorporate in the algorithm to improve the use of resources available and enhance the system in a manner that allows it to be a full blown search engine.

ABSTRACT: This search, An Enhancement of Depth-First Search Algorithm applied in Full-Text Website Search, aimed to study and improve the process of Depth-First Search Algorithm and be used for the future of Full-Text website search. According to Mabayoje and Adebayo in their study, A Full-text Website Search Engine Powered by Lucene and The Depth First Search Algorithm, published in March 2013; The Web is seen as a large graph with page at its nodes and hyperlinks as its edges and with the amount of available text data on the web growing rapidly, the need for users to search such information is dramatically increasing. Full text systems are better for quickly searching high volumes of unstructured text for the presence of any word or combining of words. This research also addressed the following problems that can be found in Depth-First Search Algorithm: (1) The Algorithm goes through unnecessary depths; (2) The algorithm does not perform a pruning technique to cut down unwanted nodes; (3) There is no guarantee in finding minimal solution exists. Prior to these problems, the proponents have conducted thorough simulations in the Depth-First Search Algorithm in order to test the validity of the stated problems. Moreover, the proponents also gathered and reviewed different researches, in form of graph traversal algorithm to support the present problems in this research. This research also contains the following objectives to be done in order to formulate solutions for the said problems. (1) To prevent the web crawling algorithm from digging unto unnecessary depths. (2) To enhance the algorithm by applying the pruning technique that will reduce the crawling time as well as the memory consumption. (3) To enhance the algorithm so that it produces less but more relevant search results. Once these following objectives were accomplished, the Depth-First Search Algorithm was enhanced. The researchers through and brainstorming solved the problems sated above accordingly. The first problem indicating that the algorithm goes through unnecessary depths was solved by checking if the URL is already visited, if it is already a visited URL it is no longer revisited and the crawler fetches a new URL from the list of pages to be visited, but if it is a URL that is yet to be visited the crawler proceeds to visiting it. The second problem that states that the algorithm does not perform a pruning technique to cut down unwanted was solved by taking into consideration the webpages metadata (a set of data that describes and gives information about the other data), it served as a criteria that allowed the cutting down of unwanted nodes and retaining the wanted ones. And lastly the third problem that states that there is no guarantee in finding minimal solution, if more than one solution exists; it was solved through the use of Fuzzy Query processing (it returns a list of results based on likely relevance and highly relevant matches usually appear near / at the top of the list), this process enabled the system to produce less but more relevant search results. In applying those changes to the existing system, the researchers were able to achieve the objectives of this study and enhance the system. The enhanced system no longer goes through unnecessary depth, it is able cut down unwanted nodes which results to lower crawling time and memory consumption, and lastly it is able to produce less search result but more relevant ones. The researchers also saw that there is still room for improving the Depth-First Search Algorithm applied in Full-Text Website Search by implementing the system in a different programing language which will produce a standalone User Interface, formulate a more efficient pruning technique to incorporate in the algorithm to improve the use of resources available and enhance the system in a manner that allows it to be a full blown search engine.

Filipiniana

5