This is The Final Part of Uses of Text Mining in Web Content Mining. In the forth part of this series, we discussed up to clustering.
Concept linkage tools are able to parse and connect similar concepts used in documents and thus help the user to find information he might not have found through normal search methods. It is almost possible to select information, rather than cumbersome search.
For example, a standard text mining program would be able to easily connect between themes A and B and between B and C. But the program would also be able to create a potential link between themes A and C with concept linkage, something that a human explorer would not have figured out so quickly, because he has to sort out the vast amount of information before he can find connections.
---
So much research has been done in the field of biomedicine that it is impossible for researchers to read everything and at the same time create links between these documents. That’s why concept linkage is a valuable concept in the field of text mining, as the software can identify links between diseases and treatments that a human being would be unable to.
The work, which was done by hand at that time, can now be automated by programs using concept linkage technology and used in all areas of research. Similar experiments as those of Swanson were already replicated with automated tools.
Concept linking is a way to find and display the terms that are highly associated with the term selected that correlate the strongest with it The Concept Linking window shows a hyperbolic tree graph with fever in the center of the tree structure.
Information visualization
Information visualization, or visual text mining, merges large text sources into a graphical hierarchy or map, and provides navigation capabilities to quickly search the large amount.
It should also be noted that such forms of representation and all other forms of visualization quickly convince without ensuring that the data on which they are based are correct. The persuasiveness of images, which often suggest scientific correctness only through the nature of the visualization, can thus become a deception.
Information visualization is also used by governments to visualize terrorist networks, or to link crimes with one another. It can provide them with a map of possible connections between suspicious activities, and it can also examine the links they did not find before.
A question and answer system may consist of a combination of different text mining technologies whose purpose is to provide inexperienced users with flexible access to information. By writing a request in natural language and not getting a series of documents containing the answer, but directly the actual answer. The current trend is very much in the direction of answering “short word questions”, such as in a definition.
For example, in a question and answer system, text mining technology information extraction can be used to analyze entities such as people, places or events. It is also possible to categorize questions so that they can be categorized into who, where, when, how, etc. Within a company intranet, this technique can be used so that as many answers to standard questions as possible are independently found by the employees.
Conclusion on Uses of Text Mining in Web Content Mining
The elaborations of the methodology and the various tasks of text mining provided a good overview of the many fields of application in web content mining.
Initially known as Arpanet, the Internet was initially used for networking between universities and research institutions, but over the last 45 years it has become the main medium for global communication, using billions of people globalization. It is due to this fact that the majority of the data in the network today consists of weak or unstructured data. Text mining is by definition predestined for the analysis of such data.
The information retrieval, as the basis of any mining process, makes it possible to filter out of the vast amounts of data on the Internet a rough preselection of documents in which the desired information occurs. This amount of documents can then be further narrowed down by means of information extraction, either by producing a more precise selection of data in which the desired data exists or by excluding the documents that are not relevant.
The two mentioned methods form the basis for today’s search engines, which, like Google, can also carry out the examination of the text content and thus form the most (especially in the private sector) most used application of text mining in web content mining.
As the amount of data on the World Wide Web continues to increase dramatically as our world becomes more digital, it’s becoming increasingly difficult to efficiently and effectively find the information you need using search engines. To make this work easier, automatic text summary is a good way to quickly and accurately summarize the content of thousands of search results.
Particularly suitable for the scientific work are the methods of categorization, clustering and concept linking, which can summarize a large number of documents in a first preselection or, in the case of the latter, even explicitly make explicit implicit relationships between the documents. While the first two methods classify the documents into specific groups so that they have a first limitation, the latter method examines the amount of documents on relationships that may exist between the content concepts.
Since speed is important in today’s business world, text mining is becoming more and more important in this area as well. Due to the increasing digitization of the data in today’s business enterprises, more and more mining methods can be used to investigate them. In the context of the rapid information gain, the methods of information visualization, “Topic Detection and Tracking” (TDT) and question-and-answer systems are particularly suitable here.
Today’s decision makers usually do not want to conduct studies over several weeks in order to recognize certain relationships in the business world, but have presented them quickly and clearly. Information visualization like no other method of text mining is suitable because it can display complicated data clearly in graphical form. Also question-answer systems are suitable in this context, since the user does not have to first familiarize himself with the syntax of search engines, but can ask his question in normal prose and get an easy-to-understand answer. The TDT is of particular importance as it can dramatically reduce the response time to new messages. So a company can respond to new developments much faster.
In conclusion, it can be stated that text mining (especially in the area of web content mining) is a relatively young field. In recent years, however, more and more companies have recognized the enormous potential of this technology and are now increasingly investing in further research. It therefore remains to be seen which further innovations we can expect for science, the economy and the private sector in the future. However, it is already out of the question that due to the progressive digitization of our world, the processing methods must be further optimized in order to be able to use the potential of the flood of data effectively.
