Medical Blogs Text Mining: An Efficient Methodology for Knowledge Identification & Sharing

Ubiquitous and economical availability of data through Internet proffers an unprecedented opportunity. The quantity and quality of knowledge driven is directly proportional to the amount of data available. Data mining renders the set of tools and techniques required to analyze large collections of data and extract useful patterns. This can endow healthcare community with great assistance to make calculated decisions given a medical situation. As the available data on the Internet is growing exponentially, generated by different communities and in different formats including emails, blogs, forums, standard web pages, and so on, the task of the text mining community becomes more challenging. The availability of such humongous data, along with offering great prospects, also creates enormous amount of challenges for the data science community, including the data and text mining communities. These challenges include the analysis, interpretation, and exploration of a variety of data typesto work on and the development of innovative algorithms and software tools to efficiently handle large quantities of data. Electronic medical data are among the more rapidly growing data genres. They are available in the form of electronic patient records, blogs authored by patients and medical professionals, medical experiences shared over the internet forums, prescriptions and invoices and many other formats. Electronic patient records have long been used for analysis of healthcare services. Knowledge available in medical blogs data, however, has not been utilized as effectively and extensively by the healthcare community. In this thesis, we propose a methodology that makes use of the state-of-the-art data and text mining techniques to take advantage of the social medical data available on the Internet for finding associations among diseases, symptoms, laboratory test, and medications, etc. The proposed methodology identifies information from large collection of unstructured texts available in the form of blogs and forum posts. The method finds associations and dissociations among the diseases and symptoms by employing interesting feature selection and association rule mining techniques. The application areas include correlating the clinical information like symptoms, diseases, etc. for the development of medical expert systems. Finding negative associations among clinical features is more important as it can greatly reduce the diagnosis space. Also the spatial and temporal relations among the medical conditions are extracted. The resultant associations and dissociations have confidence level bracketed together to help a medical professional take efficient decision. The empirical evaluations, on a variety of data sets, demonstrate the pragmatic efficacy and performance efficiency of the techniques put forward in this research.