Visible to the public Text Mining in Unclean, Noisy or Scrambled Datasets for Digital Forensics Analytics

TitleText Mining in Unclean, Noisy or Scrambled Datasets for Digital Forensics Analytics
Publication TypeConference Paper
Year of Publication2017
AuthorsXylogiannopoulos, K., Karampelas, P., Alhajj, R.
Conference Name2017 European Intelligence and Security Informatics Conference (EISIC)
KeywordsARPaD, Arrays, composability, digital forensics, Electronic mail, email analysis, Human Behavior, human factors, LERP-RSA, Metrics, Noise measurement, pattern detection, pubcrawl, Scalability, Social network services, text analytics, text mining

In our era, most of the communication between people is realized in the form of electronic messages and especially through smart mobile devices. As such, the written text exchanged suffers from bad use of punctuation, misspelling words, continuous chunk of several words without spaces, tables, internet addresses etc. which make traditional text analytics methods difficult or impossible to be applied without serious effort to clean the dataset. Our proposed method in this paper can work in massive noisy and scrambled texts with minimal preprocessing by removing special characters and spaces in order to create a continuous string and detect all the repeated patterns very efficiently using the Longest Expected Repeated Pattern Reduced Suffix Array (LERP-RSA) data structure and a variant of All Repeated Patterns Detection (ARPaD) algorithm. Meta-analyses of the results can further assist a digital forensics investigator to detect important information to the chunk of text analyzed.

Citation Keyxylogiannopoulos_text_2017