News
Towards a standardized methodology and dataset for evaluating LLM-based digital forensic timeline analysis
Congratulations to Hudan Studiawan and co-authors Frank Breitinger, and Mark Scanlon on the publication of Towards a standardized methodology and dataset for evaluating LLM-based digital forensic timeline analysis in Forensic Science International: Digital Investigation.
Co-authors: Frank Breitinger, and Mark Scanlon.
AI-generated summary of the contribution: This paper addresses the need for a standardized approach to evaluate the performance of Large Language Models (LLMs) in digital forensic timeline analysis tasks. The proposed methodology includes a dataset, timeline generation, and ground truth development, and recommends the use of BLEU and ROUGE metrics for quantitative evaluation. The paper presents a case study using ChatGPT and demonstrates the effectiveness of the proposed methodology in evaluating LLM-based forensic timeline analysis. The study also discusses the limitations of applying LLMs to forensic timeline analysis and highlights the importance of maintaining the ‘AI-assisted investigation’ and ‘human-in-the-loop’ mantras when using LLMs in digital forensics.