Dilani Widanapathiranage
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Dilani Widanapathiranage has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Scott Barnett
Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia
1 shared publication - Stefanus Kurniawan
Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia
1 shared publication - Wannita Takerngsaksiri
Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia
1 shared publication
Latest Publications
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 159-161
Towards Task-Agnostic Evaluation Framework for LLM-Embedded Applications
Evaluating Large Language Model (LLM) embedded applications presents fundamental challenges distinct from traditional software testing due to non-deterministic behaviour across diverse contexts. Existing benchmarks fail to capture domain-specific requirements, while practitioners resort to ad-hoc strategies that are time-consuming and inconsistent. This research proposes a task-agnostic evaluation framework that automatically synthesises evaluations from reusable principles rather than requiring custom configurations. The framework employs a meta-model capturing essential task properties to enable automated evaluation generation across objective and subjective tasks. Through systematic literature review, grey literature analysis, and practitioner validation, we establish guidelines for evaluation selection across diverse LLM task types. Industry case studies validate the framework’s practical effectiveness, measuring generalizability, performance, time efficiency, and workflow alignment. This research establishes methodological foundations that shift LLM evaluation from task-specific design to systematic synthesis from reusable principles.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 159-161
Open AccessTowards Task-Agnostic Evaluation Framework for LLM-Embedded Applications
Evaluating Large Language Model (LLM) embedded applications presents fundamental challenges distinct from traditional software testing due to non-deterministic behaviour across diverse contexts. Existing benchmarks fail to capture domain-specific requirements, while practitioners resort to ad-hoc strategies that are time-consuming and inconsistent. This research proposes a task-agnostic evaluation framework that automatically synthesises evaluations from reusable principles rather than requiring custom configurations. The framework employs a meta-model capturing essential task properties to enable automated evaluation generation across objective and subjective tasks. Through systematic literature review, grey literature analysis, and practitioner validation, we establish guidelines for evaluation selection across diverse LLM task types. Industry case studies validate the framework’s practical effectiveness, measuring generalizability, performance, time efficiency, and workflow alignment. This research establishes methodological foundations that shift LLM evaluation from task-specific design to systematic synthesis from reusable principles.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 116-120
TaskEval: Synthesised Evaluation for Foundation-Model Tasks
Hallucinations are a key concern when creating applications that rely on Foundation models (FMs). Understanding where and how these subtle failures occur in an application relies on evaluation methods known as evals. Prior work focuses on defining new eval methods or benchmark datasets for specific tasks. However, neither helps a software team with a task-specific FM application when there is no metric or dataset. The demand for both automated approaches and deep integration of human insight makes this a challenging problem. We address this gap by proposing an approach to synthesise a FM task-specific evaluator program that provides automation and a custom UI for capturing feedback. The core novelty of our approach lies in: (1) a task-agnostic meta-model that captures properties of any FM task, (2) an interaction protocol for efficient use of human feedback, and (3) an eval synthesiser that selects or generates an appropriate set of evals. We implement our approach in TaskEval and demonstrate the concept on two diverse FM tasks: chart data extraction and document question answering. A preliminary evaluation on the quality of our selected evals shows 93% and 90% accuracy respectively. Our research tackles a growing problem facing engineering teams, how to evaluate and review outputs from FM tasks.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 116-120
Open AccessTaskEval: Synthesised Evaluation for Foundation-Model Tasks
Hallucinations are a key concern when creating applications that rely on Foundation models (FMs). Understanding where and how these subtle failures occur in an application relies on evaluation methods known as evals. Prior work focuses on defining new eval methods or benchmark datasets for specific tasks. However, neither helps a software team with a task-specific FM application when there is no metric or dataset. The demand for both automated approaches and deep integration of human insight makes this a challenging problem. We address this gap by proposing an approach to synthesise a FM task-specific evaluator program that provides automation and a custom UI for capturing feedback. The core novelty of our approach lies in: (1) a task-agnostic meta-model that captures properties of any FM task, (2) an interaction protocol for efficient use of human feedback, and (3) an eval synthesiser that selects or generates an appropriate set of evals. We implement our approach in TaskEval and demonstrate the concept on two diverse FM tasks: chart data extraction and document question answering. A preliminary evaluation on the quality of our selected evals shows 93% and 90% accuracy respectively. Our research tackles a growing problem facing engineering teams, how to evaluate and review outputs from FM tasks.