Ayman Asad Khan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Ayman Asad Khan has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Kai-Kristian Kemell
Tampere University
4 shared publications - Mika Saari
Tampere University
4 shared publications - Pekka Abrahamsson
Tampere University
3 shared publications - Toufique Hasan
Tampere University
3 shared publications - Jussi Rasku
Tampere University
2 shared publications Alex Polyakov
Project Simple California USA
1 shared publication
Latest Publications
- Open Access
Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study
Large language models show promise for knowledge-intensive domains, yet their use in agriculture is constrained by weak grounding, English-centric training data, and limited real-world evaluation. These issues are amplified for low-resource languages, where high-quality domain documentation exists but remains difficult to access through general-purpose models. This paper presents AgriHubi, a domain-adapted retrieval-augmented generation (RAG) system for Finnish-language agricultural decision support. AgriHubi integrates Finnish agricultural documents with open PORO family models and combines explicit source grounding with user feedback to support iterative refinement. Developed over eight iterations and evaluated through two user studies, the system shows clear gains in answer completeness, linguistic accuracy, and perceived reliability. The results also reveal practical trade-offs between response quality and latency when deploying larger models. This study provides empirical guidance for designing and evaluating domain-specific RAG systems in low-resource language settings.
Open AccessTowards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study
Large language models show promise for knowledge-intensive domains, yet their use in agriculture is constrained by weak grounding, English-centric training data, and limited real-world evaluation. These issues are amplified for low-resource languages, where high-quality domain documentation exists but remains difficult to access through general-purpose models. This paper presents AgriHubi, a domain-adapted retrieval-augmented generation (RAG) system for Finnish-language agricultural decision support. AgriHubi integrates Finnish agricultural documents with open PORO family models and combines explicit source grounding with user feedback to support iterative refinement. Developed over eight iterations and evaluated through two user studies, the system shows clear gains in answer completeness, linguistic accuracy, and perceived reliability. The results also reveal practical trade-offs between response quality and latency when deploying larger models. This study provides empirical guidance for designing and evaluating domain-specific RAG systems in low-resource language settings.
- Open Access
- 30 October 2025
- Published by Springer Nature
- p. 22-31
Why Adapt RAG for Agile? Challenges, Frameworks, and the Role of Evaluator Agent
This paper proposes the adaptation of Retrieval Augmented Generation (RAG) based systems into Agile Software Development workflows, addressing the need for specialized evaluation methods that align with Agile’s iterative, feedback driven nature. Through a systematic mapping study of 20 papers, we identify existing evaluation frameworks for RAG systems and explore how they can be applied within Agile settings. The research highlights the role of Evaluator-Agents in aligning the responsiveness of RAG systems, with a focus on Multi-agent AI systems that are more efficient at handling complex, distributed tasks than Single-agent systems. The findings contribute to identify the potential of RAG and its evaluation in facilitating real time feedback and continuous improvement for Agile development.
- 30 October 2025
- Published by Springer Nature
- p. 22-31
Open AccessWhy Adapt RAG for Agile? Challenges, Frameworks, and the Role of Evaluator Agent
This paper proposes the adaptation of Retrieval Augmented Generation (RAG) based systems into Agile Software Development workflows, addressing the need for specialized evaluation methods that align with Agile’s iterative, feedback driven nature. Through a systematic mapping study of 20 papers, we identify existing evaluation frameworks for RAG systems and explore how they can be applied within Agile settings. The research highlights the role of Evaluator-Agents in aligning the responsiveness of RAG systems, with a focus on Multi-agent AI systems that are more efficient at handling complex, distributed tasks than Single-agent systems. The findings contribute to identify the potential of RAG and its evaluation in facilitating real time feedback and continuous improvement for Agile development.
- Open Access
- 9 September 2025
- Published by Springer Nature
- p. 143-158
Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
- 9 September 2025
- Published by Springer Nature
- p. 143-158
Open AccessEngineering RAG Systems for Real-World Applications: Design, Development, and Evaluation
- Open Access
AI and Agile Software Development: A Research Roadmap from the XP2025 Workshop
This paper synthesizes the key findings from a full-day XP2025 workshop on "AI and Agile: From Frustration to Success", held in Brugg-Windisch, Switzerland. The workshop brought together over 30 interdisciplinary academic researchers and industry practitioners to tackle the concrete challenges and emerging opportunities at the intersection of Generative Artificial Intelligence (GenAI) and agile software development. Through structured, interactive breakout sessions, participants identified shared pain points like tool fragmentation, governance, data quality, and critical skills gaps in AI literacy and prompt engineering. These issues were further analyzed, revealing underlying causes and cross-cutting concerns. The workshop concluded by collaboratively co-creating a multi-thematic research roadmap, articulating both short-term, implementable actions and visionary, long-term research directions. This cohesive agenda aims to guide future investigation and drive the responsible, human-centered integration of GenAI into agile practices.
Open AccessAI and Agile Software Development: A Research Roadmap from the XP2025 Workshop
This paper synthesizes the key findings from a full-day XP2025 workshop on "AI and Agile: From Frustration to Success", held in Brugg-Windisch, Switzerland. The workshop brought together over 30 interdisciplinary academic researchers and industry practitioners to tackle the concrete challenges and emerging opportunities at the intersection of Generative Artificial Intelligence (GenAI) and agile software development. Through structured, interactive breakout sessions, participants identified shared pain points like tool fragmentation, governance, data quality, and critical skills gaps in AI literacy and prompt engineering. These issues were further analyzed, revealing underlying causes and cross-cutting concerns. The workshop concluded by collaboratively co-creating a multi-thematic research roadmap, articulating both short-term, implementable actions and visionary, long-term research directions. This cohesive agenda aims to guide future investigation and drive the responsible, human-centered integration of GenAI into agile practices.
- Open Access
Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report
This paper presents an experience report on the development of Retrieval Augmented Generation (RAG) systems using PDF documents as the primary data source. The RAG architecture combines generative capabilities of Large Language Models (LLMs) with the precision of information retrieval. This approach has the potential to redefine how we interact with and augment both structured and unstructured knowledge in generative models to enhance transparency, accuracy, and contextuality of responses. The paper details the end-to-end pipeline, from data collection, preprocessing, to retrieval indexing and response generation, highlighting technical challenges and practical solutions. We aim to offer insights to researchers and practitioners developing similar systems using two distinct approaches: OpenAI's Assistant API with GPT Series and Llama's open-source models. The practical implications of this research lie in enhancing the reliability of generative AI systems in various sectors where domain-specific knowledge and real-time information retrieval is important. The Python code used in this work is also available at: https://github.com/GPT-Laboratory/RAG-LLM-Development-Guidebook-from-PDFs.
Open AccessDeveloping Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report
This paper presents an experience report on the development of Retrieval Augmented Generation (RAG) systems using PDF documents as the primary data source. The RAG architecture combines generative capabilities of Large Language Models (LLMs) with the precision of information retrieval. This approach has the potential to redefine how we interact with and augment both structured and unstructured knowledge in generative models to enhance transparency, accuracy, and contextuality of responses. The paper details the end-to-end pipeline, from data collection, preprocessing, to retrieval indexing and response generation, highlighting technical challenges and practical solutions. We aim to offer insights to researchers and practitioners developing similar systems using two distinct approaches: OpenAI's Assistant API with GPT Series and Llama's open-source models. The practical implications of this research lie in enhancing the reliability of generative AI systems in various sectors where domain-specific knowledge and real-time information retrieval is important. The Python code used in this work is also available at: https://github.com/GPT-Laboratory/RAG-LLM-Development-Guidebook-from-PDFs.