AKAyman Asad Khan

Ayman Asad Khan

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ayman Asad Khan has a total of 28 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study

    by Toufique Hasan, Ayman Asad Khan, Mika Saari, Vaishnavi Bankhele, Pekka Abrahamsson

    Large language models show promise for knowledge-intensive domains, yet their use in agriculture is constrained by weak grounding, English-centric training data, and limited real-world evaluation. These issues are amplified for low-resource languages, where high-quality domain documentation exists but remains difficult to access through general-purpose models. This paper presents AgriHubi, a domain-adapted retrieval-augmented generation (RAG) system for Finnish-language agricultural decision support. AgriHubi integrates Finnish agricultural documents with open PORO family models and combines explicit source grounding with user feedback to support iterative refinement. Developed over eight iterations and evaluated through two user studies, the system shows clear gains in answer completeness, linguistic accuracy, and perceived reliability. The results also reveal practical trade-offs between response quality and latency when deploying larger models. This study provides empirical guidance for designing and evaluating domain-specific RAG systems in low-resource language settings.

  • Open Access

    Why Adapt RAG for Agile? Challenges, Frameworks, and the Role of Evaluator Agent

    by Ayman Asad Khan, Md Toufique Hasan, Mika Saari, Kai-Kristian Kemell, Jussi Rasku

    This paper proposes the adaptation of Retrieval Augmented Generation (RAG) based systems into Agile Software Development workflows, addressing the need for specialized evaluation methods that align with Agile’s iterative, feedback driven nature. Through a systematic mapping study of 20 papers, we identify existing evaluation frameworks for RAG systems and explore how they can be applied within Agile settings. The research highlights the role of Evaluator-Agents in aligning the responsiveness of RAG systems, with a focus on Multi-agent AI systems that are more efficient at handling complex, distributed tasks than Single-agent systems. The findings contribute to identify the potential of RAG and its evaluation in facilitating real time feedback and continuous improvement for Agile development.

  • Open Access

    Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation

    by Toufique Hasan, Muhammad Waseem, Kai-Kristian Kemell, Ayman Asad Khan, Mika Saari, Pekka Abrahamsson


  • Open Access

    AI and Agile Software Development: A Research Roadmap from the XP2025 Workshop

    by Zheying Zhang, Tomas Herda, Victoria Pichler, Pekka Abrahamsson, Geir K. Hanssen, Joshua Kerievsky, Alex Polyakov, Mohit Chandna, Marius Irgens, Kai-Kristian Kemell, et al.

    This paper synthesizes the key findings from a full-day XP2025 workshop on "AI and Agile: From Frustration to Success", held in Brugg-Windisch, Switzerland. The workshop brought together over 30 interdisciplinary academic researchers and industry practitioners to tackle the concrete challenges and emerging opportunities at the intersection of Generative Artificial Intelligence (GenAI) and agile software development. Through structured, interactive breakout sessions, participants identified shared pain points like tool fragmentation, governance, data quality, and critical skills gaps in AI literacy and prompt engineering. These issues were further analyzed, revealing underlying causes and cross-cutting concerns. The workshop concluded by collaboratively co-creating a multi-thematic research roadmap, articulating both short-term, implementable actions and visionary, long-term research directions. This cohesive agenda aims to guide future investigation and drive the responsible, human-centered integration of GenAI into agile practices.


  • Open Access

    Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report

    by Ayman Asad Khan, Toufique Hasan, Kai Kristian Kemell, Jussi Rasku, Pekka Abrahamsson

    This paper presents an experience report on the development of Retrieval Augmented Generation (RAG) systems using PDF documents as the primary data source. The RAG architecture combines generative capabilities of Large Language Models (LLMs) with the precision of information retrieval. This approach has the potential to redefine how we interact with and augment both structured and unstructured knowledge in generative models to enhance transparency, accuracy, and contextuality of responses. The paper details the end-to-end pipeline, from data collection, preprocessing, to retrieval indexing and response generation, highlighting technical challenges and practical solutions. We aim to offer insights to researchers and practitioners developing similar systems using two distinct approaches: OpenAI's Assistant API with GPT Series and Llama's open-source models. The practical implications of this research lie in enhancing the reliability of generative AI systems in various sectors where domain-specific knowledge and real-time information retrieval is important. The Python code used in this work is also available at: https://github.com/GPT-Laboratory/RAG-LLM-Development-Guidebook-from-PDFs.