Scientific Paper Recommendation System: Application of Sentence Transformers and Cosine Similarity Using arXiv Data
Open Access
- 3 August 2025
- journal article
- Published by Politeknik Negeri Batam in JOURNAL OF APPLIED INFORMATICS AND COMPUTING
- Vol. 9 (4) , 1374-1382
Abstract
Searching for relevant scientific literature faces complex challenges due to the proliferation of academic publications. This research develops a semantic similarity-based scientific paper recommendation system by utilizing Sentence Transformer (all-MiniLM-L6-v2 model) and cosine similarity algorithm on arXiv dataset (15,504 papers in Computer Science). The system is implemented as a Streamlit-based interactive web application that accepts user queries and recommends related papers based on semantic similarity. Performance evaluation using Precision, Mean Average Precision (MAP), Mean Reciprocal Rank (MRR), and Normalized Discounted Cumulative Gain (NDCG) metrics showed that embedding text from the Introduction section without pre-processing yielded the best performance (NDCG: 0.7590; MAP: 0.6960; MRR: 0.7254), outperforming Abstract-based or text combination approaches. A user test of 45 respondents confirmed the effectiveness of the system: 95.5% expressed satisfaction with the relevance of the recommendations, and 93.3% confirmed a significant reduction in manual search time. The findings prove that retaining the raw text structure in the Introduction is optimal for semantic representation. Development suggestions include multidomain dataset expansion and transformer model optimization for accuracy improvement.Keywords
This publication has 17 references indexed in Scilit:
- On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top- n Recommendation2024
- K-Means and Alternative Clustering Methods in Modern Power SystemsIEEE Access, 2023
- Scientific paper recommendation systems: a literature review of recent publicationsInternational Journal on Digital Libraries, 2022
- Design and Implementation of an Improved K-Means Clustering AlgorithmMobile Information Systems, 2022
- Paper recommendation based on heterogeneous network embeddingKnowledge-Based Systems, 2020
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- VOPRec: Vector Representation Learning of Papers with Text Information and Structural Identity for RecommendationIEEE Transactions on Emerging Topics in Computing, 2018
- Research-paper recommender systems: a literature surveyInternational Journal on Digital Libraries, 2015
- K‐means clustering: A half‐century synthesisBritish Journal of Mathematical and Statistical Psychology, 2006
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951