Sherif E. Hussein

Retrieval-augmented generation (RAG) systems show promise in specialized knowledge domains, but the tobacco research field lacks standardized assessment frameworks for comparing different large language models (LLMs). This gap impacts public health decisions that require accurate, domain-specific information retrieval from complex tobacco industry documentation. To develop and validate a tobacco domain-specific evaluation framework for assessing various LLMs in RAG systems that combines automated metrics with expert validation. Using a Goal-Question-Metric paradigm, we evaluated two distinct LLM architectures in RAG configurations: Mixtral 8 × 7B and Llama 3.1 70B. The framework incorporated automated assessments via GPT-4o alongside validation by three tobacco research specialists. A domain-specific dataset of 20 curated queries assessed model performance across nine metrics including accuracy, domain specificity, completeness, and clarity. Our framework successfully differentiated performance between models, with Mixtral 8 × 7B significantly outperformed Llama 3.1 70B in accuracy (8.8/10 vs. 7.55/10, p < 0.05) and domain specificity (8.65/10 vs. 7.6/10, p < 0.05). Case analysis revealed Mixtral’s superior handling of industry-specific terminology and contextual relationships. Hyperparameter optimization further improved Mixtral’s completeness from 7.1/10 to 7.9/10, demonstrating the framework’s utility for model refinement. This study establishes a robust framework specifically for evaluating LLMs in tobacco research RAG systems, with demonstrated potential for extension to other specialized domains. The significant performance differences between models highlight the importance of domain-specific evaluation for public health applications. Future research should extend this framework to broader document corpora and additional LLMs, including commercial models.

Background The tobacco industry targets future generations to maintain its profits. One of its tactics is to maintain its presence noticed at the level of retail environment. Measures to address this high presence are identified in the literature. Our study examines the effects of six of these identified tobacco retail reduction measures in reducing the total number of tobacco retailers and the number of retailers within 500, 1000 and 2000 m from youth-oriented facilities. Methods Data were collected by scraping Google Places on examination points including locations, opening hours, type and subtype of tobacco retailers in addition to educational, youth, health and religious facilities. The six studied measures were enacted using Python codes to assess the reduction percentage. The measures included restricting tobacco retail outlets per density of population, requiring a minimum distance between tobacco retailers, banning tobacco retail outlets within a minimum distance from specific facilities, banning tobacco sale in specific retail outlets, restricting tobacco retail outlets per geographic area and limiting the number of hours in which tobacco can be sold. Results The data collected showed a high presence of tobacco retailers around vital facilities, particularly youth-oriented ones. The six scenarios implemented showcased a positive reduction in the number of tobacco retailers in total and around youth-oriented facilities. The total reduction of retailers varied from 4% up to total elimination of availability. Discussion and conclusions Our study presents an example of measurable reduction impact of six tobacco retail reduction measures. The high percentage in reduction achieved, especially around youth-oriented facilities, is worth the attention of policy-makers to be considered as countermeasures for the high tobacco industry presence in retail.

Background: Accurate sentiment analysis and intent categorization of tobacco and e-cigarette-related social media content are critical for public health research, yet they necessitate specialized natural language processing approaches.Objective: To compare pre-trained and fine-tuned Flan-T5 models for intent classification and sentiment analysis of tobacco and e-cigarette tweets, demonstrating the effectiveness of pre-training a lightweight large language model for domain specific tasks.Methods: Three Flan-T5 classification models were developed: (1) tobacco intent, (2) e-cigarette intent, and (3) sentiment analysis. Domain-specific datasets with tobacco and e-cigarette tweets were created using GPT-4 and validated by tobacco control specialists using a rigorous evaluation process. A standardized rubric and consensus mechanism involving domain specialists ensured high-quality datasets. The Flan-T5 Large Language Models were fine-tuned using Low-Rank Adaptation and evaluated against pre-trained baselines on the datasets using accuracy performance metrics. To further assess model generalizability and robustness, the fine-tuned models were evaluated on real-world tweets collected around the COP9 event.Results: In every task, fine-tuned models performed much better than pre-trained models. Compared to the pre-trained model's accuracy of 0.33, the fine-tuned model achieved an overall accuracy of 0.91 for tobacco intent classification. The fine-tuned model achieved an accuracy of 0.93 for e-cigarette intent, which is higher than the accuracy of 0.36 for the pre-trained model. The fine-tuned model significantly outperformed the pre-trained model's accuracy of 0.65 in sentiment analysis, achieving an accuracy of 0.94 for sentiments.Conclusion: The effectiveness of lightweight Flan-T5 models in analyzing tweets associated with tobacco and e-cigarette is significantly improved by domain-specific fine-tuning, providing highly accurate instruments for tracking public conversation on tobacco and e-cigarette. The involvement of domain specialists in dataset validation ensured that the generated content accurately represented real-world discussions, thereby enhancing the quality and reliability of the results. Research on tobacco control and the formulation of public policy could be informed by these findings.

Introduction: Sentiment analysis has become a crucial area of research in natural language processing in recent years. The study aims to compare the performance of various sentiment analysis techniques, including lexicon-based, machine learning, Bi-LSTM, BERT, and GPT-3 approaches, using two commonly used datasets, IMDB reviews and Sentiment140. The objective is to identify the best-performing technique for an exemplar dataset, tweets associated with the WHO Framework Convention on Tobacco Control Ninth Conference of the Parties in 2021 (COP9).Methods: A two-stage evaluation was conducted. In the first stage, various techniques were compared on standard sentiment analysis datasets using standard evaluation metrics such as accuracy, F1-score, and precision. In the second stage, the best-performing techniques from the first stage were applied to partially annotated COP9 conference-related tweets.Results: In the first stage, BERT achieved the highest F1-scores (0.9380 for IMDB and 0.8114 for Sentiment 140), followed by GPT-3 (0.9119 and 0.7913) and Bi-LSTM (0.8971 and 0.7778). In the second stage, GPT-3 performed the best for sentiment analysis on partially annotated COP9 conference-related tweets, with an F1-score of 0.8812.Discussion: The study demonstrates the effectiveness of pre-trained models like BERT and GPT-3 for sentiment analysis tasks, outperforming traditional techniques on standard datasets. Moreover, the better performance of GPT-3 on the partially annotated COP9 tweets highlights its ability to generalize well to domain-specific data with limited annotations. This provides researchers and practitioners with a viable option of using pre-trained models for sentiment analysis in scenarios with limited or no annotated data across different domains.

The prediction of tweets associated with specific topics offers the potential to automatically focus on and understand online discussions surrounding these issues. This paper introduces a comprehensive approach that centers on the topic of "harm reduction" within the broader context of tobacco control. The study leveraged tweets from the period surrounding the ninth Conference of the Parties to review the Framework Convention on Tobacco Control (COP9) as a case study to pilot this approach. By using Latent Dirichlet Allocation (LDA)-based topic modeling, the study successfully categorized tweets related to harm reduction. Subsequently, various machine learning techniques were employed to predict these topics, achieving a prediction accuracy of 91.87% using the Random Forest algorithm. Additionally, the study explored correlations between retweets and sentiment scores. It also conducted a toxicity analysis to understand the extent to which online conversations lacked neutrality. Understanding the topics, sentiment, and toxicity of Twitter data is crucial for identifying public opinion and its formation. By specifically focusing on the topic of “harm reduction” in tweets related to COP9, the findings offer valuable insights into online discussions surrounding tobacco control. This understanding can aid policymakers in effectively informing the public and garnering public support, ultimately contributing to the successful implementation of tobacco control policies.

Recently, the process of fish species classification has become one of the most challenging problems addressed by researchers. In this work, a robust scheme to classify fish images based on robust feature extraction from shape signatures is proposed. First, the image contour is fitted using one of the common approaches named radial basis function neural network (RBFNN) fitting to obtain image centroid. Afterward, prominent features from the shape signature are extracted. These features are representative of fish shapes because they can distinguish the characteristics of each class as well as being relatively robust to scale and rotation changes. Finally, for the classification process purpose, RBFNN is used again for image classification against one of the most commonly used classification techniques called support vector machine (SVM). The proposed paradigm has been applied to a standard fish dataset acquired from a live video dataset grouped into twenty-three clusters representing specific fish species. The resulting accuracy based on SVM and RBFNN was 90.41% and 98.04%, respectively.

Agent-based modeling and simulation is a promising technique utilized to study and simulate interdependencies of complex infrastructures. This technique has two relevant properties which could be exploited in the field of healthcare, namely, the re-use of simulation models and the sharing of expertise. This research proposes a methodology to exploit the benefits of multi-agent systems with respect to the study of interdependencies between rehabilitation center personnel. The design procedure began by proposing an architectural layout which can best facilitate overall patient movement and improve healthcare services provided. Then, a simulation of the various personnel, along with the patients, was analyzed and implemented, using NetLogo, an agent-based simulation engine and environment. The simulation results showed how multi-agent systems could shed light on interdependent system performance and assist in their quantification. These results could then be used to make recommendations concerning the ideal number of personnel to be allocated to the rehabilitation center, thus ensuring its maximum performance.

Diagnosis is a critical preventive step in Coronavirus research which has similar manifestations with other types of pneumonia. CT scans and X-rays play an important role in that direction. However, processing chest CT images and using them to accurately diagnose COVID-19 is a computationally expensive task. Machine Learning techniques have the potential to overcome this challenge. This paper proposes two optimization algorithms for feature selection and classification of COVID-19. The proposed framework has three cascaded phases. Firstly, the features are extracted from the CT scans using a Convolutional Neural Network (CNN) named AlexNet. Secondly, a proposed features selection algorithm, Guided Whale Optimization Algorithm (Guided WOA) based on Stochastic Fractal Search (SFS), is then applied followed by balancing the selected features. Finally, a proposed voting classifier, Guided WOA based on Particle Swarm Optimization (PSO), aggregates different classifiers’ predictions to choose the most voted class. This increases the chance that individual classifiers, e.g. Support Vector Machine (SVM), Neural Networks (NN), k-Nearest Neighbor (KNN), and Decision Trees (DT), to show significant discrepancies. Two datasets are used to test the proposed model: CT images containing clinical findings of positive COVID-19 and CT images negative COVID-19. The proposed feature selection algorithm (SFS-Guided WOA) is compared with other optimization algorithms widely used in recent literature to validate its efficiency. The proposed voting classifier (PSO-Guided-WOA) achieved AUC (area under the curve) of 0.995 that is superior to other voting classifiers in terms of performance metrics. Wilcoxon rank-sum, ANOVA, and T-test statistical tests are applied to statistically assess the quality of the proposed algorithms as well.

In this research, an efficient scheme to identify leaf types is proposed. In that scheme, the leaf boundary points are fitted in a continuous contour using Radial Basis Function Neural Networks (RBFNN) to calculate the centroid of the leaf shape. Afterwards, the distances between predetermined points and the centroid were computed and normalized. In addition, the time complexity of the features’ extraction algorithm was calculated. The merit of this scheme is objects’ independence to translation, rotation and scaling. Moreover, different classification techniques were evaluated against the leaf shape features. Those techniques included two of the most commonly used classification methods; RBFNN and SVM that were evaluated and compared with other researches that used complex features extraction algorithms with much higher dimensionality. Furthermore, a third classification method with an optimization technique for the SVM using Salp Swarm Algorithm (SSA) was utilized showing a significant improvement over RBFNN and SVM.