Jurnal Testing dan Implementasi Sistem Informasi

Improving FAQ Retrieval for Academic Regulations Using Semantic Embeddings and LLM Question Augmentation

2026-05-17T08:32:31+07:00

Academic regulations in higher education are often documented in lengthy and formal handbooks, making it difficult for students to find relevant information using everyday language. This study developed a semantic FAQ retrieval system for academic regulations using IndoSBERT and question augmentation. The FAQ corpus was constructed from official academic and internship documents, resulting in 92 FAQ entries across 33 topical categories. Seed questions were generated from category–keyword pairs and expanded using simple rule-based augmentation and FLAN-T5-based paraphrasing. The dataset was evaluated using an 80:10:10 train–validation–test split. IndoSBERT was fine-tuned with Multiple Negatives Ranking Loss under three configurations: baseline, baseline with simple augmentation, and baseline with simple plus LLM-based augmentation. Retrieval performance was measured using Recall@1, Recall@3, Recall@5, and Mean Reciprocal Rank. The best result was achieved by the simple plus LLM augmentation configuration, with Recall@1 of 0.7848, Recall@5 of 0.8987, and MRR of 0.8396. These findings show that LLM-based question augmentation improves semantic retrieval robustness while keeping answers grounded in curated academic regulations.

An Expert System for Early Detection of Mental Health Conditions Using Certainty Factor and DASS-42

2026-05-17T08:42:29+07:00

Mental health problems such as depression, anxiety, and stress continue to increase in many countries, while access to professional services is still limited. Many digital screening systems use fixed scoring methods and do not consider uncertainty in user responses. This study developed a web-based expert system by combining the Depression Anxiety Stress Scales (DASS-42) and the Certainty Factor (CF) method to represent uncertainty in overlapping emotional symptoms and provide more flexible screening results. The knowledge base was prepared through consultation with a licensed clinical psychologist and converted into 42 production rules based on the DASS-42 items. Each rule was assigned a confidence value according to expert judgment. The system uses forward chaining to combine active rules and calculate confidence scores for depression, anxiety, and stress at the same time. System evaluation was conducted using 50 community cases aged 18–35 years and compared with independent expert assessment. The overall accuracy reached 86% (43 of 50 cases). The accuracy for each category was 88.2% for depression, 82.3% for anxiety, and 87.5% for stress. Most classification errors occurred between anxiety and stress, which may be related to overlapping symptoms in the DASS-42 instrument. The findings indicate that the proposed system can support early mental health screening through interpretable confidence-based results. However, this study used a limited dataset and only one expert in knowledge development. The system is intended as a screening support tool and not as a replacement for clinical diagnosis.

Analysis of User Reviews for The Mytelkomsel App Using Naïve Bayes and Random Forest Methods

2026-05-17T08:45:16+07:00

While sentiment analysis of local application reviews predominantly utilizes native Indonesian data, these datasets frequently suffer from colloquial ambiguities and informal structures that degrade classifier performance. This study addresses this gap by implementing a language-filtering mechanism to separate and analyze English and Indonesian user opinions from the MyTelkomsel application, specifically justifying the inclusion of English reviews due to their superior grammatical structure and syntactic consistency, which inherently enhances feature extraction. A systematic methodology was employed, encompassing data collection from the Google Play Store, comprehensive pre-processing (case folding, tokenization, stopword removal, and stemming), and Term Frequency-Inverse Document Frequency (TF-IDF) vectorization. Evaluated using Naïve Bayes and Random Forest algorithms on 25,000 customer feedbacks, the models were compared across accuracy, precision, recall, and F1-score. The empirical results demonstrated that Random Forest outperformed Naïve Bayes, achieving a higher accuracy of 86.85% compared to 86.36%. This superiority stems from Random Forest’s robust capability to mitigate class imbalance and minimize error distribution across sentiment categories. Ultimately, this approach provides precise, actionable insights into service quality, enabling Telkomsel to effectively distinguish user satisfaction, target operational improvements, and mitigate customer churn.

Design and Evaluation of a Decision Support System for Classifying Tourism Site Crowding and Recommending Governance Responses in Bunaken National Park

2026-05-17T08:48:57+07:00

Effective governance of marine protected areas (MPAs) requires reliable mechanisms to translate multidimensional ecological and social data into coordinated institutional action. Despite widespread adoption of carrying capacity frameworks, a significant "implementation gap" persists between theoretical conservation thresholds and operational decision-making at the site level. This study addresses that gap by designing, implementing, and evaluating a Decision Support System (DSS) artifact tailored for Bunaken National Park (BNP), Indonesia. Grounded in Design Science Research (DSR) principles, the artifact employs a deterministic, rule-based classification engine that processes four normalized input dimensions visitor density, social carrying capacity, infrastructure load, and governance readiness to compute a Composite Crowding Index (CCI). The CCI is mapped through an explicit IF-THEN rule engine to four crowding categories (Low, Moderate, High, Extreme), each linked to a validated governance action package. A deterministic rule-based approach was chosen over probabilistic or machine-learning alternatives to ensure full decision traceability, which is a non-negotiable requirement for public-sector governance. System robustness was evaluated through structured scenario testing across 140 logic-coverage cases, assessed against four criteria: output consistency (100%), expert rule alignment (97.8%), decision traceability (100%), and processing efficiency (<1.15 seconds per scenario). The artifact successfully automates the mapping of site-level crowding status to discrete, auditable governance actions. The theoretical contribution lies in formalizing subjective management reasoning into a transparent, reproducible DSS that bridges sustainability science and institutional practice in high-pressure marine tourism environments.

Design and Evaluation of an AI-Assisted Digital KPI Information System for Employee Performance Monitoring and Recommendation

2026-05-17T08:50:22+07:00

Manual and fragmented KPI-based evaluation often weakens employee performance monitoring by causing delays, input errors, and inconsistent interpretation. This study addresses that problem by designing and evaluating an AI-assisted digital KPI information system that integrates competency assessment, digital KPI records, performance scoring, dashboard-based monitoring, and recommendation support in a single environment. The study contributes by transforming the conventional competency–KPI–performance framework into an operational decision-support artifact with embedded AI classification. Using a prototype-based approach, the system was evaluated with a structured dataset of 340 employee-year records from 2021–2025. Three machine learning models were compared for classifying employee performance into High, Moderate, and Low categories. The results show that the system is functionally feasible and usable for KPI-based performance monitoring, while Random Forest achieved the best classification performance with 0.9853 accuracy and 0.9852 F1-score. The findings indicate that the proposed system can improve the structure of digital KPI monitoring and provide AI-assisted support for managerial review and follow-up actions. The study contributes theoretically by extending KPI-based performance management into an intelligent information system context and practically by offering a feasible model for organizations operating under limited implementation conditions.