| Summary: | The establishment of an AI coworker dedicated to advancing corporate learning and promoting inclusivity represents a significant advancement in the adoption of artificial intelligence (AI) within corporate environments. As organizations increasingly integrate AI technologies, it is crucial to understand how these tools impact workplace dynamics and whether they inadvertently exclude individuals based on gender identity, ethnicity, or other demographic characteristics. Maintaining employee well-being and fostering an inclusive environment are essential in this evolving landscape. This research is coordinated with Arctic Group AB as the stakeholder. It focuses on building an AI coworker designed to process educational video materials and interact with users in a way that supports inclusivity. The AI coworker utilizes Retrieval-Augmented Generation (RAG) for video transcription to deliver relevant responses and facilitate access to learning resources. The built system includes a text input for user queries, a visual avatar for interactive visuals, and a panel for selecting relevant educational materials for use in chat contexts. The performance of Retrieval-Augmented Generation was assessed using a dataset of 37 question-and-answer pairs focused on agile learning topics from Arctic Group video materials. Various language models, including llama3-8b-8192, llama-3.1-8b-instant, mixtral-8x7b-32768, gemma2-9b-it, and gpt-4o-mini, were evaluated on four key metrics:(1) Response vs. Reference Answer, assessing how closely model responses aligned with predetermined reference answers;(2) Response vs. Input, measuring the relevance and practical usefulness of each response to the user’s original question;(3) Response vs. Retrieved Documents, evaluating the factual consistency of responses with content from documents retrieved by the RAG system; and(4) Retrieved Documents vs. Input, determining how well the retrieved documents matched the intent of the user’s initial query. Considering the subjective nature of bias ...
|
|---|