FAQs – Staff
A TECHNICAL OPERATION AND FUNCTIONALITY
A1. How Does the AI Assistant Work?
The AI Assistant in Blackboard works differently from a public chatbot like ChatGPT because it is designed to focus only on the course materials and data that it has access to (which is referred to as a walled garden approach), better ensuring its answers are relevant and accurate. The answers always cite the source too, which means you can see exactly where the information came from to generate the answer.
When someone asks the AI Assistant a question, it first goes through a pre-search phase. This is like a planning step where the AI Assistant figures out:
What the question is (e.g., is it asking about course content, technical help, or something else?) The assistant also rephrases the question in a few different ways to ensure a fuller and broader search.
Where to look for the answer within the course materials
What to do if the question is out of scope
Instead of searching the whole internet, the chatbot only looks at the information it’s connected to, like lecture notes and other materials specific to the course. It will direct the student elsewhere if it can’t find the answer or the question isn’t about the course. This is provided through ‘hardcoded’ escalation paths and would often, for example, mean re-directing to the Student Centre
The AI Assistant avoids making things up (so-called “hallucinations”) by focusing only on trusted course materials. It’s more thoughtful about giving accurate answers because it works within clear limits, unlike ChatGPT, which can answer more broadly across any topic and provides no insight into where the information was sourced from. This also helps avoid students getting answers to nonsensical questions they may ask.
This system helps students get the right information quickly while keeping the learning experience reliable and focused on their needs.
A2. What Content does the AI Assistant ‘look at’ in a Blackboard site?
Please look at this document for further details.
A3. Can the AI Assistant handle sudden spikes in user queries during peak times?
The AI Assistant is designed to handle multiple queries simultaneously. However, during peak times, response times may vary. It’s important to manage user expectations and provide support through other channels when necessary.
A4. Is the AI Assistant multi-lingual?
The Module AI Assistant supports a number of languages to differing degrees. This document, “What Languages does LearnWise support” provides an outline of the languages and the level of support you can expect
A5. How often is the AI Assistant updated to maintain the accuracy of the information provided?
The AI Assistant undergoes regular weekly updates and continuous learning to remain accurate and relevant. Users’ feedback helps identify areas for improvement, which are then addressed in subsequent updates.
QUALITY AND RELIABILITY
B1. How accurate and reliable is the AI Assistant’s information?
All information used comes solely from the Blackboard site concerned. The assistant only answers information about the course content and course information using a combination of large language models to compile answers. So, if the course content is accurate, then the chatbot responses should be too. Hallucinations are minimised by forcing the AI Assistant to only generate answers from the course content. However, as with all Generatiove AI driven systems, it is possible for mistakes to be made. Therefore, it is key that any answer is cross-checked with the sources the AI Assistant references at the end of the answer.
B2. Can the AI Assistant understand and address complex or nuanced questions?
AI Assistants are good at handling straightforward queries based on the course content. However, they may struggle with more complex or nuanced questions that require deep human insight. It’s important that students use the AI Assistant as a supplementary tool and not a replacement for human educators. Having help desks embedded into the assistant so that students are able to escalate problems to a human if necessary is also an important feature.
B3. How does the AI Assistant handle biases in its responses?
All responses come from course data; hence, they are as biased as the course data. It is essential to balance their use with traditional learning methods to ensure students develop critical thinking and problem-solving skills. Educators should encourage students to use the AI Assistant as a resource rather than a crutch. It cannot, for example, write essays, or complete assignments on behalf of the student.
EDUCATIONAL IMPACT AND USAGE
C1. Will the AI Assistant reduce face-to-face interactions between students and educators?
No, whilst the tool can be used as a study buddy and generate quiz questions around the course content, it is not equipped to deliver courses and is no substitute for face-to-face teaching. The purpose of it is to support students in their studies, not to replace teachers.
AI Assistants are good at handling straightforward queries based on the course content. However, they may struggle with more complex or nuanced questions that require deep human insight. It’s important that students use the AI Assistant as a supplementary tool and not a replacement for human educators.
C2. Is there a risk of students and educators becoming overly dependent on the AI Assistant?
While AI Assistants can be incredibly helpful, it’s important to balance their use with traditional learning methods to ensure students develop critical thinking and problem-solving skills. Educators should encourage students to use the AI Assistant as a resource rather than a crutch. It is designed to help instructors and students, not act as a barrier to communication between them.
C3. Can the AI Assistant be used to help students engage with module content?
Yes, it can. Some example questions students could ask are:
Multiple choice quiz:
Please give me some multiple-choice questions for this week’s topic. Ask the questions one at a time.
Exam study:
Please help me prepare for the final exam. I want 10 flashcards based on the relevant course pages.
Practice exercises:
Generate 5 exercises based on this week’s main topic. Give me the exercises one at a time.
What should I study this week?
Create a study plan for this module please.
ADMINISTRATION AND ETHICS
D1. What measures are in place to ensure students use the AI Assistant ethically and do not plagiarise?
Clear guidelines on the ethical use of the chatbot should be provided to students. Administrators can monitor the use of the chatbot and take necessary actions if instances of plagiarism are detected.
D2. Can a module leader have access to the chat history for their module?
At the moment no, but there are plans on the development roadmap to introduce access to a specific module’s chat inbox for module leaders.
D3. Can a module leader find out how the AI Assistant is being used on their module?
Yes, this is possible. In the AI Assistant interface at the top right there are two options. The first, View History, is visible to any user and shows their personal history of ‘chats’ with the AI Assistant. The ‘View Dashboard’ link provides metrics that will indicate how the AI Assistant is being used. Under the ‘Insights’ tab on the dashboard you can see for example the number of conversations and the number of unique users.
The ‘Trends’ tab provides the option for to run reports. Note that there has to have been a minimum of 30 conversations before the report will run. The report will essentially summarise themes emerging from an analysis of the conversations that have happened between two selected dates.
Please email additional questions to g.saunders@westminster.ac.uk
AI Assistants can enhance learning by providing information and support, but their use should be balanced with traditional methods for a comprehensive educational experience.