Image generated using Adobe Firefly

Generative AI can be an invaluable tool for enhancing education, offering numerous benefits. However, it's important to be aware of the limitations and considerations when utilizing generative AI in educational settings.

Note that these limitations are likely to differ between different generative AI tools, and change as generative AI tools advance over time.

Some considerations around current limitations

Table
Limitations	Description	Example
Lack of context understanding leads to assumptions	If users aren’t specific enough, AI models will make assumptions. Consequently, the output might be off-topic or lack depth. It is advised to ask the models’ assumptions.	See an example of how different ways of prompting generative AI may lead to the AI making different assumptions.
Bias	Generative AI tools are trained on vast and diverse datasets which may lean towards certain viewpoints. Be cautious of potential biases, including possible alignment with commercial objectives or political prejudices. Apply your apply critical thinking skills to analyze and contextualize the outputs you receive from generative AI. This process should include cross-verifying the information received from outputs and forming your own informed perspective.	For example, political prejudice is a significant issue brought up by ChatGPT and other chatbots built using large language models (LLMs). Click on the dropdown arrow to see an example of this. When asked about political/social issues, ChatGPT was often left-leaning. This can be seen below when ChatGPT 3.5 was prompted with the following questions. You can view this conversation as a shared ChatGPT link. However, when prompted to provide justification, ChatGPT assures users that it does not form its own opinion and there is likely more evidence in its database supporting legalization of cannabis. When asked for the sources behind the responses, ChatGPT furthers its stance that there are more reliable sources and further research that needs to be done by the user. When asked the same thing, Google Bard generated a more neutral stance in some instances and often far more explanation. Consider using multiple sources when researching.
Hallucination	Generative AI is prone to hallucination. This is where the models produces responses that are not always factual. Remember that just because it sounds authoritative, it doesn't mean it is correct. Consider the impact if the generated text contains factual inaccuracies.	If the AI's ideas are inaccurate in brainstorming, there may be less impact than where it is being used to produce a user manual for a piece of medical technology. The incident involves a lawyer who encountered challenges when relying on an AI language model, ChatGPT, for legal research (https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt-for-tricking-them-into-citing-fake-cases/102462028).
Limited judgement	While AI can mimic creative processes, it generally lacks true autonomy of judgement of the creative thought process, as it relies on patterns from its training data. This explains for the same prompt, different generative AI tools outputs similar response.	Chat GPT and Gemini generated similar list using the same prompt