Understanding the risks and limitations

Before you begin using generative AI (genAI), you need to be aware of some of the risks and limitations of using these platforms.

Remember: always read assessment instructions carefully and check with your unit chair if it is acceptable to use genAI in your assessment.

Is it current?

Generated content may not be up to date but it may give the impression that it is.

• Cross-check information with recent, relevant and credible sources.

Is it accurate?

Generated content may be inaccurate or entirely false, while appearing plausible and authoritative.

• Fact-check all generated claims by consulting the original source and other credible sources.
• Quality-assure any generated outputs by evaluating their credibility.

Can the source of information be traced?

An essential aspect of academic writing is providing references to demonstrate that you have built your understanding on credible sources of information. GenAI may provide limited, inadequate or misattributed sources.

• Locate the original source to ensure that ideas have been correctly attributed.
• Locate and use credible sources to ensure that ideas and claims have been reported correctly. Begin with recommended unit readings and Library databases.

Is it biased?

Generated content can only reflect the data on which it has been trained. If training data lacks diversity, due to historical inequalities, failure to represent marginalised voices, or skewed sampling, the AI model will perpetuate these biases. Be aware too that genAI models are commercial tools and the algorithms they run on may be influenced by commercial objectives.

• Be sceptical and critically examine any potential biases in output.
• Cross-check information with other credible sources.

How will I assess the quality of the output?

In addition to sometimes being out-of-date and prone to inaccuracies, genAI output can be formulaic and generic, especially in its first responses. The quality of the output can be poor and the reasoning shallow.

• Spend some time breaking down the questions you want to ask before you begin.
• Refine your responses through the effective use of iterative prompts.
• Return to recommended course materials to check any claims.
• To learn more about writing in your discipline, compare the style and quality of generated output to your recommended unit readings.

Do I need to consider breaches of copyright?

While generated content may be 'unique', it is also based on the data used to train the AI. Training data may have been used without the permission of the copyright owner.

In addition, input submitted to a genAI platform via your prompts may grant the AI service the right to re-use and distribute this content, which may result in a breach of copyright.

• Be aware that generated output may include material that breaches copyright.
• Avoid submitting text, PDFs, images, sound or video that you do not own the copyright to. This includes course materials.

Are there any privacy risks?

All input and output may be stored by the owner of the AI platform. Once again, input submitted to a genAI platform via your prompts may grant the AI service the right to re-use and distribute this content. What will happen with this information in the future? What happens if the inputs and outputs are connected to you and become publicly available?

• Never enter any personal details.
• Never enter any confidential data related to your research (or anyone else’s).