This guide is designed to help U of T students, faculty and researchers navigate the use of GenAI tools such as ChatGPT, MS CoPilot, and others in applied research contexts. Jump to a section or for more information, please refer to the recommended resources below.

Jump to:

GenAI as a research tool
Best practices
U of T MS CoPilot enterprise edition
Prompt engineering basics
Evaluating AI tools

Recommended resources

Large Language Models (LLMs) such as ChatGPT are machine learning models designed for natural language processing. They anticipate and mimic human language.

They are not appropriate tools for all steps of research activity.

Concerns around hallucinations, quality of outputs, academic integrity (citations, plagiarism), copyright infractions, and lack of nuance in scholarly conversations may override the convenience of using such tools in your research.

Useful research applications can include brainstorming, keyword generation, simple data analysis (with prompting), backwards searching, language tasks (summarizing, editing, translation).  

Outsourcing your market analysis and critical thought to a machine requires understanding of:  

• The purpose/context it was created for
• The sources that it was trained on (titles, authors, scope, dates of publication, access to updates, internet access)
• The biases of the trainers (we cannot eliminate bias by using an AI)
• The permissions and restrictions of e.g. sharing copyrighted material with the AI
• Privacy laws and regulations of e.g. uploading research data for analysis

1. Think critically about the tools you are using

• Is it the best tool for your specific purpose?
• Are there potential risks or impacts to consider?
• Evaluate AI tools using the ROBOT framework

2. Refer to use and allowance guidelines

• Check with your instructor, faculty or department, journal editor, etc before using a GenAI tool in an assignment or article.
• Adhere to copyright and academic integrity policies

3. Choose wisely

• Opt for U of T's enterprise edition of Microsoft CoPilot, which adheres to institutional privacy and security requirements
• Use prompt engineering to improve outputs

4. Check results carefully

• Are the sources real and if so, are they credible?
• Has the model followed your instructions?

5. Be transparent and ethical in your use of GenAI tools

• Use to supplement, not replace, your original creative thought and output
• Be honest and upfront about your use of AI tools
• Prioritize issues of data and personal privacy, data security, and algorithmic bias issues

If you opt to use an LLM for your research or assignment, University of Toronto recommends the U of T Microsoft CoPilot protected enterprise edition.

• Built on ChatGPT4 
•  Adheres to U of T privacy and security requirements
• Considered safe for use with up to Level 3 data 
• Chat data is not saved, used to train models, or made available to Microsoft  

Prompt engineering: Designing and refining questions or instructions to provide context and elicit a specific response from an AI tool   

Prompts include: 

• Role - specify an expertise, persona or character to assume
• Task - the specific instructions you want the model to perform
• Goal or purpose - the intended use of the output (e.g. audience)
• Process - order or steps to be taken to achieve the task
• Iteration - Building on the conversation to correct or hone in on the desired result
• Format - the style, structure or constraints of the output you are seeking 

There are many different types of prompts. 

To learn more: 

https://www.promptingguide.ai/

https://platform.openai.com/docs/guides/prompt-engineering

https://element451.com/blog/chatgpt-cheatsheet

The ROBOT test developed by the LibrAIry, can be used to critically evaluate an AI application before or while using it. The following is reproduced with permission under Creative Commons Licence 4.0.

Being AI Literate does not mean you need to understand the advanced mechanics of AI. It means that you are actively learning about the technologies involved and that you critically approach any texts you read that concern AI, especially news articles. 

We have created a tool you can use when reading about AI applications to help consider the legitimacy of the technology.

Reliability

Objective

Bias

Ownership

Type

Reliability

• How reliable is the information available about the AI technology?
• If it’s not produced by the party responsible for the AI, what are the author’s credentials? Bias?
• If it is produced by the party responsible for the AI, how much information are they making available? 
  • Is information only partially available due to trade secrets?
  • How biased is they information that they produce?

Objective

• What is the goal or objective of the use of AI?
• What is the goal of sharing information about it?
  • To inform?
  • To convince?
  • To find financial support?

Bias

• What could create bias in the AI technology?
• Are there ethical issues associated with this?
• Are bias or ethical issues acknowledged?
  • By the source of information?
  • By the party responsible for the AI?
  • By its users?

Owner

• Who is the owner or developer of the AI technology?
• Who is responsible for it?
  • Is it a private company?
  • The government?
  • A think tank or research group?
• Who has access to it?
• Who can use it?

Type

• Which subtype of AI is it?
• Is the technology theoretical or applied?
• What kind of information system does it rely on?
• Does it rely on human intervention? 

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

To cite in APA: Hervieux, S. & Wheatley, A. (2020). The ROBOT test [Evaluation tool]. The LibrAIry. https://thelibrairy.wordpress.com/2020/03/11/the-robot-test