ChatGPT and the Importance of AI Governance

ChatGPT: The next big disrupter?

ChatGPT, the artificial intelligence chatbot developed by OpenAI, has become the fastest growing consumer product in history, reaching 100 million monthly active users within a mere 2 months of its launch. It has caused shockwaves across the education, media and marketing industries and has stoked fears of broader job losses amongst white collar workers. Microsoft has already incorporated ChatGPT’s underlying technology into its Bing chatbot and has declared its intention to make Google, the ‘800-pound gorilla’ of internet search, ‘dance’. As ChatGPT begins to be integrated into more products, some AI experts have even predicted that it could become the next big platform, reshaping the web.

For businesses, ChatGPT offers an opportunity: potentially boosting productivity, reducing costs and enhancing customer experience. However, ChatGPT has a number of limitations, and its use introduces risks that will need to be carefully managed.

In this article, we offer guidance on how organisations can harness the benefits of ChatGPT while effectively addressing the risks associated with its use.

TL;DR

ChatGPT is a chatbot that uses artificial intelligence to perform a variety of tasks, including engaging in conversation, generating human-like text and even writing code.
ChatGPT, along with other large language models (LLMs), represents a significant advancement in natural language processing, exhibiting greater proficiency than previous AI tools.
Despite its potential usefulness, ChatGPT has certain limitations that may lead to undesirable outcomes if users rely on it inappropriately.
To minimize the risk of misuse, it is crucial for organizations to implement a written ChatGPT usage policy and to train employees on both the policy and ChatGPT's limitations.
ChatGPT is just the tip of the artificial intelligence iceberg. Organisations incorporating artificial intelligence into their operations should establish appropriate AI risk management frameworks.

What is ChatGPT?

ChatGPT is a chatbot that can perform a variety of tasks, including engaging in conversation, generating human-like text and even writing code. Initially launched as a free public release by OpenAI in November 2022, OpenAI also offers a premium version of the product through a paid subscription model and pay-as-you-go API access to some of its underlying technology.

ChatGPT's core technology lies within the field of artificial intelligence. To contextualise ChatGPT within the broader landscape of AI products, ChatGPT can be described as a generative AI product built on top of a large language model that consists of an artificial neural network trained using various deep learning techniques. There is obviously a lot of jargon in that description. So to assist, here is a list of some of the key artificial intelligence terms commonly used in relation to ChatGPT and how they relate to each other:

Artificial intelligence refers to technologies that enable machines to perform tasks traditionally associated with human intelligence, including making predictions, recommendations, or decisions.
Artificial neural networks are computing systems that are loosely inspired by the human brain and which consist of interconnected units (or artificial neurons) that are often organised into layers.
Machine learning refers to techniques used in the field of artificial intelligence to enable computer programs to learn from training data rather than relying on predefined rules.
Deep learning is a subset of machine learning that involves training artificial neural networks to model and solve complex problems. The word ‘deep’ signifies the fact that these artificial neural networks typically have many layers of neurons.
Large language models (LLMs) are artificial neural networks trained on vast quantities of text data, often sourced from the internet.
Generative AI is a recently popularised term that is being used to refer to new artificial intelligence products like ChatGPT and image generation tools (like Stable Diffusion and Dall-E 2) that can create new, and often creative, content (as opposed to AI tools that focus on other tasks like analysis or classification).

Is ChatGPT the first?

While similar large language models existed before ChatGPT's release (such as Google's LaMDA, Meta's OPT, and Nvidia's Megatron-Turing NLG), ChatGPT was the product that sparked the public’s imagination as it was the first product that made a powerful large language model widely available to the general public via an easily accessible chatbot interface. It has since been joined by a series of similar chatbots (like Microsoft's Bing, Google's Bard and Anthropic's Claude), but with recent upgrades ChatGPT continues to push the boundaries and is regarded by some commentators to be the ‘most verbally dextrous’ of the publicly available chatbots.

What can ChatGPT do?

ChatGPT and other LLMs are powerful tools and perform natural language processing tasks with much greater proficiency than previous AI tools. The figure below demonstrates some of the key tasks for which ChatGPT seems specifically well adapted:

Can I delegate all my work to ChatGPT?

Not yet. While it may prove to be an extremely useful tool, ChatGPT has several limitations that may result in undesirable results if inappropriate reliance is placed on it. These limitations include:

Hallucinations	LLMs are programs that generate text based on statistical models of language. They are not grounded in, or constrained by, any concept of truthfulness, common sense or rules of logic. As a result, LLMs are notorious for generating what some have labelled confidently worded ‘bullshit’ (which is sometimes also referred to, with some poetic licence, as a ‘hallucination’). As this output is generated with confidence and without acknowledging any sources, it can be challenging for users to identify the errors.
Unreliable	LLMs are unreliable. Not only do they rely on statistical models of language, but there is also an element of randomness incorporated into every output, which means that identical inputs can result in different outputs. This means that ChatGPT can perform a task correctly on the first nine attempts and then fail catastrophically on the tenth attempt.
Fails at certain tasks	ChatGPT’s fluency and confident tone can generate the illusion that it possesses some underlying general intelligence. However, there are some tasks in which ChatGPT is incredibly flawed. For example, even the latest version of ChatGPT’s large language model (GPT-4) consistently fails in basic tasks like counting and arithmetic. This is because the underlying language model that underpins it is based on probability – it is not a calculator or some kind of inference engine that applies logical rules.
Outdated content	The majority of ChatGPT’s training occurred on a dataset current only to September 2021 and, as a result, it struggles with facts that occurred after that date. OpenAI has recently announced a web browsing plugin that will allow ChatGPT to search the web; however, as this does not involve updating the underlying LLM, it remains unclear to what extent the web browser plugin will resolve issues caused by the limited currency of the dataset.
Limited access to specialist information	ChatGPT’s underlying model was largely trained on publicly available internet resources. Organisations that wish to adapt the tool to use their internal knowledge bases (eg, to create customer service chatbots) will need to integrate this knowledge via the APIs - a task which might be easier said than done.
Toxicity and bias	The outputs of LLMs reflect the biases in their training materials. ChatGPT’s initial training materials included large portions of the internet and, as such, it sometimes produces extremely toxic output. Although OpenAI has put in place some measures to address this issue (including through content filtering and by specifically training ChatGPT to avoid toxic output), it will be difficult to fully resolve these issues given the statistical nature of LLMs. In fact, attempts to train ChatGPT to avoid toxic and biased outputs give rise to their own issues, as this involves training by humans, each of whom will necessarily have their own biases.
Confidentiality and Privacy	Conversations with ChatGPT using the consumer-facing product may be used by OpenAI to improve, and train future versions of, the product. A recent bug in the application allowed some users to view the titles of other users’ conversation histories. So, it would be a mistake to input any confidential, sensitive or personal information into the publicly accessible version of ChatGPT right now. OpenAI has recently announced that it will no longer use data submitted through the APIs for training purposes. However, organisations will still need to consider data and privacy issues (including data security and cross-border data transfers).

Do you have a ChatGPT use policy?

Despite the potential risks associated with using ChatGPT and other LLMs, it's important not to overlook their potential benefits. LLMs are rapidly improving, and will likely provide at least some productivity savings. Of course, there will be situations where the benefits of using these tools outweigh the risks, while in other cases, the risks will not justify the benefits. As a result, organisations cannot generally afford to impose a blanket ban on their employees' use of this technology, nor should they remain silent or allow unrestricted use.

Therefore, it is crucial for organisations to establish a written policy that governs how employees are permitted to use ChatGPT in the workplace. This may include:

Prohibiting or discouraging certain use cases (eg, relying on ChatGPT for factual information)
Recommending approved use cases (eg, as a writing aid for specific content)
Specifying the types of data that should not be input into ChatGPT (customer data, personal information, trade secrets, other confidential or sensitive information etc)
Requiring employees to verify the accuracy of ChatGPT's output
Mandating consideration of potential biases in the generated content
Assessing other risks associated with using ChatGPT, such as confidentiality, legal professional privilege, copyright, privacy, and competition law

Organisations should also consider providing training and demonstrations on ChatGPT and similar tools to employees, especially as many employees may already be using these tools. This will help educate employees about the risks involved and ensure compliance with the company's ChatGPT use policy.

In light of the rapidly evolving nature of this field, it is also essential to conduct regular risk assessments to identify and mitigate potential harm, as well as for internal legal departments and the broader business to maintain ongoing communication to discuss use cases, opportunities, and risks associated with ChatGPT.

Do you have an AI risk management framework?

ChatGPT is just the tip of the artificial intelligence iceberg – a surprise success that has managed to enter into the public consciousness. However, organisations are increasingly adopting (and in some cases already have adopted) a diverse range of AI products for various purposes, including streamlining operations, optimising decision-making, and enabling personalised experiences for their customers.

While many organisations have risk management frameworks in place for data and traditional software, the use of artificial intelligence introduces unique risks, especially when it is relied upon to make important decisions. These risks warrant a broader assessment, including by taking into account considerations such as accuracy, reliability, safety, fairness, bias, discrimination, and transparency, and legal considerations such as use of intellectual property and data privacy.

These risks are not merely theoretical: we have moved far beyond the days of researchers and ethicists pontificating about how artificial intelligence should respond to ever-increasingly contrived ‘trolley problems’. In recent months, we have seen a major tech company lose US$100 billion in market value due to errors caused by a chatbot, a news organisation discovering errors in over 50% of articles generated by its AI tool, and even one company’s ‘AI companion’ chatbot having to be neutered following reports by users of unwelcome sexual advances.

So if your organisation does not yet have an AI risk management framework in place, now might be the right time to establish one. A best practice AI risk management framework will involve the following:

AI Principles	AI Principles define the overall approach for how AI can be responsibly used within your organisation
AI Governance Framework	An AI Governance Framework sets out the policies and procedures that will put your AI Principles into practice. This includes: introducing clearly defined roles and responsibilities for decision-making and oversight, both during implementation and throughout the AI lifecycle; and ensuring that policies applicable to the AI lifecycle are introduced and disseminated throughout your organisation (for example policies relating to data use, fairness, transparency and risk management).
AI Impact Assessments	AI Impact Assessments operationalise your AI Principles and AI Governance Framework by prompting your teams to think critically about AI and how to manage the risks its poses when either introducing a new AI project or making significant changes to an existing AI project.

A step into the unknown

ChatGPT has immense potential. Nevertheless, it is essential to recognise its limitations and proactively manage potential risks. By implementing a ChatGPT use policy and a robust risk management framework, organisations can successfully jump on the AI bandwagon while safeguarding their customers and businesses.

Over the coming months we will explore some of the legal issues associated with ChatGPT and generative AI. Subscribe by selecting ‘Tech & Data’ as your area of interest here.

Note: ChatGPT was used as a tool to assist in writing this article (including by translating some clunky legalese into plain English). However, all research was undertaken by human lawyers, and the majority of the final text was produced by their independent intellectual effort. Any hallucinations in the text remain those of the lawyers and any errors are solely the result of deficiencies in human intelligence.