Is AI Biased?

Peter Gravell
2 Feb 2023 • 6 min read

TLDR; Yes - most AIs are biased, so care is needed when using them.

Real Fast Reports uses AI similar to ChatGPT to help teachers write school reports. These AIs take written text as their input and generate written text as their output. We can call them "Generative Natural Language AIs" to differentiate them from other types of AI.

For example, if you input

"Tell me three hot things:"

the AI may generate an output like:

"hot sauce, hot yoga, hot tubs"

How on Earth can an AI do this? How can it respond to a written command? And how does it know about hot yoga?

How they work is complicated, but the important thing for us is that these AIs are "trained" by being fed lots of written language, e.g. from books or newspapers or the internet. The training texts are processed to create a language model which allows the AI to respond to written input and generate original written output (that may or may not make sense).

Generative Natural Language AI examples

I asked four different AIs to "Tell me three hot things." These are the (totally real and unedited) outputs I got:

AI Number One:

1. Hot sauce
2. Hot tubs
3. Hot yoga

AI Number Two:

1. A man with a great smile
2. A passionate kiss
3. A spontaneous adventure

AI Number Three:

1. A hot, sweaty body pressed against yours
2. The sound of your skin against his
3. The way you feel when you're in his arms

AI Number Four:

1. A good cold drink of water
2. A good, hot bath
3. A hot, steamy shower

Disclaimer: asking about "hot things" was the first random example that came into my head (I had a hot cup of tea in my hand at the time). I was expecting literal responses like "fire" and "the sun", so I was taken by surprise and could not stop laughing when I read these outputs!

As you can see, the different AIs responded very differently - both in their interpretation of the word "hot", and in their ability to make any sense (how is "A good cold drink of water" hot?). But they all demonstrate what these AIs can do - take written input and create written output.

Note that the AIs are not just searching through a database of "facts" to produce these responses. The AIs are much more sophisticated than that, and are capable of writing original text that never appeared in their training set. You can test this with super weird inputs like "Write a short guide about how to tile a floor with freshly caught mackerel". I'll put the response at the end.

Read the example responses again

Not only are they hilarious, but you may also notice something. Three of the items are "gendered":

• A man with a great smile
• The sound of your skin against his
• The way you feel when you're in his arms

Notice, that of the three gendered items, 100% of them refer to males. 0% refer to females.

Is this just chance?

In this case it's hard to say. However, trying some different inputs reveals many examples of gender bias in AI outputs. And they are most definitely not hilarious - in fact they are very concerning.

Why are AIs biased?

As I mentioned, these AIs are trained on written text. This text may be from books, newspapers, or "the internet". Therefore if the training material is biased in any way, it should not be surprising if the AI output is also biased. The AI has no independent "conscience" to tell it what is good and bad.

Unfortunately, there are many biases in our culture which are reflected in the books, news articles and Twitter posts we write. Gender and racial biases spring immediately to mind. If you train an AI with sexist texts, it may write sexist output. If you train an AI with racist texts, it may write racist output.

Are all AIs biased?

Focussing specifically on large Generative Natural Language AIs at the start of 2023, the answer is probably "Yes". There are various ways that AIs could be made less biased, and this is something that AI vendors are working on, but at present all written AI output should be read and considered carefully for bias before use.

I am unaware of any AIs that have been trained solely on writing that is free of sexism, racism, and other prejudice. And actually this may be hard or even controversial to do - who gets to decide which training material is allowed and which isn't? Where do you draw the line?

Is Real Fast Reports biased?

This was an important question which we considered before launching, and it continues to be at the forefront of our minds.

Fortunately, the way we use AI leaves relatively little scope to introduce bias. The core information expressed in our reports is provided by teachers in bullet point format - the AI is only responsible for converting these bullet points into fluent prose.

However, although the scope for bias is small, the possibility exists. We have taken a number of steps to reduce the risk of bias, e.g. we replace all student names that we send to the AI, so that "Fatima" and "Fiona" are not treated differently.

By taking various steps like this we are confident that Real Fast Reports displays an extremely low level of gender or racial bias (if any). However it is still essential for teachers to read the reports generated critically, and we continue to conduct tests to check for any possible bias in our system.

Footnotes

The four AIs I used for the examples were Davinci, Curie, Babbage and Ada from OpenAI.

How to tile a floor with mackerel, according to AI:

1. Start by cleaning the mackerel. Use a sharp knife to remove the skin, head, and tail.
2. Cut the mackerel into thin slices and place them on a cutting board.
3. Apply a thin layer of adhesive to the floor, making sure to cover the entire area.
4. Place the mackerel slices onto the adhesive, making sure to overlap them slightly.
5. Use a roller or a trowel to press the mackerel into the adhesive.
6. Allow the adhesive to dry completely before walking on the tiles.
7. Once the adhesive is dry, apply a sealant to the mackerel tiles to protect them from moisture.
8. Congratulations - you are now the proud owner of a mackerel tiled floor!