". It's just programmed to seem human." That's a sentence from Ann Althouse, in a post reacting to the frontpage article in the NYTimes recounting an exchange with Microsoft's trial version of a chat bot.
I'm jumping in where I have no knowledge, but that's not the way I understand chat bots like ChatGPT, etc. Aren't they "learning models"? To me that means the programmer is responsible for the IQ of the model, of the bot, but not the content of the responses. So it seems that ChatGPT et.al. will be showing us an average person, "average" based on the context the bot is learning from, which seems to be the usual suspects--white, european, educated etc.
I suppose by controlling the content from which the bot learns the developers can create different personas--say develop a basic personality, then give it a collection of the 500 greatest books in some category with instructions to give the words in them triple the weight.