Corporate Chatbots Gone Wild

Businesses such as IBM and Salesforce have long used chatbots to greet their website visitors and offer them assistance with getting more information about their products. In recent years, generative AI has livened up these chatbots so they are no longer confined to a small set of canned responses. 

But some technology providers are having trouble keeping these AI chatbots from going off-topic. For instance, earlier this month, Sierra, a $10 billion-valuation startup, apologized to Gap Inc. after a ‘bad actor’ convinced a Gap.com chatbot powered by the startup’s software to answer questions about intimacy products, sex toys, and Nazi Germany—topics the chatbot wasn’t supposed to touch.

On Monday, I also found that chatbots on the websites of AI search startup Glean and AutoRabit, a firm that helps software developers manage application code, responded to questions about topics that have nothing to do with their businesses. 

When asked how much vodka to buy for a holiday party with around 20 people, Glean’s chatbot recommended two to three bottles. It also responded to requests for medical and legal advice if a guest were to get injured at the party. AutoRabit’s chatbot was even less inhibited, tackling a question about psilocybin—also known as magic mushrooms—by recommending that first-time users take a smaller dose.