You can get this behaviour through all sorts of means.

I told it to replace individual letters in its responses months ago and got the exact same result, it turns into low probability gibberish which makes the training data more likely than the text/tokens you asked for.