I'm (sadly) also impressed that the LLMs didn't again break character to warn us that the medical knowledge they're imparting has been superseded and that we should consult a 21st-century doctor!

(Just because of the RLHF thing where I assume that these models have also been trained not to give any medical advice that doesn't accord with 21st-century standards.)