Without knowledge of how to optimize answers for ‘truth,’ they’re modeling what humans do–tell stories, hedge, prevaricate, lie, do bad math, and sometimes, eventually, suss out the truth.
Long-term evaluation strategies can help determine whether candidates possess a genuine understanding of their field, weeding out individuals who depend excessively on AI to complete their tasks.
…a nice balance of enthusiasm and skepticism for AI risks, digestibility, timeliness, and humanity…