I drop in a PDF showing panel schedules, and ask it to tell me what electrical loads are higher than 30 amps, and it confidently gives me a list of loads with their ampacity, and it's all made up.
The problem I have with LLMs, is if you know enough to double check it, it's often quicker to just look it up yourself rather than babysit it, and if you don't know enough, then you don't know when it's making stuff up. I'm simply not confident enough in it to trust it... yet.
