Your phone might be as secure as it can be, but the Dr's phone/smart speaker/doorbell/etc may have picked up on what was said, detected your device/s via bluetooth and attributed it to you.
It's not just your devices watching and listening, it's everyone else's devices as well.