Oh dear! I laugh, but it's the kind of thing I'd be capable of too, even though I've generally got a pretty good sense of direction. If I stopped and thought "Okay, the sun rises there, so north is over there," I'd be fine. But it's very easy to go "well [town] is north of us, [town] is vaguely that way when I'm standing on the corner down there, that's north!" and not pay attention to the fact that the road bends* or something, and just not stop to think about the data that's in conflict.

Once I've stopped to sort it out, I'll generally remember that okay, that landmark is north so we're traveling southwest right now, or whatever, but I don't always stop to sort it out for quite a while.

*I live in Boston. Straight roads at right angles to each other are a weird and disconcerting rarity.