Quickly, point in the general direction of the North Pole with one arm. With your other arm, point to the South Pole. What's the angle between your arms?
If you look like Fig. 1, take a moment to think about it. And if you're still thinking about it, peek at the figures at the bottom of the page.
Since we're used to pointing in the direction of north or south, sticking our arms straight out seems perfectly natural as we conflate the direction north with the location of North Pole. But I suspect there's another reason why many of us instinctively keep our arms horizontal when asked to point to the poles: Despite over two thousand years of knowledge about the shape and size of our planet, we still have a built-in tendency to think of it as flat! Look outside, what do you see? A flat Earth. When we look at a map (other than a globe, of course), we subtly reinforce our sense that we're on a two-dimensional surface. I'm not suggesting that anyone today is a genuine Flat-Earther, of course. I'm saying that it wasn't that silly for countless generations of intelligent people who lived before the ancient Greeks to believe the Earth was flat.
By the way -- and contrary to the popular myth promulgated by Washington Irving -- no one really thought Columbus would fall off the edge of the world as he sailed westward across the Atlantic. Globes had been existence since at least the time of Pythagoras, around 500 BCE. Three hundred years later, Eratosthenes of Cyrene came up with a good estimate for the circumference of the Earth by comparing the angle made by shadows in different parts of Egypt.
As the illustration shows, lines from any point on a circle's circumference to the ends of a diameter are at 90 degrees, so no matter where you are (with a couple of exceptions), when you point to the two poles, your arms form a right angle. The North Pole is always down, below your horizon, as is the South Pole, and the poles are at the ends of one of the Earth's diameters, its axis of rotation.
What are the two exceptions?
Barry Evans (firstname.lastname@example.org) thinks about falling off the edge of the world when he kayaks around the far side of Indian Island.