At first, these caves seem very simple -- you just go forward until you come to a dead end, and that's all you can do. (Except notice a glove someone left behind.)
But if you click on the solid wall of ice at the dead end, Nancy says she could break it down with something sharp. Do you have anything like that?
You do if you picked up the hook from the fish gutting station in town. Hack away to find more caves.
From this point on, the caves get more dangerous. It's easy to get lost, and you need to be very careful about using that hook again. Always save before you use the hook on a wall because most of them will come down on you.
In fact, it's a good idea to save before you go in, then save the game with a different name when you find something interesting. Second Chance will help you if you bring a wall down, but it does nothing for you if you get lost.
The first time you enter the ice caves, there are only two things to find here: that glove, which you can't take, and another collection of those little flower sketches you've been finding.
To get to the glove, from the entrance, go forward, right, right.