In order to reach the ice caves, you first need to get access to the snowmobile.
How and when do I use the snowmobile?
You'll also need a working flashlight.
Where do I get a working flashlight?
To get to the ice caves, ride the snowmobile out of town. At the intersection, go forward to reach the ice caves.