These caves are confusing, to be sure. It's easy to get lost, and it all looks pretty much the same.
For most of the game, the only things to find in here are a glove that you can't take and a batch of those flower sketches you've been finding around.
Here are directions to get in, get those sketches, and get back out again.
Sketches in Ice Caves
When you get to the endgame, you'll need to go farther. Until then, just be glad you got out again.
Endgame: Tracking Down the Treasure