I think its a great idea to challenge players with relatively "mundane" problems, such as sleep deprivation, weather, cold / hot environments, high altitude, other .
Obviously, though, your goblins will not be able to harrow the players as they move through, without incurring the fatigue...