Narration is vital for Travel. I go for a GM narrates, players roll, players narrate cycle. Without that, journeys are a dice rolling slog.
As for numbers in play, I’ve tried The One Ring with 5 or 6 players (lots of Travel Hazards, but they can handle them) and with 2 (few Hazards, but each is more of a struggle). It balances itself to an extent, though I’ve not run the numbers. One thing though is that Travel can be fairly miserable for a player with a Travel skill lower than 2 (or better, 3).