It's really quite simple...
The body is an amazing machine. You wouldn't think that looking at Rosie O'Donnel but still...it is. Your body will adapt to the toll you place upon it.
I've swam 100 laps (back AND forth being 1 lap) across an indoor 25-meter Olympic pool...took me about 2 hours & 10 minutes. The more you swim, the more your body & muscles will adapt to the demand and the easier it'll get...simple as that. At first it'll be tough, but after each swim you'll push your ability further & further. Muscle will build, energy storage will develop and you'll be doing it in no time.
My suggestions...
1) Get an underwater ipod case and discover the fun of swimming with tunes! It makes things much easier and music has proven to give you as much as a 30% perceived reduction in exertion!! I can't swim without my music!
2) Get the goggles that are slightly bigger then those competitive little ones. The little ones cut into your eye-socket and hurt. The slightly larger ones rest more on your bone and are far more comfortable!
3) Make sure you hydrate well, it's very easy for neglect proper hydration because you're in water and your mouth/body miss the cues you have for thirst!
Most people aren't the least bit used to doing upper-body cardio. We can walk, run, job and bike for hours....but swimming is one of those rare exercises that actually incorporate slow-twitch muscles. It'll take time to get your swimming muscles upto speed with doing endurace...but rest assured; they're there and ready to do their thing! The average untrained person will do 4 laps and think they're gonna drown. Once you get your beat down, you'll do 50 no problem!
Eat right, fuel proper and don't underestimate a solid recovery/endurance agent in a drink during/after workouts! Accelerade works well for me.