Modernism was designed to celebrate sunshine, coming as it did right after the long darkness of the first world war. So it is fitting that the coastal town of Eastbourne, one of the sunniest places in Britain, should now have a new art gallery fashioned along Modern lines, with giant windows, bright white walls and seductive curves, all bound up in a faintly nautical style. Elsewhere, Eastbourne boasts lofty stucco terraces, tea dances, a lawn club, neo-gothic schools and clusters of churches, Eastbourne has a genteel, somewhat behind the times air. Charles Dickens and Lewis Carroll holidayed here, and, in the 19th century, the architect Henry Currey was charged with turning the rapidly expanding seaside community into "a town by gentlemen, for gentlemen".

The new Towner gallery gives the place some real edge, even if its architecture is understated rather than overtly sensational. Designed by Rick Mather Architects, its curves are perfectly in keeping with the rolling South Downs, Britain's newest national park, which can be seen from its windows.

The old Towner gallery, which opened in 1923, was housed in Manor House, a handsome Georgian building in the old town. Its collection grew from an initial holding of 22 paintings - of sheep, dogs, virtuous scenes and the South Downs - left to the people of Eastbourne by alderman John Chisholm Towner on his death in 1920.

Over the years, this collection grew to include work by artists who had moved to the area - and, recently, under the energetic direction of the Towner's current curator, Matthew Rowe, to works that tackle the very idea of living on the edge,both literally and metaphorically. By the late 1990s, the collection was too big for its existing home (it now numbers 4,000 works). A decision was made to sell Manor House and build a new gallery nearer the seafront and its circling, squawking gulls. There were promises of £8.6m from various sources, including the arts council, the Heritage Lottery Fund and the local council.

Mather is the architect behind a string of elegant museums and galleries across the US and Britain, including the recent renovation of Oxford's Ashmolean Museum. The new Towner's white, concrete building feels just as seaside design should - and very like Erich Mendelsohn and Serge Chermayeff's De la Warr Pavilion, down the coast at Bexhill-on-Sea. That mid-1930s municipal pavilion, beautifully restored by John McAslan and Partners, has been, in large part, the inspiration for this gallery.

From the outset, the new gallery was planned to be a part of Eastbourne's Devonshire Park cultural centre, a fascinating collection of older civic buildings. Doors at the back of the Towner lead into the lobby of the Congress Theatre, a handsome Modernist building designed by Bryan and Norman Westwood. This, in turn, holds architectural hands with the lovely Winter Garden, next door to Cavendish Park theatre, a grand Victorian design. It all adds up to a delightful architectural mix-and-match, from playful Victorian pomp to crisp international modernism.

The Towner's main entrance leads into an airy lobby with a bookshop and a bright, swirling stairwell. The art begins upstairs, housed on two beautifully crafted floors. On the first is the Towner Collection. Here, in a sequence of rooms shaped on a domestic scale, with warm timber floors and gentle lighting, are paintings, sculptures and photographs. It's a pleasure to see Victorian dogs at play sharing space with the likes of Wolfgang Tillmans's End of Land, a photograph from 2002 which shows a young woman stretched out on the precipitous edge of Beachy Head, a stone's throw away from here.

There are more surprises in store; Mather's architecture isn't the only thing that punches above its weight. This relatively small public gallery also boasts works by Vanessa Bell, Anya Gallaccio, Henry Moore, Victor Passmore, Julian Opie, Tacita Dean, Olafur Eliasson - even Pablo Picasso. (It's a circulating collection, so not all the paintings are on show at any one time.) What makes the Towner so special, though, is the way in which the many views from it, so carefully framed, rival those of the framed artworks hanging on its walls - and the fact that daylight reaches into nearly every corner of its public spaces.

Mather has also succeeded in connecting the galleries to the townscape and the rolling Downs beyond. A massive lift that serves both visitors and artworks has views out through a tall slit of a window, over rooftops and hills.The first-floor concrete corridor connecting the collections and the store, for example, is lit by windows at both ends, and from the side, too. This is the least gloomy or claustrophobic of art galleries. The only sunless room is the spacious, first-floor store, in which the hundreds of paintings not currently on display hang from sliding screens. Parties of schoolchildren will be welcomed here, enjoying the equivalent of a magic show as unexpected clusters of paintings are wheeled in and out of view. (Adults can also see them, by appointment.)

The second floor, though, offers the architectural pièce de résistance. Here, pretty much the whole floor has been given over to one giant gallery for temporary exhibitions. With its five-metre high ceilings, exposed concrete skeleton and finely calibrated artificial lighting, this room shows how even a white-box gallery can have architectural character and a modicum of daylight: at either end, this vast gallery is teased with shafts of sunlight from hidden glazed slits.

The opening show, Nowhere Man, is by Iván Navarro, a Chilean artist whose fluorescent figures, (based on Otl Aicher's celebrated pictograms for the 1972 Munich Olympics), are designed to evoke a sense of the erasure of the individual, both physically and psychologically - as happened during the dictatorship of General Pinochet. Suddenly, Eastbourne seems a less cosy place.

From here, a second, daylit corridor leads into the second-floor cafe. Bright, white, charged with sea air, this is a fine place to sit, chat and while away the time gazing at the complex views surrounding the Towner and to enjoy the aerial antics of marauding gulls. They clearly love Mather's white cliff of a building. A cleaning programme has already been put into effect.

This gallery does Eastbourne proud. The architects have proved that a modern building can be a good neighbour to the "gentlemanly" seaside town. They have understood that Eastbourne, especially with the South Downs' new national park status and the revival of English seaside holidays, is likely to become more popular than it has been in many years. They have not tried to mimic Brighton, and they have shown that a small, thoroughly thought-through art gallery can look beyond the South Downs to the world of the artistic imagination. Mather has brought the spirit of the south coast - its light, its moods, its sense of infinite reach and boundless possilibities - right into the heart of Eastbourne.

Jonathan Glancey