On a sunny spring day, it's easy to re-imagine Eastbourne as the 19th century Empress of Watering Places it once was. That common, sepia-tinted perception has not helped this town of conferences and coach tours to meet the 21st century confidently under the jutting shoulder of Beachy Head. But art will.

The new £8.6m Towner Gallery, designed by Rick Mather Architects, ushers in what may become a Tate St Ives effect in a setting of significance to 20th-century British modern art. There is one key difference: as a gallery, the Towner is superior to Tate's operationally constipated Cornish outstation.

The Towner is not only significant to Eastbourne. It's a sign of big changes in the cultural offers of a string of towns along the south coasts of Sussex and Kent - changes led by bold architectural interventions. Brighton may be Clerkenwell-on-Sea, but the white noise of its cultural hegemony is no longer quite so deafening. The cultural gravities along the south coast are changing, propelled by the "urban renaissance" policies of the South East England Development Agency.

The trigger for this seaside evolution was John McAslan Partners' superb modernisation and extension of the careworn Grade I listed De La Warr Pavilion at Bexhill-on-Sea. The pavilion, widely regarded as the finest example of British inter-war modernist architecture, took a decade to transform in what was a model of perseverance, shared equally between McAslan and Rother District Council. The pavilion is now a major visitor attraction, and Bexhill's seafront and museum are next up for improvement. Pennies have dropped, and tourist pounds are being generated. It's no accident that the first of the hip Big Sleep chain of hotels (owned by Cosmo Fry and John Malkovich) is slated to open in Eastbourne in the future. And if a quiet place like Bexhill can get on the national arts radar and draw tens of thousands of visitors a year, what else might be possible? Littlehampton, west of Brighton, blipped brightly on the cool-things-to-do radar when the architectural impresario, Peter Murray, commissioned Thomas Heatherwick to design the now rather too breathlessly feted East Beach Café; which, in turn, prompted the modest, but charming, West Beach Café designed by Asif Khan.

In Deal, Niall McLaughlin's box-fresh £500,000 makeover of the pier has given the town a new wish-you-were-here postcard image. At Hastings, the Jerwood Foundation has plans for a gallery on the seafront. And Folkestone has gone saltily supernova, courtesy of a massively impending marina scheme by Foster + Partners, and big urban and cultural projects underwritten by the Saga billionaire, Roger de Haan, who has already used noted designers including Hopkins Architects and Alison Brooks Architects.

In Eastbourne, the Towner is a monument to pared down architectural logic, and the pin-sharp vision of its artistic director, Matthew Rowe, an energetic character whose understanding of both popular and art-historical demands is exemplary. He is categoric in saying that the Towner works far better than Tate St Ives, where he was once a curator.

But let's be clear: the Towner isn't rock'n'roll architecture. Despite moments of considerable grace and striking materiality, it is neither iconic, nor tricksily ironic. Mather is a modernist designer, but a restrained one. And he will have noted at least two buildings in the town - the Eastbourne Centre, and the South Cliff apartment block - which suggest that local planners have not always been confident judges of modern architecture. That discomfort has had an effect on one aspect of the design of the Towner, but not a fatal one. More on that in a moment.

In schemes for the Dulwich Picture Gallery, Hammersmith's Lyric Theatre, and the National Maritime Centre, Mather has shown himself as an architect who solves architectural problems as simply as possible; he's an economist of form, a less-is-more advocate. His design of the Lyric, for example, maxes out a minimal site to great graphic and programmatic effect. He had more room at the Towner, but only on one side of it: the building is not only fused to the side of the Congress Theatre, but part of the gallery's ground floor can be used as extra milling space during the conference season - though it's currently housing two very interesting art outreach projects.

On the two upper levels, Mather and his project architect David Watson have produced an absolutely clear divide between the two galleries and other spaces. The galleries are behind concrete walls whose surface is a coarsely abstract porridge of grey tones, yet shines - a brut beton lucida glimmering with reflected light from the south-facing end wall.

And that divide, glazed at both ends and running right through the building from front to back, establishes the architecture's relationship with its urban context: the sea's horizon is just visible from the second floor's circulation area and, at the other end, one looks down on the scene of the pre-Wimbledon Stella Artois tennis tournament. From the café indented high up in the flank wall, there are unobstructed views of the Downs.

The Towner offers something else that's quite special - an archive room open to academics and visitors in which the whole collection of 4,000 works, including a large number hung on 5m long, floor-to-ceiling sliding frames can be viewed by arrangement.

The other key player in this rationalised game of views and building division is the outsize passenger lift: a third of it is glazed, and so is that ascending section of the building's envelope. One rises with clear site of the Downs towards Beachy Head - a view precisely on axis to the end wall of the Towner's collections gallery; on it is hung Wolfgang Tillmans' giant image, End of Land I, showing a girl lying on the very lip of Beachy Head.

This delightful conceit sets up the visitor for the People's Choice collections show, an inaugural treasure trove that brings us Gear, Ravilious, Wynter, Unsworth, Mockford and Pasmore lit superbly by Mather-designed lightboxes in rooms whose volumes mimic the domestic spaces of the 18th and 19th century townhouse that was the first home of Alderman John Chisolm Towner's art cache.

The Chilean sculptor Iván Navarro's neon installation in the much bigger exhibitions gallery is less engaging, despite the fact that the space works well. His subject is the Munich Olympics massacre; but the evenly glowing, flat-mounted, super graphic athletic symbols convey no tension, or hint of subject-matter.

There are more interesting tensions in the overall form of the building. Mather has produced a form that seeks two kinds of clarity. The flank wall, Mr Whippy white in the sun, is a modernist sculpture of some virtuosity, punctuated by graceful cut-outs and mouldings marking balconies and glazing recesses.

Jay Merrick
The Independent