But the gallery's exterior is different. When I examined some photos of it before my visit, its faintly neo-expressionist wall made me wonder if Mather, one of the last rational-minimalists, had finally caught the baroque bug. 'Oh, we like curves,' he says later on the gallery tour.

Actually, in reality there's just the one curve - the site perimeter - but it's used for all it's worth. It drives the division of the plan into two parts: a square, black-zinc-fronted, concrete-wrapped gallery half; and a white, curvy-walled half, separated by the big corridor. The curve gives the white half its expressionist tectonic, its long slot for the lift view, its deep-punctured artworks access and its curving cut-out balcony. Curiously, that curved facade folds into nothing as you leave the building. Coming up aslant from Eastbourne's seafront and Martello tower, you can't see it at all. The sinuous bit opens up only if you come right round one side. Maybe it is a baroque trick after all: curious and clever, but just too small and cool for the chunky, disjointed townscape.

The Towner is the latest addition to Rick Mather Architects' set of museums and galleries, which also includes the Dulwich Picture Gallery extension, the expansion of London's Wallace Collection, the Virginia Museum of Fine Arts in the US and, coming soon, the Ashmolean in Oxford. As Mather obligingly recounts, all use similar elements: street frontage; views out to reduce 'museum fatigue'; easy-to-follow routes; the look of the building as a function of its site; environmental factors like concrete and displacement ventilation. As complex as these spaces are, they appear simple in the Towner.

The gallery's original site was between the fabulously dumpy Grade II-listed Winter Garden, designed by Henry Currey and built in 1875, and the 1960s Congress Theatre, with its Festival-Hall-meets-East-Berlin interior and dour exterior by Bryan and Norman Westwood. But Mather wanted the site to be beyond the also Grade II-listed Congress Theatre - bigger, with another axis to the sea and his preferred neighbour. From the Devonshire Park tennis club behind the buildings, the logic of this is clear: a string of three, chronologically arranged villas. But from the large-scale road junction to the front, something odd is going on. The Towner pretends to be part of the Congress Theatre, pulling its curvy side out at the last moment. It's an interesting composition, but it has to be said - in context, it's underwhelming.

Inside, though, you're clearly in a bit of urban-industrial class. The lightbox ceiling panels in the big, serious galleries impose a geometry under which the flexible space can be divided up easily, but always proportionately. Typically for Mather, the Towner is well-tempered environmentally, with exposed concrete ceilings for heat retention and displacement ventilation through the walls. The building feels controlled.

This is not everyone's cup of tea, and regretful mutterings (from taxi drivers and art critics alike) about the lovely 18th-century manor house that housed the Towner collection from its formation in 1923 add to that prejudice. But the old Towner didn't provide contemporary access standards, and that's the key. The best thing about the new Towner is the marriage of its patrician-feeling architecture with a programme that utterly contradicts it. Its astonishing outreach programme, impressive enough in its work with schools, also includes young offenders,
the homeless and other highly disadvantaged people. Richard Beales, outreach and inclusion manager, calls Towner an 'aspirational building'. Something grittier, looser might not have conferred the sense of recognised value in the way Mather's cool urban-Palladianism does.

Another programming treat is the fabulous storage galley, where you can see any of the works in store on great sliding racks. Like the lift, it's a thrilling opening of back-of-house activities to the public. But there's shrewd work everywhere from artistic director Matthew Rowe. The shows with which the Towner reopened earlier this month demonstrate the flexibility of the galleries - one remains a big box, while the ground floor divides into two big galleries, one opening, surprisingly, across the site's full width.

Indeed, this highly ordered building has a lot of flexibility and the capacity to surprise. It's much bigger than it looks, and can open at all levels into the Congress Theatre. There's cross-programming with the annual International Ladies Tennis Tournament held at Devonshire Park. The whole building, especially the café, is geared to overlook it, but the ground floor will also act as the tournament entrance. Towner's underlying order helps this remarkable piece of cross-programming to work.

While I admire all that Mather has done with the Towner immensely, it gives me the frustrating sense of a well-trained animal held firmly in check. If a young, inexperienced Rick Mather were one of my students, I'd be trying to get him to do something against type. Current baroque hype has produced nothing with the spatial thrill of a Guarino Guarini, but Mather's work has that absolute sense of volume, form and materiality that suggests he might be able to achieve this. He could design with more complex geometries, or with the contextual problems he's avoiding. I'm sure he's entirely capable of shifting into another league, but I wonder if he wants to do that.

Kester Rattenbury