"We have no intention of changing men's habits, but we have hopes of proving to them how fragile their thoughts are, and on what cellars they have erected their unsteady houses."

Surrealist Declaration of 27 January, 1925

For the surrealists a house was not a concrete structure, it was a metaphor to get at the marvellous, the startlingly strange. Despite being trumpeted as the first exhibition to "explore the relevance of surrealism for architecture", what this new The Surreal House exhibition at the Barbican does best is explore the richness of the house as a symbol - of the body, of imprisonment, of fragility, of the womb. Equally, much has been made of the architect-designed gallery but, except for the occasional flourish - a nipple doorbell (Prière de toucher, Marcel Duchamp, 1947 - originally the front cover of a book), a piano hanging from the ceiling (Concert for Anarchy, Rebecca Horn, 1990) - this is no gaudy funhouse. Rather, it's a compartmentalising and confusing of the gallery space so that the works are tucked into dark corners or linked by surprising sight lines; it's a disorientating, allusive dream world.

As in a dream, here there is no strict corralling or rationalising of content. The exhibition's foundations are in the Surrealist movement of the '20s and '30s, but it drifts around the centuries and from painting and sculpture to film and architecture, to pull in any building more ambiguous than the Modernists'  "machine for living in" (ironically, The Surreal House is sponsored by Ikea).

We start, as did so much of surrealism, with Freud and his notions of the subconscious and the uncanny. For Freud, the house could be equated with the body, a container of the self. While this was hardly a new idea - on display is an engraving after Hieronymus Bosch's Tree Man (c. 1505), in which a human body doubles as a deformed, cracked shell of a house - it is one that the surrealists would return to obsessively.  André Masson, for instance, after being wounded in the First World War, imagined himself as a minotaur split open to reveal a labyrinthine interior.

For Donald Rodney, a sufferer from sickle cell anaemia, the body was a much more fragile dwelling, and while in his hospital bed he made a tiny house out of his own skin (1996-7). This idea of the house - and by extension the world that we live in - as fragile and unstable, was another surrealist theme and explains their enthusiasm for Buster Keaton, who in Steamboat Bill Jr (1928) stands still while a building facade collapses around him. Later, Czech animator Jan Švankmajer would, in Jabberwocky (1971), make a similar point: set in an anarchic playroom, dolls are dismembered and toy castles knocked down; the established order crumbles.

Of course, one person's house of straw is another's cage. Louise Bourgeois' Femme Maison series, with its half-house, half-female figure, testifies to the manner in which, for women, the domestic appears to offer sanctuary but is instead a prison. And the house can be integrated into our selves in other ways. In Ed Kienholz's fantastically macabre installation The Wait (1964-5), an old woman's skeleton sits in a dusty living room with a jar over its skull, surrounded by family photos – like the room, dead and empty but for memories; when an old man in Andrei Tarkovsky's The Sacrifice (1986) torches his beloved dacha he destroys a part of himself.

Perhaps where house and dream intersect most vividly is the haunted house. Edward Hopper's House by a Railroad (1925) is certainly uncanny, with its half-shuttered windows hinting at terrible mysteries within, while Švankmajer's Down to the Cellar (1982) teems with the shadowy monsters that lurk in the depths of our subconscious.

In the upper gallery the focus switches to architecture and, after the density of the first part of the exhibition, what feels like a stretching after relevant material. So we get a film of André Breton's Paris flat, crammed with geegaws, and photos of the scantily-clad ladies swimming round Salvador Dali's Dream of Venus pavilion (built for the 1939 New York world fair), but no sense of a surrealist blueprint for a new model of real-life buildings. Instead they claimed as their own a French postman, Ferdinand Cheval, who spent 33 years building his Palais idéal in a style that can be charitably described as naive rococo. Later, Rem Koolhas would look to Dali's famous Sleep (1937) for inspiration for his house on stilts, the Villa Dall'Ava (1985-1991).

The most marvellous architectural ideas here are not bricks and mortar projects at all. The intrauterine house, as advocated by Tristan Tzara, fed the idea of house as metaphor back into architecture, to make the womb-like cave the basis for a dwelling that would "symbolises prenatal comfort"; Frederick Kiesler worked for years on the concept of an Endless House, a series of egg-like pods opening onto each other in a flow of organic forms, unlike the straight lines and rationalism of the Modernists. The unsteady surrealist house found physical expression in the work of Gordon Matta-Clark, who in 1974 cut a house in two, and in the theories of French-Swiss architect Bernard Tschumi: reflecting on the ruin of Le Corbusier's Villa Savoye, Tschumi concluded that buildings only truly came alive when on the point of collapse.

Solid, functional houses then, were largely antithetical to the surrealist project. Rather, the point, as this exhibition eloquently demonstrates, was to draw attention to the hidden, the subconscious and the astonishing. Imaginative curating, while occasionally veering into the extraordinarily tangential, brings new resonances out of the individual pieces by placing them in this context. We are invited to consider them outside of any chronological movement or strict theme and so we form more personal relationships with them. This results in an illuminating and intense experience, but also ultimately, after we leave, the lack of an overarching shape to hold the pieces together in the memory causes them to collapse and fall apart, and the exhibition fades like a dream.

The Surreal House is showing at the Barbican Art Gallery, London EC2Y 8DS, until 12 September 2010. Standard tickets are £8 online, £10 full price. For more information, visit the Barbican website.