Is it a good idea if architects start seeing themselves as artists? Rowan Moore salutes a refreshingly rigorous argument

Ours is a time when art looks more and more like architecture, and architecture looks quite like art. Now rising at the 2012 Olympic Park is the Orbit, a pile of steel composed by the artist Anish Kapoor, which has things like lifts and stairs, serious engineering, and the scale of a building. Olafur Eliasson has just finished a spectacular glass wrapping to the Harpa Concert Hall in Reykjavik which has attracted a lot more attention than the parts by the actual architects of the project, Henning Larsen.

The Serpentine Gallery in London, a place dedicated to visual art, presents an annual pavilion, designed by an architect, as if it were the work of an artist, which is then sold to collectors. Architects themselves profess to be inspired, with varying degrees of credibility, by the likes of the American artist James Turrell. "Minimalism" has turned from an artistic movement to an architectural style to an interior design option. Office towers purport to be "sculptural", or else use tricks of perception borrowed from conceptual art. This co-mingling is the subject of The Art-Architecture Complex and, according to the book's author Hal Foster, it is "now a primary site of image-making and space-shaping in our cultural economy". As the half-sinister title suggests, with its echoes of Eisenhower's warnings about the military-industrial complex, and the suggestion of complexes in the psychological sense, the merging of art and architecture is not necessarily a good thing. It can become, suggests Foster, a means of blurring our consciousness, a new opiate of the people supplied by corporations and governments as they use "iconic" artworks and buildings to sell cities and property to investors.

He starts by taking us through major architectural movements of the last half-century, including the way pop art influenced both postmodernism and what became the hi-tech architecture of Richard Rogers, Renzo Piano and Norman Foster, which then led to a "global style" of steel and glass, more or less the same everywhere. In the case of Rogers, this global style takes the form of "pop civics" – law courts and assembly buildings and our beloved Millennium Dome, which profess accessibility and public engagement. In the case of Renzo Piano the result is "light modernity": elegant, refined structures that might be a Hermès store in Tokyo or a cultural centre in New Caledonia.

Foster describes the influence of Russian suprematist and constructivist art on Zaha Hadid, and the effect of conceptual art on the Americans Diller Scofidio + Renfro, the creators of New York's High Line park. Also, the use of both minimalism and pop by the Swiss Herzog & de Meuron, creators of Tate Modern and the Beijing Bird's Nest stadium. He then examines the question from the other side, looking at the spaces and constructions of artists like Dan Flavin, Donald Judd, Robert Irwin and (especially) Richard Serra, before concluding with an extensive interview with Serra.

For him the stuggle is between the "imagistic" (bad) and "embodiment and emplacement" (good), or between "stunned subjectivity and arrested sociality supported by spectacle" and "sensuous particularity of experience in the here-and-now". One supports our sense of who we are, in relation to ourselves and other people; the other is a ruse of globalised capitalism to induce numb passivity, "in the guise of our activation". This is performed through something called the "experience economy", a modern version of the ancient Roman panem et circenses, only without the bread. All pretence that the cultural is separate from the economic, says Foster, is finished.

Of course, one of the features of building-sized artworks, and of artistic buildings, is that they require a lot of money to make, which implies a compelling economic argument to pay for them. (Hal Foster, a native of Seattle, and now a professor at Princeton, was a classmate of Bill Gates, which may or may not give him special insight into big money.) All the architects he describes succumb, one way or another, to the curse of the imagistic, as do many of the artists. Richard Serra emerges as a hero of the embodied and emplaced, with his large, physical sculptures where you can see the marks of their making, and which require you to walk round and into them.

There are, nevertheless, consolations: Foster is appreciative of, for example, "the mixed condition" in the work of Diller Scofidio + Renfro, by which he means their combinations of art, video and architecture, and of the new park of the High Line with the old viaduct on which it is formed. Despite worrying that they "might be a front for capitalist modernity", he sees the possibility that they might "not simply smoothen".

As an architecture writer reading Foster, who comes from the direction of art theory, I find it refreshing to encounter a degree of intellectual rigour (if also, sometimes, opacity) you don't find too often on my side of the fence. Indeed, it requires a certain gentleness on his part, when dealing with the artistic pretentions of architects, to stop them collapsing too quickly under his probing. On the other hand, he sometimes treats buildings too much as artworks – as things to be looked at and walked around, that stand or fall by their inherent conceptual strength – rather than as things of use, to be inhabited, which are enmeshed in function and finance.

I'd also question his polarity: is image always such a bad thing, and can it in any case be avoided? But his basic premise is compelling – and he uses it to powerful effect – to reveal the gap between the reported effects of buildings and art pieces, and their actual ones.