A small vessel perched on top of the Queen Elizabeth Hall has become London's most coveted hotel room

The river Thames has a way of defeating plans for its jollification. For decades architects have looked on its great, tempting emptiness and felt an irresistible urge to propose beaches, inhabited bridges, lidos, zones for festivals fluttering with pennants and balloons, places to promenade as if it were the edge of the Mediterranean. In the 1980s Richard Rogers imagined an archipelago of pleasure, with the forms and construction methods of oil rigs remade into towers and pinnacles of fun. Most recently, the architects Gensler proposed the floating hospitality suite they called the London River Park.

Mostly these plans don't happen. The river flows on, lugubrious and imperturbable, which is possibly because, as Joseph Conrad observed, it is not really a fun sort of thing. "And this also," he wrote in Heart of Darkness, "has been one of the dark places of the earth," as he embarked on that book's journey into forms of savagery that lay beneath a veil of civilisation. For him it was the "sleepless river" of a "monstrous" and "brooding" city. "What greatness had not floated on the ebb of that river," he also wrote, "into the mystery of an unknown earth!"

One Thames project that has happened is A Room for London, a boat-like object perched high on the roof of the Queen Elizabeth hall at the Southbank Centre, as if stranded there by a receding deluge. Where many Thames proposals want to put things of land on to water, this puts something riverine – a boat – on to land. It is a temporary structure, a cross between building and sculpture, by the architect David Kohn and the artist Fiona Banner. It contains a single hotel room which anyone can in theory book, if with rather more difficulty than Olympic tickets. When nights for the first six months were made available they sold out in 12 minutes; the next batch goes on sale on Thursday (at £120 a night).

This little space is the production of an impressive array of cultural impresarios: the Southbank Centre, Artangel, and Living Architecture, the organisation set up by the writer Alain de Botton to build beautiful new houses which can be rented for holidays. It comes, like many cultural projects in 2012, with an Olympic tag, being officially part of the cultural Olympiad. As well as paying guests, writers, artists and musicians have been invited to stay there, and be creative.

From the outside the jaunty vessel seems to fall within the "fun" category of Thames projects. It juts perkily into the void, and three little wind turbines, like displaced propellers, whirr on the top of a triangular rig. It is a toy, palpably and deliberately incongruous. It is a folly. But it turns out that its makers also had Conradian ambitions. The boat is called the Roi des Belges, after the vessel in which Conrad himself sailed up the river Congo, in the journey that would inspire Heart of Darkness. Inside there is a cabinet containing old maps of the Thames and the Congo, in reference to the parallels that Conrad made between the two rivers. An octagonal table and a box of dominos echo similar objects described in the master's novels.

There are other inspirations. The intricate house and museum of the architect Sir John Soane is cited by David Kohn as a help in designing the "episodic" sequence of small spaces that are inside the boat, as you progress from a little vestibule to a galley, to a bedroom that opens up to penthouse views of the river, bracketed by the Palace of Westminster to the left, and St Paul's Cathedral to the right. Alongside the river maps there is a copy of a drawing by Soane's collaborator JM Gandy that shows Soane's Bank of England as if it were a Roman ruin, and which might be taken as a comment, if desired, on financial calamity, or on the fragility of civilisation described by Conrad. Kohn also mentions the baroque architect Nicholas Hawksmoor as an influence, even though his heavy white stone churches would come top of most lists of Structures Least Likely to Float. The spire-like superstructure of A Room for London refers to these churches, and to the spires of London in general.

The main point, says Kohn, is to combine the intimate and the epic, in a way not unlike the relation of domesticity to vastness that you get in boats. "The interiors feel comfortable and you know what to do there, but it's not just an easy or twee kind of comfort. You are connected to the Thames, to a wider world, also to what one thinks of the world. You have a relationship to disputed, uncertain territory."

In all this the intention was to avoid kitsch and creating a one-line joke. The timber-lined interior, stained in places in rich pinkish-red, is not pushed to the point where it is literally boat-like in every detail, but rather seeks other architectural qualities, which is where the influence of Soane comes in. It was also important to Kohn and Banner that the structure was exactingly well made, by the specialist company Millimetre. "It is solid; it has a kind of earnestness," says Kohn, which keeps it away from being a stage set.

And so the lucky purchasers of nights in the hotel room, the intellectual aesthete's equivalent of Willy Wonka's Golden Ticket, will be able to contemplate the "venerable stream" much as Conrad's characters did in the cruising yawl Nellie. At sunset they will be able to watch the gloom "become more sombre every minute, as if angered by the approach of the sun". They can, should they want to, think their thoughts about the world and their place in it.

A Room for London is small, and temporary, and will only be fully enjoyed by a few people. It is not a prototype for future Thames-side development, and offers no solutions to the problems of urban regeneration. It may, even, not quite match the fathomless profundity of its inspirations, being rather an enjoyable and well-made jeu d'esprit. But I have a feeling it will give satisfactions that other Olympic projects will not match: it is intelligent, witty, pleasurable, and is based on observing its surroundings as they actually are, rather than imposing a bombastic idea of what they should be.