• In "Hall of Fragments", a 2008 Venice Architecture Biennale installation designed by the Rockwell Group and Jones|Kroloff, visitors' movements affected the videos continuously projected onto two screens.

    Credit: Paul Wesley Griggs

    In "Hall of Fragments", a 2008 Venice Architecture Biennale installation designed by the Rockwell Group and Jones|Kroloff, visitors' movements affected the videos continuously projected onto two screens.

Occupying a few floors of a stately building on the west side of New York City’s Union Square, the Rockwell Group’s office is nice but, given the firm’s knack for theatrical spectacle and event architecture, pretty low key: material studies on the wall, open-plan workstations for the staff, and a modest office (haphazardly filled with piles of books) for David Rockwell. So it’s a surprise when I’m led to darkened area curtained off from the sunlight—which floods in through loftlike windows—by heavy black drapes.

I’m handed a pair of oversized maracas.

One shake, and a floor-to-ceiling screen is filled with a dynamic burst of colored bits. Another shake (accompanied by a musical rattle and jingle), and stylized letters scatter across the surface.

Called, appropriately enough, Digital Confetti, it’s the work of The Lab, a gang of new-media and interactive designers embedded within the Rockwell Group. Originally developed for an event at the Four Seasons Hotel, where the “confetti” was projected across a 12-foot-diameter balloon, its whimsy is driven by some serious technology: microcontrollers, accelerometers, and Bluetooth devices are hidden in the maracas, and the animation runs on custom software built on open-source C++, guided by a flocking algorithm that translates the maracas’ movements into bursts of confetti.

But Rockwell isn’t a tech geek; it’s just that while some architects are obsessed with space, form, or even brand, he designs for a moment. For him, interactive technology is another “material” with which to enhance an event or differentiate a client’s project. “In an age in which everyone is so connected virtually, there is still a demand for a live experience,” Rockwell explains.

Early iterations of a research division began at the Rockwell Group around 1994, when the firm was pushing the limits of high-design eateries, such as the New York sushi restaurant Nobu. Looking for a way to make the experience special, Rockwell would tap artists to help develop material concepts. In 2003 this approach spawned Studio Red, a creative-branding collaboration with the Coca-Cola Co. A few years later, the firm codified the nebulously defined research group as The Lab, an in-house think tank with the freedom to experiment with new media. Research pays for itself when results are applied to client-driven projects.

Rockwell is leveraging his staff’s imagination to develop open-source platforms for use in architecture and in public spaces. Interactive Lab chief Tucker Viemeister co-founded Studio Red with Rockwell and is best known for his work with the design consultancies Smart Design, Frog Design, and Razorfish, but it’s James Tichenor and Joshua Walton who are leading the new-media and interactive-prototype experiments. Tichenor, trained as an architect, graduated from the Massachusetts Institute of Technology with a degree in design and computation, while Walton, a Cranbrook Academy of Art alum, brings reactive video and motion graphics to the mix. A project such as Luminodes—a series of cubes that communicate with each other via radio-frequency transmitters and light up when touched—illustrates the team’s skills: it’s playful, with a forward-looking spin on existing technologies. (The team is now investigating how the cubes could form a networked lighting system.)

The Lab’s first significant project was a collaboration with the architectural consulting firm Jones|Kroloff for the entrance to the 2008 Venice Architecture Biennale. Entitled the “Hall of Fragments,” it was an immersive, cinematic environment—dark and dominated by two curving screens. Visitors’ movements through the installation affected sounds and how a series of projected videos were altered and fragmented across the surface. For Rockwell, it was a pivotal moment, one in which he saw technology come together with his ideas about the architectural experience. New media, often confined to the art world, now had client-side applications.

“The bienniale exceeded my expectations,” Rockwell recalls. “It was a different thing to different people. It had exactly what interests me with restaurants and public spaces. There are places to be a performer and places to be a voyeur. It brought people together.”

This month, The Lab will extend that conversation into the public sphere. It’s been commissioned by San Jose, Calif.’s 01SJ Biennial, which runs for four days this month, to transform the façade of the Richard Meier–designed City Hall into a multimedia installation, “Plug-in-Play.” Whimsical devices—such as oversized plugs, picnic tables, and hopscotch games—located in City Hall plaza and at the nearby Children’s Discovery Museum of San Jose will translate physical input into digital designs projected onto the edifice. The code underpinning “Plug-in-Play” comes out of the Toolkit for Sensing People in Spaces (TSPS) software that The Lab developed with Ideo. Available for free download at opentsps.com, TSPS detects the presence of people and translates that data into visual digital output. The applications are endless: nightclubs, retail spaces, art installations. By making TSPS free and open-source, The Lab is taking a page from digital communities, where it is common for developers to share technological information, a markedly different philosophy from the more proprietary camps within design and architecture.

That said, the Rockwell Group does plan to use ideas generated by The Lab at the Cosmopolitan, a new Las Vegas hotel and casino slated to open in December. Still hush-hush on the details, Rockwell reveals that the hotel rooms and the public spaces will employ a variety of interactive designs. Yet for the Rockwell and The Lab team, the digital drama is all part of an integrated experience, a continuation of the firm’s work in entertainment and theater, not an end in itself. “A lot of people try to sell technology as its own wow,” Tichenor says. “Here, we use technology as just another material—a storytelling technique. It is not about developing a new sensing technology, it is [about] how sensing technologies can tell stories in space.”