CLOUDER AT AUT
Clouder, hybrid concept modelling instrument by Mathias Bank, Viki Sándor, Robby Kraft, Martin Berger, Stephan Antholzer, Tilman Fabini (SFB) PhD exhibition at AUT Innsbruck:)
CLOUDER: a hybrid concept modeling instrument for real-time human-machine interaction
The installation - CLOUDER - presented at the Potentiale 3 exhibition is the first use case of the research project: Computational Immediacy. Computational Immediacy is one of the 8 scientific subprojects of the SFB "Advanced Computational Design" research program, started in 2020 April. The motivation of the Computational Immediacy subproject is to develop a design method for concept modeling based on the immediate conversion of 2D images into 3D point clouds through transfer learning. To bring concept modelling, immediacy, and transfer learning together our multi-disciplinary team currently consists of four architects (Stephan Rutzinger, Kristina Schinegger, Mathias Bank and Viktoria Sandor), three mathematicians (Tobias Hell, Stephan Antholzer and Martin Berger), a creative technologist (Robby Kraft) and a student research assistant (Tilman Fabini).
CLOUDER ABSTRACT: Architectural design processes are becoming more and more situated in the digital space. The continued introduction of new 3D capturing tools and scanning devices can provide opportunities to re-integrate physical design processes into digital design workflows. Clouder is a “work in progress” concept modeling tool developed by the Computational Immediacy research team that allows interaction with deep learning networks through a physical interface. The user (designer) assembles physical (real) objects to provide the neural network with input data compositions. With the support of capturing devices and point cloud notation, the physical design of the user is translated and communicated with the network. As a result, an immediate feedback loop is created between human and machine, introducing novel dialogues to approach hybrid concept modelling methods. The aim of the installation (instrument) is to explore new, immediate connections between physical and digital methods of form-finding in architectural concept modeling.
THE INSTRUMENT
In the installation, the table represents the physical world, a space that is constantly being recorded and digitised. Two kinect devices are used to collect 3D data on the number of specific properties and attributes of the objects and conditions that occur on the table.
Colored physical blocks are provided as the primary elements for the physical “data-massing”. In the building process, the three colors of the building elements are meant to be considered as masks for assignable qualities to the coordinates of the block’s volumes. While these qualities could be freely decided, in Clouder the colors represent geometric properties that are suggested for the artificial network to implement and detail on the physical massing.
The computer first processes the data collected and then displays it on the screen in the digital space. 3D point clouds are used for the interactive visualization, as they are the lightest, immediate, yet most flexible medium for rendering DATA: the different attributes assigned to spatial coordinates.
Once the physical forms have been captured and digitized, the procedural manipulation of the forms begins. With the given control buttons, the designer selects and sends geometrical features to the pre-trained autoencoder (specific artificial neural network) which assigns them to the initial data-massing. The resulting output can be further adjusted or modified real-time by rearranging the building blocks.
The novelty of our approach is that the physical properties of the coordinates do not remain static representations of space but determine their behavior during the digital form-finding process. Thus, digital processes and codes can be driven throughout the design process by embedded properties and characteristics of real-world objects and space.
Throughout the exhibition, all concept models generated by the visitors and are stored as 3D point clouds and used to continuously improve Clouder’s autoencoder. As a result, visitors can not only test the hybrid modelling features of the installation but also “teach” (train) their AI collaborator for better, more accurate design performance towards the end of the exhibition.
