Page under construction.
Latest paper: Multimodal Word Sense Disambiguation in Creative Practice – ICMLA 2020
Language is ambiguous; many terms and expressions convey the same idea. This is especially true in design fields, where conceptual ideas are generally described by high-level, qualitative attributes, called design intents. Words such as “organic”, sequences like “the chair is a mixture between Japanese aesthetics and Scandinavian design” or complex structures such as “we made the furniture layering materials like ‘a bird weaving its nest'” represent design intents. Furthermore, most design intents do not have unique visual representations, and are highly entangled within the design artifact. This leads to complex relationships between language and images in design contexts.
Despite advances in machine learning (ML) in vision-and-language representations, state-of-the-art ML models are unable to efficiently disentangle such relationships and are, consequently, incapable of modeling their joint distribution. A real-time understanding of design intents could open new design scenarios (e.g. voice-assisted natural language input), that reduce procedures based on intent reinterpretation as imperative commands —move, circle, radius, extrude, vertical— required by digital design engines.
As an alternative to current design frameworks, we propose a first-of-its-kind multimodal object-agnostic framework to disentangle, visualize and generate design intents to support a transition from CAD—Computer-Aided Design— to AIAD—Artificial Intelligence-Aided Design.