Articles

Generative AI (ChatGPT and image generation) for architectural and civil engineering domains

Generative AI (Generative AI), such as ChatGPT (interactive AI) and Midjourney (image generation), has been the topic of much discussion and is expected to revolutionize society and industries.

What is Generative AI?

Definition of Generative AI

Since generative AI is in its infancy, definitions of the term differ slightly.

Generative AI refers to a category of AI that uses systems called neural networks to analyze data, find patterns, and use those patterns to generate or create new output, such as text, photos, video, code, or data.

Microsoft (Microsoft News Center, 2023)

Generative AI refers to deep learning models that can generate high-quality text, images, and other content based on data that has been trained.

IBM (Martineau, 2023)

Generative AI refers to algorithms (such as ChatGPT) that can be used to create new content such as voice, code, images, text, simulations, and video.

McKinsey & Company (McKinsey & Company, 2023 )

In summary.

Deep learning is used to
Based on trained data
Generate new content based on learned data

The key point of the definition of generative AI is that it uses deep learning to generate new content based on learned data.

History of Generative AI

There are two main streams of development in generative AI: natural language processing and computer vision.

In natural language processing, it started with Hidden Markov Models (HMMs) (Knill & Young, 1997)and Gaussian Mixture Models (GMMs) (Reynolds & Others, 2009), which generated continuous data such as speech and time series; since the advent of deep learning, the performance of generative models has improved significantly, with Recurrent neural networks (RNN) being introduced for language modeling tasks and being able to model relatively long dependency relations; after the Transformer architecture (Vaswani et al., 2017) was developed, this model has been the main backbone of various subsequent generative models, including GPT, to the present.
コンピュータビジョンでは、Texture synthesis (Efros & Leung, 1999) and Texture mapping (Heckbert, 1986) から始まり、これらのアルゴリズムは、手作業で設計された特徴量に基づき、複雑で多様な画像を生成するというものでしたが、大量に生成する能力には限界がありました。2014年には、Generative Adversarial Networks (GANs) (Goodfellow et al., 2020)やVariational Autoencoders (VAEs)など、拡散生成モデル(Song & Ermon, 2019) などの手法も開発され、画像生成プロセスをよりきめ細かく制御し、高品質な画像を生成することができるようになり、その後、Transformer architecture (Vaswani et al., 2017)をコンピュータービジョンと組み合わせることでこの概念をさらに進め、画像ベースの出力を可能にしました。そしてこれによって、マルチモーダルなタスクを実現することも可能にしました。

In the future, we can expect to see more accurate generation and multimodalization, where the types and combinations of inputs and outputs will become more diverse.

Types of Generative AI

The major generative AI models that have emerged in recent years are classified according to the form of input and generation as shown in the figure below (Gozalo-Brizuela & Garrido-Merchan, 2023).

The input is often text, as it is based on theTransformer architecture (Vaswani et al., 2017), which is specialized for natural language processing ; the generated formats include not only text and images, but also 3D, video, audio, program code, etc. Flaming and VisualGPT have images as input format.

Classification of the main generative AI models (Gozalo-Brizuela & Garrido-Merchan, 2023)

How Generative AI is Used in Architecture and Civil Engineering

This section describes how generative AI can be used in the architectural and civil engineering domain, beginning with an overview of how AI has been used in the architectural and civil engineering domain, followed by a description of the potential applications of generative AI.

Examples of AI Applications to Date

Planning and Design Process

Design generation and emulation (Huang & Zheng, 2018; Liu et al, 2017)
Design evaluation (Lorenz et al, 2018; Y. Zhang et al, 2018)
Prediction of structural response and performance, interpretation of experimental data, and pattern recognition of structural health monitor data in structural design (Sun et al, 2021)
Create BIM models from laser scanner data (Tang et al, 2010)
Analyze BIM logs and investigate ways to improve design productivity (Pan & Zhang, 2020)

Automated facility design service (PillarPlus)

https://pillarplus.com/ai-tech/

Construction process

Create processes that minimize project finance costs(Alavipour & Arditi, 2019 )
Detect improper use of protective equipment such as safety shoes, helmets, and work clothes, exposure to hazardous areas, fall hazards, and failure to follow safety procedures and planned workflows (Fang et al., 2020)
Automate manufacturing(Hatami Mohsen et al., 2019)
Predicting logistics costs during the construction phase using support vector machines (SVM ) (Tian et al., 2009 )
Prediction of building energy consumption(Himeur et al., 2021; Wang & Srinivasan, 2017)
Automatically identifying construction waste using computer vision(Kuritcyn et al., 2015)

A service that installs cameras on construction equipment to avoid collisions with workers (ViAct.ai)

https://www.viact.ai/vimac

A service that detects high-risk accident scenes from cameras at construction sites (mign)

https://prtimes.jp/main/html/rd/p/000000018.000100410.html

Operation and Maintenance Processes

Automatic detection and assessment of defects and damage (cracks, delamination, corrosion, holes, joint damage, etc.) in various types of social infrastructure, such as buildings, bridges, tunnels, roads, sewer pipes , etc. (C. Zhang et al., 2020)
Detect and diagnose building energy system (e.g. HVAC) failures (Y. Zhao et al, 2019)
Predict building age(Tooke et al, 2014)
Electric load forecasting for smart grids (Raza & Khosravi, 2015)

How Generative AI is Used

Based on how AI has been used in the building and civil engineering domain to date, as described above, this section describes how generative AI can be used.

Planning and design process

Interview phase

Summarize and abstract interviews with clients and government into text data.
Automatically output estimates based on interviews with clients.

Literature Review Phase

Pick up important information from requirement documents of competitive bidding projects.
A dialogue model tuned to construction projects allows the AI to ask questions about zoning and other regulations in each area.

Design phase

Automatically outputs design images based on interviews with the client.
Displays design images and generates multiple related images.
Generate 3D models and import them into BIM/CIM.

Automatic architectural perspective generation tool (mign)

https://prtimes.jp/main/html/rd/p/000000013.000100410.html

Construction Process

Procurement phase

New quotations and purchase orders can be created based on information from previous quotations and purchase orders.

Construction phase

Interactive model tuned to construction work, allowing users to ask questions if they have any doubts about how to proceed with construction.

Interactive AI (mign) trained on construction-related regulations

https://prtimes.jp/main/html/rd/p/000000020.000100410.html

Operation and maintenance management process

Monitoring phase

Input images of the structure and air conditioning, and output a failure diagnosis (FDD) report

Maintenance phase

Input images and videos of deteriorated parts of the building or structure and automatically generate a maintenance report.

Summary

We have reviewed the history and types of generative AI, examples of AI applications to date, and discussed how generative AI can be used, essentially replacing the advanced skills and knowledge of humans between input and output.