Generative AI in Architecture and Construction: Practical Use Cases for ChatGPT and Image Generation

Generative AI—conversational systems like ChatGPT and image generators like Midjourney—has become a major talking point, and it is widely expected to transform society and entire industries. In this article we focus on the architecture and civil engineering fields, including real estate, construction, and urban development. After reviewing the history of generative AI and looking at the latest research and real-world services, we outline examples of how generative AI could be put to use going forward.

What Is Generative AI?

Defining Generative AI

Because generative AI is still in its infancy, definitions vary slightly from one source to another. Below are some examples of how leading companies and research institutions define it.

Generative AI refers to a category of AI that uses systems called neural networks to analyze data, identify patterns, and use those patterns to generate or create new outputs such as text, photos, video, code, and data.

Generative AI refers to deep learning models that can generate high-quality content, such as text and images, based on the data they were trained on.

Generative AI refers to algorithms (such as ChatGPT) that can be used to create new content, including audio, code, images, text, simulations, and video.

To sum up, the essential elements of generative AI are that it

• works from training data,

• leverages deep learning, and

• generates new content.

These three points capture the core of what defines generative AI.

A Brief History of Generative AI

Generative AI has evolved along two main lines: natural language processing and computer vision.

• In natural language processing, the story begins with Hidden Markov Models (HMMs) (Knill & Young, 1997) and Gaussian Mixture Models (GMMs) (Reynolds & Others, 2009), which were used to generate sequential data such as speech and time series. With the advent of deep learning, the performance of generative models improved dramatically. Recurrent neural networks (RNNs) were introduced to language modeling tasks, making it possible to model relatively long-range dependencies. After the development of the Transformer architecture (Vaswani et al., 2017), this model became the principal backbone of GPT and many subsequent generative models—a position it holds to this day.

• In computer vision, the field began with Texture synthesis (Efros & Leung, 1999) and Texture mapping (Heckbert, 1986). These algorithms could generate complex and varied images based on hand-crafted features, but their capacity for large-scale generation was limited. From 2014 onward, new approaches emerged, including Generative Adversarial Networks (GANs) (Goodfellow et al., 2020), Variational Autoencoders (VAEs), and diffusion models (Song & Ermon, 2019), which allowed finer control over the image generation process and the production of high-quality images. Combining the Transformer architecture (Vaswani et al., 2017) with computer vision then pushed the concept further, enabling image-based outputs—and, in turn, making multimodal tasks possible.

Looking ahead, we can expect continued gains in generation accuracy, along with a shift toward multimodality, with an ever-wider variety of input and output types and combinations.

Types of Generative AI

The figure below classifies the major generative AI models that have emerged in recent years by their input and output formats (Gozalo-Brizuela & Garrido-Merchan, 2023).
Since most are built on the Transformer architecture (Vaswani et al., 2017), which was designed for natural language processing, many of them take text as input. Output formats, meanwhile, extend well beyond text and images to include 3D, video, audio, program code, and more. Some models, such as Flaming and VisualGPT, take images as input.

Applications of Generative AI in Architecture and Civil Engineering

This section looks at how generative AI could be applied in architecture and civil engineering. We first give an overview of how AI has been used in these fields to date, then explore the potential of generative AI. Throughout, we divide the field into three processes: planning and design, construction, and operation and maintenance.

How AI Has Been Used So Far

For more detail, please see this article.

Planning and Design Process

• Design generation and emulation (Huang & Zheng, 2018; Liu et al, 2017)

• Design evaluation (Lorenz et al., 2018; Y. Zhang et al., 2018)

• In structural design: predicting structural response and performance, interpreting experimental data, and pattern recognition in structural health monitoring data, among others (Sun et al, 2021)

• Creating BIM models from laser scanner data (Tang et al., 2010)

• Analyzing BIM logs to identify ways to improve design productivity (Pan & Zhang, 2020)

Construction Process

• Generating schedules that minimize project financing costs (Alavipour & Arditi, 2019)

• Detecting improper use of protective equipment such as safety boots, helmets, and work clothing, exposure to hazardous zones, fall risks, and failure to follow safety procedures or planned workflows (Fang et al., 2020)

• Manufacturing automation (Hatami Mohsen et al., 2019)

• Predicting logistics costs during the construction phase using support vector machines (SVMs) (Tian et al., 2009)

• Predicting building energy consumption (Himeur et al., 2021; Wang & Srinivasan, 2017)

• Automatically identifying construction waste using computer vision (Kuritcyn et al., 2015)

Operation and Maintenance Process

• Automatically detecting and assessing defects and damage—cracks, delamination, corrosion, holes, joint damage, and so on—across many types of infrastructure, including buildings, bridges, tunnels, roads, and sewer pipes (C. Zhang et al., 2020)

• Detecting and diagnosing faults in building energy systems such as HVAC (Y. Zhao et al, 2019)

• Estimating the age of buildings (Tooke et al., 2014)

• Electrical load forecasting for smart grids (Raza & Khosravi, 2015)

How Generative AI Can Be Used

Building on the overview above of how AI has been applied in architecture and civil engineering to date, we now turn to how generative AI can be put to use.

Planning and Design Process

• Interview stage

  • Summarize and distill interviews with clients and government agencies into text data.

  • Automatically generate cost estimates based on client interviews.

• Research stage

  • Extract key information from the requirement documents of competitive bidding projects.

  • Use a conversational model tuned for construction work to ask the AI about local regulations such as zoning.

• Design stage

  • Automatically generate design concept images based on client interviews.

  • Show a design image and generate multiple related images.

  • Generate 3D models and import them into BIM/CIM.

Construction Process

• Procurement stage

  • Create new estimates and purchase orders informed by the data in past estimates and purchase orders.

• Construction stage

  • Use a conversational model tuned for construction work to ask questions about how to carry out the work whenever they arise.

Operation and Maintenance Process

• Monitoring stage

  • Input images of structural elements or HVAC equipment and output a fault detection and diagnosis (FDD) report.

• Maintenance stage

  • Input images or video of deteriorated areas of buildings and structures and automatically produce a maintenance report.

Conclusion

We have surveyed the history and types of generative AI and the ways AI has been applied to date, and explored how generative AI could be used. At its core, generative AI can be seen as replacing the advanced human skill and knowledge that sits between input and output. By improving the architecture and civil engineering fields, it holds great promise as a technology that can contribute significantly to enriching people's lives.

If you are interested, please also see our article on how to use extensions such as LangChain, which enables integration with external data, and AutoGPT, which enables task automation.

Finally, at mign we continually harness cutting-edge technology to create world-first solutions for architecture and civil engineering, with the goal of building a more prosperous society. We are currently focusing our development efforts on generative AI (ChatGPT and image generation) and are able to develop solutions related to the topics discussed above. If you are interested, please feel free to get in touch.

Contact form (website)
https://www.mign.io/contacts/new

Email
contact@mign.io

Press releases on ChatGPT-related solutions specialized for architecture and civil engineering
https://prtimes.jp/main/html/rd/p/000000020.000100410.html
https://prtimes.jp/main/html/rd/p/000000023.000100410.html

Press releases on our image-generation AI solution specialized for architecture and civil engineering
https://prtimes.jp/main/html/rd/p/000000013.000100410.html
https://prtimes.jp/main/html/rd/p/000000022.000100410.html

Other press releases
https://prtimes.jp/main/html/searchrlp/company_id/100410

References

• Alavipour, S. M. R., & Arditi, D. (2019). Time-cost tradeoff analysis with minimized project financing cost. Automation in Construction, 98, 110–121.

• Efros, A. A., & Leung, T. K. (1999). Texture synthesis by non-parametric sampling. Proceedings of the Seventh IEEE International Conference on Computer Vision, 2, 1033–1038 vol.2.

• Fang, W., Love, P. E. D., Luo, H., & Ding, L. (2020). Computer vision for behaviour-based safety in construction: A review and future directions. Advanced Engineering Informatics, 43, 100980.

• Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139–144.

• Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023). ChatGPT is not all you need. A State of the Art Review of large Generative AI models. In arXiv [cs.LG]. arXiv. http://arxiv.org/abs/2301.04655

• Hatami Mohsen, Flood Ian, Franz Bryan, & Zhang Xun. (2019). State-of-the-Art Review on the Applicability of AI Methods to Automated Construction Manufacturing. Journal of Computing in Civil Engineering, 368–375.

• Heckbert, P. S. (1986). Survey of Texture Mapping. IEEE Computer Graphics and Applications, 6(11), 56–67.

• Himeur, Y., Ghanem, K., Alsalemi, A., Bensaali, F., & Amira, A. (2021). Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives. Applied Energy, 287, 116601.

• Knill, K., & Young, S. (1997). Hidden Markov Models in Speech and Language Processing. In S. Young & G. Bloothooft (Eds.), Corpus-Based Methods in Language and Speech Processing (pp. 27–68). Springer Netherlands.

• Kuritcyn, P., Anding, K., Linß, E., & Latyev, S. M. (2015). Increasing the safety in recycling of construction and demolition waste by using supervised machine learning. Journal of Physics. Conference Series, 588, 012035.

• Lorenz, C.-L., Packianather, M., Spaeth, A., & Bleil De Souza, C. (2018). Artificial neural network-based modelling for daylight evaluations. SimAUD 2018, Delft, The Netherlands. https://orca.cardiff.ac.uk/id/eprint/112040/

• Martineau, K. (2023, April 20). What is generative AI? IBM Research Blog; IBM. https://research.ibm.com/blog/what-is-generative-AI

• McKinsey & Company. (2023, January 19). What is generative AI? McKinsey & Company. https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-generative-ai

• Microsoft News Center. (2023, April 4). AI explained. Stories. https://news.microsoft.com/2023/04/04/ai-explained/

• Pan, Y., & Zhang, L. (2020). BIM log mining: Exploring design productivity characteristics. Automation in Construction, 109, 102997.

• Reynolds, D. A., & Others. (2009). Gaussian mixture models. Encyclopedia of Biometrics, 741(659-663). http://leap.ee.iisc.ac.in/sriram/teaching/MLSP_16/refs/GMM_Tutorial_Reynolds.pdf

• Song, Y., & Ermon, S. (2019). Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32. https://proceedings.neurips.cc/paper/2019/hash/3001ef257407d5a371a96dcd947c7d93-Abstract.html

• Tang, P., Huber, D., Akinci, B., Lipman, R., & Lytle, A. (2010). Automatic reconstruction of as-built building information models from laser-scanned point clouds: A review of related techniques. Automation in Construction, 19(7), 829–843.

• Tian, J., Gao, M., & Zhou, S. (2009). The Research of Building Logistics Cost Forecast Based on Regression Support Vector Machine. 2009 International Conference on Computational Intelligence and Security, 1, 648–652.

• Tooke, T. R., Coops, N. C., & Webster, J. (2014). Predicting building ages from LiDAR data with random forests for building energy modeling. Energy and Buildings, 68, 603–610.

• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper/7181-attention-is-all

• Wang, Z., & Srinivasan, R. S. (2017). A review of artificial intelligence based building energy use prediction: Contrasting the capabilities of single and ensemble prediction models. Renewable and Sustainable Energy Reviews, 75, 796–808.

• Zhang, C., Chang, C.-C., & Jamshidi, M. (2020). Concrete bridge surface damage detection using a single‐stage detector. Computer-Aided Civil and Infrastructure Engineering, 35(4), 389–409.

• Zhang, Y., Burton, H. V., Sun, H., & Shokrabadi, M. (2018). A machine learning framework for assessing post-earthquake structural safety. Structural Safety, 72, 1–16.