Daniele Grandi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Daniele Grandi has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- Christopher McComb
Carnegie Mellon University
5 shared publications - Kosa Goucher-Lambert
Department of Mechanical Engineering, University of California, Berkeley, California, USA
4 shared publications - Allin Groom
Autodesk (United Kingdom)
3 shared publications - Brandon Cramer
Autodesk (United States)
3 shared publications - Mohammadmehdi Ataei
Autodesk Research 661 University Avenue M5G 1M1 Toronto ON Canada
3 shared publications - Yash Patawari Jain
Carnegie Mellon University Department of Mechanical Engineering
3 shared publications
Latest Publications
- Open Access
- 15 November 2026
- Published by MDPI AG in Biomimetics
- Vol. 11 (11) , 832
Morphology-Level Structuring of Biological Analogies for Multifunctional Bio-Inspired Design: An Industry-Informed Formative Evaluation
Existing bio-inspired design resources are effective for accessing biological analogies but provide limited support for synthesising multiple biological strategies into multifunctional engineering concepts. The Multifunctional Bio-inspired Design (MBID) framework addresses this gap through a morphology-informed structuring layer that links biological features to functional and geometric design domains. Through an Autodesk Canada-supported formative evaluation involving five industry professionals, this study examines how designers perceive and use MBID in conjunction with Ask Nature across two sequential design phases: concept generation using Ask Nature alone and using MBID in conjunction with Ask Nature. All participants completed the design phases in the same order, limiting the ability to distinguish the effects of MBID from potential order effects. A Likert-scale questionnaire was used to assess participants’ perceptions. The findings suggest that MBID is most valuable not as a standalone repository, but as a structured synthesis layer that supports multifunctional bio-inspired ideation and facilitates the integration of multiple biological features into coherent conceptual solutions. The study also highlights significant usability, onboarding, and downstream engineering translation challenges that influence practical adoption. Overall, the work provides formative evidence that morphology-level structuring may support multifunctional bio-inspired ideation while emphasising the importance of interaction design in the practical use of structured bio-inspired design frameworks.
- 15 November 2026
- Published by MDPI AG in Biomimetics
- Vol. 11 (11) , 832
Open AccessMorphology-Level Structuring of Biological Analogies for Multifunctional Bio-Inspired Design: An Industry-Informed Formative Evaluation
Existing bio-inspired design resources are effective for accessing biological analogies but provide limited support for synthesising multiple biological strategies into multifunctional engineering concepts. The Multifunctional Bio-inspired Design (MBID) framework addresses this gap through a morphology-informed structuring layer that links biological features to functional and geometric design domains. Through an Autodesk Canada-supported formative evaluation involving five industry professionals, this study examines how designers perceive and use MBID in conjunction with Ask Nature across two sequential design phases: concept generation using Ask Nature alone and using MBID in conjunction with Ask Nature. All participants completed the design phases in the same order, limiting the ability to distinguish the effects of MBID from potential order effects. A Likert-scale questionnaire was used to assess participants’ perceptions. The findings suggest that MBID is most valuable not as a standalone repository, but as a structured synthesis layer that supports multifunctional bio-inspired ideation and facilitates the integration of multiple biological features into coherent conceptual solutions. The study also highlights significant usability, onboarding, and downstream engineering translation challenges that influence practical adoption. Overall, the work provides formative evidence that morphology-level structuring may support multifunctional bio-inspired ideation while emphasising the importance of interaction design in the practical use of structured bio-inspired design frameworks.
- Open Access
RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models
Product recalls provide valuable insights into potential risks and hazards within the engineering design process, yet their full potential remains underutilized. In this study, we curate data from the United States Consumer Product Safety Commission (CPSC) recalls database to develop a multimodal dataset, RECALL-MM, that informs data-driven risk assessment using historical information, and augment it using generative methods. Patterns in the dataset highlight specific areas where improved safety measures could have significant impact. We extend our analysis by demonstrating interactive clustering maps that embed all recalls into a shared latent space based on recall descriptions and product names. Leveraging these data-driven tools, we explore three case studies to demonstrate the dataset's utility in identifying product risks and guiding safer design decisions. The first two case studies illustrate how designers can visualize patterns across recalled products and situate new product ideas within the broader recall landscape to proactively anticipate hazards. In the third case study, we extend our approach by employing a large language model (LLM) to predict potential hazards based solely on product images. This demonstrates the model's ability to leverage visual context to identify risk factors, revealing strong alignment with historical recall data across many hazard categories. However, the analysis also highlights areas where hazard prediction remains challenging, underscoring the importance of risk awareness throughout the design process. Collectively, this work aims to bridge the gap between historical recall data and future product safety, presenting a scalable, data-driven approach to safer engineering design.
Open AccessRECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models
Product recalls provide valuable insights into potential risks and hazards within the engineering design process, yet their full potential remains underutilized. In this study, we curate data from the United States Consumer Product Safety Commission (CPSC) recalls database to develop a multimodal dataset, RECALL-MM, that informs data-driven risk assessment using historical information, and augment it using generative methods. Patterns in the dataset highlight specific areas where improved safety measures could have significant impact. We extend our analysis by demonstrating interactive clustering maps that embed all recalls into a shared latent space based on recall descriptions and product names. Leveraging these data-driven tools, we explore three case studies to demonstrate the dataset's utility in identifying product risks and guiding safer design decisions. The first two case studies illustrate how designers can visualize patterns across recalled products and situate new product ideas within the broader recall landscape to proactively anticipate hazards. In the third case study, we extend our approach by employing a large language model (LLM) to predict potential hazards based solely on product images. This demonstrates the model's ability to leverage visual context to identify risk factors, revealing strong alignment with historical recall data across many hazard categories. However, the analysis also highlights areas where hazard prediction remains challenging, underscoring the importance of risk awareness throughout the design process. Collectively, this work aims to bridge the gap between historical recall data and future product safety, presenting a scalable, data-driven approach to safer engineering design.
- Top Cited Papers
- 16 January 2025
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-12
Elicitron: A Large Language Model Agent-Based Simulation Framework for Design Requirements Elicitation
Requirement elicitation, a critical, yet time-consuming and challenging step in product development, often fails to capture the full spectrum of user needs. This may lead to products that fall short of user expectations. This article introduces a novel framework that leverages large language models (LLMs) to automate and enhance the requirement elicitation process. LLMs are used to generate a vast array of simulated users (LLM agents), enabling the exploration of a much broader range of user needs and unforeseen use cases. These agents engage in product experience scenarios, explaining their actions, observations, and challenges. Subsequent agent interviews and analysis uncover valuable user needs, including latent ones. We validate our framework with three experiments. First, we explore different methodologies for the challenge of diverse agent generation, discussing their advantages and shortcomings. We measure the diversity of identified user needs and demonstrate that context-aware agent generation leads to greater diversity. Second, we show how our framework effectively mimics empathic lead user interviews, identifying a greater number of latent needs than conventional human interviews. Third, we show that LLMs can be used to analyze interviews, capture needs, and classify them as latent or not. Our work highlights the potential of using LLMs to accelerate early-stage product development with minimal costs and increase innovation.
- 16 January 2025
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-12
Top Cited PapersElicitron: A Large Language Model Agent-Based Simulation Framework for Design Requirements Elicitation
Requirement elicitation, a critical, yet time-consuming and challenging step in product development, often fails to capture the full spectrum of user needs. This may lead to products that fall short of user expectations. This article introduces a novel framework that leverages large language models (LLMs) to automate and enhance the requirement elicitation process. LLMs are used to generate a vast array of simulated users (LLM agents), enabling the exploration of a much broader range of user needs and unforeseen use cases. These agents engage in product experience scenarios, explaining their actions, observations, and challenges. Subsequent agent interviews and analysis uncover valuable user needs, including latent ones. We validate our framework with three experiments. First, we explore different methodologies for the challenge of diverse agent generation, discussing their advantages and shortcomings. We measure the diversity of identified user needs and demonstrate that context-aware agent generation leads to greater diversity. Second, we show how our framework effectively mimics empathic lead user interviews, identifying a greater number of latent needs than conventional human interviews. Third, we show that LLMs can be used to analyze interviews, capture needs, and classify them as latent or not. Our work highlights the potential of using LLMs to accelerate early-stage product development with minimal costs and increase innovation.
- 15 January 2025
- Published by Wiley in Journal of Mechanical Design
- Vol. 147 (4) , 1-9
MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models
Material selection plays a pivotal role in many industries, from manufacturing to construction. Material selection is usually carried out after several cycles of conceptual design, during which designers iteratively refine the design solution and the intended manufacturing approach. In design research, material selection is frequently modeled as a structured decision-making process, where optimization techniques, whether single-objective or multiobjective, are employed to identify solutions that best meet the design requirements. However, traditional approaches can be limited by their reliance on existing knowledge and data, which may not adequately capture the full range of considerations involved in material selection. In this article, we introduce MSEval, a novel dataset comprised expert material evaluations across a variety of design briefs and criteria. This dataset is designed to serve as a benchmark to facilitate the evaluation and modification of machine learning models in the context of material selection for conceptual design. By focusing on a diverse set of design tasks and criteria, MSEval enables a more nuanced understanding of the material selection and the thought process, providing valuable insights for both human designers and AI systems.
- 15 January 2025
- Published by Wiley in Journal of Mechanical Design
- Vol. 147 (4) , 1-9
MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models
Material selection plays a pivotal role in many industries, from manufacturing to construction. Material selection is usually carried out after several cycles of conceptual design, during which designers iteratively refine the design solution and the intended manufacturing approach. In design research, material selection is frequently modeled as a structured decision-making process, where optimization techniques, whether single-objective or multiobjective, are employed to identify solutions that best meet the design requirements. However, traditional approaches can be limited by their reliance on existing knowledge and data, which may not adequately capture the full range of considerations involved in material selection. In this article, we introduce MSEval, a novel dataset comprised expert material evaluations across a variety of design briefs and criteria. This dataset is designed to serve as a benchmark to facilitate the evaluation and modification of machine learning models in the context of material selection for conceptual design. By focusing on a diverse set of design tasks and criteria, MSEval enables a more nuanced understanding of the material selection and the thought process, providing valuable insights for both human designers and AI systems.
- 23 December 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-9
Do Large Language Models Produce Diverse Design Concepts? A Comparative Study with Human-Crowdsourced Solutions
Access to large amounts of diverse design solutions can support designers during the early stage of the design process. In this article, we explored the efficacy of large language models (LLMs) in producing diverse design solutions, investigating the level of impact that parameter tuning and various prompt engineering techniques can have on the diversity of LLM-generated design solutions. Specifically, we used an LLM (GPT-4) to generate a total of 4000 design solutions across five distinct design topics, eight combinations of parameters, and eight different types of prompt engineering techniques, leading to 50 LLM-generated solutions for each combination of method and design topic. Those LLM-generated design solutions were compared against 100 human-crowdsourced solutions in each design topic using the same set of diversity metrics. Results indicated that, across the five design topics tested, human-generated solutions consistently have greater diversity scores. Using a post hoc logistic regression analysis, we also found that there is a meaningful semantic divide between humans and LLM-generated solutions in some design topics, but not in others. Taken together, these results contribute to the understanding of LLMs’ capabilities and limitations in generating a large volume of diverse design solutions and offer insights for future research that leverages LLMs to generate diverse design solutions for a broad range of design tasks (e.g., inspirational stimuli).
- 23 December 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-9
Do Large Language Models Produce Diverse Design Concepts? A Comparative Study with Human-Crowdsourced Solutions
Access to large amounts of diverse design solutions can support designers during the early stage of the design process. In this article, we explored the efficacy of large language models (LLMs) in producing diverse design solutions, investigating the level of impact that parameter tuning and various prompt engineering techniques can have on the diversity of LLM-generated design solutions. Specifically, we used an LLM (GPT-4) to generate a total of 4000 design solutions across five distinct design topics, eight combinations of parameters, and eight different types of prompt engineering techniques, leading to 50 LLM-generated solutions for each combination of method and design topic. Those LLM-generated design solutions were compared against 100 human-crowdsourced solutions in each design topic using the same set of diversity metrics. Results indicated that, across the five design topics tested, human-generated solutions consistently have greater diversity scores. Using a post hoc logistic regression analysis, we also found that there is a meaningful semantic divide between humans and LLM-generated solutions in some design topics, but not in others. Taken together, these results contribute to the understanding of LLMs’ capabilities and limitations in generating a large volume of diverse design solutions and offer insights for future research that leverages LLMs to generate diverse design solutions for a broad range of design tasks (e.g., inspirational stimuli).
- Top Cited Papers
- 23 December 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-16
DesignQA: A Multimodal Benchmark for Evaluating Large Language Models’ Understanding of Engineering Documentation
This research introduces DesignQA, a novel benchmark aimed at evaluating the proficiency of multimodal large language models (MLLMs) in comprehending and applying engineering requirements in technical documentation. Developed with a focus on real-world engineering challenges, DesignQA uniquely combines multimodal data—including textual design requirements, CAD images, and engineering drawings—derived from the Formula SAE student competition. Unlike many existing MLLM benchmarks, DesignQA contains document-grounded visual questions where the input image and the input document come from different sources. The benchmark features automatic evaluation metrics and is divided into segments—Rule Comprehension, Rule Compliance, and Rule Extraction—based on tasks that engineers perform when designing according to requirements. We evaluate state-of-the-art models (at the time of writing) like GPT-4o, GPT-4, Claude-Opus, Gemini-1.0, and LLaVA-1.5 against the benchmark, and our study uncovers the existing gaps in MLLMs’ abilities to interpret complex engineering documentation. The MLLMs tested, while promising, struggle to reliably retrieve relevant rules from the Formula SAE documentation, face challenges in recognizing technical components in CAD images and encounter difficulty in analyzing engineering drawings. These findings underscore the need for multimodal models that can better handle the multifaceted questions characteristic of design according to technical documentation. This benchmark sets a foundation for future advancements in AI-supported engineering design processes. DesignQA is publicly available at online.
- 23 December 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-16
Top Cited PapersDesignQA: A Multimodal Benchmark for Evaluating Large Language Models’ Understanding of Engineering Documentation
This research introduces DesignQA, a novel benchmark aimed at evaluating the proficiency of multimodal large language models (MLLMs) in comprehending and applying engineering requirements in technical documentation. Developed with a focus on real-world engineering challenges, DesignQA uniquely combines multimodal data—including textual design requirements, CAD images, and engineering drawings—derived from the Formula SAE student competition. Unlike many existing MLLM benchmarks, DesignQA contains document-grounded visual questions where the input image and the input document come from different sources. The benchmark features automatic evaluation metrics and is divided into segments—Rule Comprehension, Rule Compliance, and Rule Extraction—based on tasks that engineers perform when designing according to requirements. We evaluate state-of-the-art models (at the time of writing) like GPT-4o, GPT-4, Claude-Opus, Gemini-1.0, and LLaVA-1.5 against the benchmark, and our study uncovers the existing gaps in MLLMs’ abilities to interpret complex engineering documentation. The MLLMs tested, while promising, struggle to reliably retrieve relevant rules from the Formula SAE documentation, face challenges in recognizing technical components in CAD images and encounter difficulty in analyzing engineering drawings. These findings underscore the need for multimodal models that can better handle the multifaceted questions characteristic of design according to technical documentation. This benchmark sets a foundation for future advancements in AI-supported engineering design processes. DesignQA is publicly available at online.
- 14 November 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-33
Evaluating Large Language Models for Material Selection
Material selection is a crucial step in conceptual design due to its significant impact on the functionality, aesthetics, manufacturability, and sustainability impact of the final product. This study investigates the use of large language models (LLMs) for material selection in the product design process and compares the performance of LLMs against expert choices for various design scenarios. By collecting a dataset of expert material preferences, the study provides a basis for evaluating how well LLMs can align with expert recommendations through prompt engineering and hyperparameter tuning. The divergence between LLM and expert recommendations is measured across different model configurations, prompt strategies, and temperature settings. This approach allows for a detailed analysis of factors influencing the LLMs' effectiveness in recommending materials. The results from this study highlight two failure modes: the low variance of recommendations across different design scenarios and the tendency toward overestimating material appropriateness. Parallel prompting is identified as a useful prompt-engineering method when using LLMs for material selection. The findings further suggest that, while LLMs can provide valuable assistance, their recommendations often vary significantly from those of human experts. This discrepancy underscores the need for further research into how LLMs can be better tailored to replicate expert decision-making in material selection. This work contributes to the growing body of knowledge on how LLMs can be integrated into the design process, offering insights into their current limitations and potential for future improvements.
- 14 November 2024
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 25 (2) , 1-33
Evaluating Large Language Models for Material Selection
Material selection is a crucial step in conceptual design due to its significant impact on the functionality, aesthetics, manufacturability, and sustainability impact of the final product. This study investigates the use of large language models (LLMs) for material selection in the product design process and compares the performance of LLMs against expert choices for various design scenarios. By collecting a dataset of expert material preferences, the study provides a basis for evaluating how well LLMs can align with expert recommendations through prompt engineering and hyperparameter tuning. The divergence between LLM and expert recommendations is measured across different model configurations, prompt strategies, and temperature settings. This approach allows for a detailed analysis of factors influencing the LLMs' effectiveness in recommending materials. The results from this study highlight two failure modes: the low variance of recommendations across different design scenarios and the tendency toward overestimating material appropriateness. Parallel prompting is identified as a useful prompt-engineering method when using LLMs for material selection. The findings further suggest that, while LLMs can provide valuable assistance, their recommendations often vary significantly from those of human experts. This discrepancy underscores the need for further research into how LLMs can be better tailored to replicate expert decision-making in material selection. This work contributes to the growing body of knowledge on how LLMs can be integrated into the design process, offering insights into their current limitations and potential for future improvements.
- Open Access
MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models
Material selection plays a pivotal role in many industries, from manufacturing to construction. Material selection is usually carried out after several cycles of conceptual design, during which designers iteratively refine the design solution and the intended manufacturing approach. In design research, material selection is typically treated as an optimization problem with a single correct answer. Moreover, it is also often restricted to specific types of objects or design functions, which can make the selection process computationally expensive and time-consuming. In this paper, we introduce MSEval, a novel dataset which is comprised of expert material evaluations across a variety of design briefs and criteria. This data is designed to serve as a benchmark to facilitate the evaluation and modification of machine learning models in the context of material selection for conceptual design.
Open AccessMSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models
Material selection plays a pivotal role in many industries, from manufacturing to construction. Material selection is usually carried out after several cycles of conceptual design, during which designers iteratively refine the design solution and the intended manufacturing approach. In design research, material selection is typically treated as an optimization problem with a single correct answer. Moreover, it is also often restricted to specific types of objects or design functions, which can make the selection process computationally expensive and time-consuming. In this paper, we introduce MSEval, a novel dataset which is comprised of expert material evaluations across a variety of design briefs and criteria. This data is designed to serve as a benchmark to facilitate the evaluation and modification of machine learning models in the context of material selection for conceptual design.
- Open Access
- 4 October 2023
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 24 (1) , 1-28
HG-CAD: Hierarchical Graph Learning for Material Prediction and Recommendation in Computer-Aided Design
To support intelligent computer-aided design (CAD), we introduce a machine learning architecture, namely HG-CAD, that recommends assembly body material through joint learning of body- and assembly-level features using a hierarchical graph representation. Specifically, we formulate the material prediction and recommendation process as a node-level classification task over a novel hierarchical graph representation of CAD models, with a low-level graph capturing the body geometry, a high-level graph representing the assembly topology, and a batch-level mask randomization enabling contextual awareness. This enables our network to aggregate geometric and topological features from both the body and assembly levels, leading to competitive performance. Qualitative and quantitative evaluation of the proposed architecture on the Fusion 360 Gallery Assembly Dataset demonstrates the feasibility of our approach, outperforming selected computer vision and human baselines while showing promise in application scenarios. The proposed HG-CAD architecture that unifies the processing, encoding, and joint learning of multi-modal CAD features indicates the potential to serve as a recommendation system for design automation and a baseline for future work.
- 4 October 2023
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 24 (1) , 1-28
Open AccessHG-CAD: Hierarchical Graph Learning for Material Prediction and Recommendation in Computer-Aided Design
To support intelligent computer-aided design (CAD), we introduce a machine learning architecture, namely HG-CAD, that recommends assembly body material through joint learning of body- and assembly-level features using a hierarchical graph representation. Specifically, we formulate the material prediction and recommendation process as a node-level classification task over a novel hierarchical graph representation of CAD models, with a low-level graph capturing the body geometry, a high-level graph representing the assembly topology, and a batch-level mask randomization enabling contextual awareness. This enables our network to aggregate geometric and topological features from both the body and assembly levels, leading to competitive performance. Qualitative and quantitative evaluation of the proposed architecture on the Fusion 360 Gallery Assembly Dataset demonstrates the feasibility of our approach, outperforming selected computer vision and human baselines while showing promise in application scenarios. The proposed HG-CAD architecture that unifies the processing, encoding, and joint learning of multi-modal CAD features indicates the potential to serve as a recommendation system for design automation and a baseline for future work.
- 23 June 2023
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 24 (1) , 1-17
What’s in a Name? Evaluating Assembly-Part Semantic Knowledge in Language Models Through User-Provided Names in Computer Aided Design Files
Semantic knowledge of part-part and part-whole relationships in assemblies is useful for a variety of tasks from searching design repositories to the construction of engineering knowledge bases. In this work, we propose that the natural language names designers use in computer aided design (CAD) software are a valuable source of such knowledge, and that large language models (LLMs) contain useful domain-specific information for working with this data as well as other CAD and engineering-related tasks. In particular, we extract and clean a large corpus of natural language part, feature, and document names and use this to quantitatively demonstrate that a pre-trained language model can outperform numerous benchmarks on three self-supervised tasks, without ever having seen this data before. Moreover, we show that fine-tuning on the text data corpus further boosts the performance on all tasks, thus demonstrating the value of the text data which until now has been largely ignored. We also identify key limitations to using LLMs with text data alone, and our findings provide a strong motivation for further work into multi-modal text-geometry models. To aid and encourage further work in this area we make all our data and code publicly available.
- 23 June 2023
- Published by Wiley in Journal of Computing and Information Science in Engineering
- Vol. 24 (1) , 1-17
What’s in a Name? Evaluating Assembly-Part Semantic Knowledge in Language Models Through User-Provided Names in Computer Aided Design Files
Semantic knowledge of part-part and part-whole relationships in assemblies is useful for a variety of tasks from searching design repositories to the construction of engineering knowledge bases. In this work, we propose that the natural language names designers use in computer aided design (CAD) software are a valuable source of such knowledge, and that large language models (LLMs) contain useful domain-specific information for working with this data as well as other CAD and engineering-related tasks. In particular, we extract and clean a large corpus of natural language part, feature, and document names and use this to quantitatively demonstrate that a pre-trained language model can outperform numerous benchmarks on three self-supervised tasks, without ever having seen this data before. Moreover, we show that fine-tuning on the text data corpus further boosts the performance on all tasks, thus demonstrating the value of the text data which until now has been largely ignored. We also identify key limitations to using LLMs with text data alone, and our findings provide a strong motivation for further work into multi-modal text-geometry models. To aid and encourage further work in this area we make all our data and code publicly available.