TRIZ-Based Framework for Evaluating Creativity of AI-Generated Solutions to Ill-Defined Engineering Problems

The assessment of creativity and true innovativeness in solutions to ill-defined engineering problems remains a complex challenge. This applies to both human designers, typically evaluated through subjective or psychometric methods, and modern AI systems, whose performance metrics often overlook the essence of inventive thinking. Current evaluation approaches, including AI benchmarks, effectively measure linguistic fluency, analytical ability, and knowledge retrieval, but fail to capture divergent thinking, out-of-the-box reasoning, and the technical quality of proposed solutions. Consequently, objective and reproducible metrics for genuine inventiveness are still lacking. This study introduces a novel TRIZ-based methodological framework to address this gap, operationalizing structured metrics to quantify inventive character, technical soundness, and deviation from conventional approaches. Designed for universal applicability, the framework evaluates solutions generated by both LLMs and human problem solvers across two core dimensions: conceptual reframing and technical exploration. It measures the ability to overcome psychological inertia, redefine system boundaries, and generate original solutions, including the use of inventive principles and cross-domain analogies. To ground the evaluation in real-world innovation, the framework incorporates a patent-based validation layer that assesses technical novelty and feasibility by cross-referencing solutions with global patent databases and Non-Patent Literature, linking the metrics to established indicators of technological innovation. Experiments on multiple LLMs show that creativity and innovativeness are distinct dimensions. Structurally creative outputs do not necessarily correspond to high technical novelty under patent validation. The proposed metrics offer a structured, interpretable, and reproducible tool for evaluating inventive reasoning and comparing AI-driven innovation.