{"id":47286,"date":"2025-04-22T16:30:46","date_gmt":"2025-04-22T07:30:46","guid":{"rendered":"https:\/\/automaton-media.com\/en\/?p=47286"},"modified":"2025-04-22T16:30:48","modified_gmt":"2025-04-22T07:30:48","slug":"ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans","status":"publish","type":"post","link":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/","title":{"rendered":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0"},"content":{"rendered":"\n

A machine learning research organization called Hao AI Lab recently published results of an unusual test meant to evaluate the reasoning skills of popular AI models. What\u2019s peculiar about it is the benchmark they chose for the test \u2013 Capcom’s Ace Attorney. The researchers had AI models like OpenAI-o1 and Gemini 2.5 Pro play Phoenix Wright: Ace Attorney<\/strong>, the first game in the franchise, to evaluate their ability to spot contradictions and present correct evidence in trials. <\/p>\n\n\n\n

According to Hao AI Lab<\/a>, the reason why Ace Attorney acts as such a good benchmark for evaluating AI\u2019s capabilities is because it tests not only memorization, but multiple complex skills \u2013 long-context reasoning (spotting contradictions by cross-referencing with prior dialogue and evidence), visual understanding (identifying the exact image that disproves false testimonies) and strategic decision-making (deciding when to press a witness, present evidence, or hold back). <\/p>\n\n\n\n

\"Ace<\/figure>\n\n\n\n

Hao AI Lab evaluated four AI models in this way \u2013 OpenAI’s o1, Google’s Gemini 2.5 Pro, Anthropic’s Claude 3.7 Sonnet (extended thinking mode), and Meta’s Llama-4 Maverick. To start from the conclusion, none of the AI models managed to win all five trials in the game, with Llama-4 Maverick bombing the first episode and Claude 3.7 Sonnet getting a game over in the middle of the second episode. On the other hand, OpenAI and Google’s AI models managed to make it to episode four, but neither won the trial. You can watch a timelapse of their struggle<\/a> in this side-by-side comparison.\u00a0<\/p>\n\n\n\n

\"AI<\/figure>\n\n\n\n

Interestingly, an actual Ace Attorney developer reacted to the whole thing. Masakazu Sugimori, who created the OSTs for the first two mainline Ace Attorney games and voiced Manfred von Karma, expressed surprise<\/a> at the game\u2019s unexpected application on X. <\/p>\n\n\n\n

\u201cHow should I put this, I never thought the game I worked on so desperately 25 years ago would come to be used in this way, and overseas at that (laughs).<\/em>\u00a0

That said, I find it interesting how the AI models get stumped in the first episode. Takumi and Mikami were very particular about the difficulty level of Episode 1 \u2013 it’s supposed to be simple for a human. Maybe this kind of deductive power is the strength of humans?\u201d<\/em>\u00a0<\/p>\n\n\n\n

In a follow-up post<\/a>, Sugimori expands on his comment about Ace Attorney director Shu Takumi and executive producer Shinji Mikami. \u201cThe reason why Takumi and Mikami were so particular about balancing the difficulty level of Ace Attorney\u2019s first episode was because \u2018there was no other game like it in the world at the time.\u2019 It had to be a difficulty that would be acceptable to a wide playerbase, but it had to avoid being insultingly simple too. They were going for the kind of difficulty that gives you a sense of satisfaction when the solution hits you.\u201d  Much like its creators intended, Ace Attorney\u2019s first episode is the definition of beginner-friendly, so AI clearly still has a long way to go. <\/p>\n\n\n\n

\"Ace<\/figure>\n\n\n\n

While he seemed overall positive about video games playing a role in the evolution of technology, Sugimori also commented<\/a> \u201cat the same time, I think we humans won\u2019t lose to AI. While we may lose when it comes to performing individual tasks, the role of humans is to do the thinking, or rather, the directing.\u201d <\/p>\n","protected":false},"excerpt":{"rendered":"

Ace Attorney composer and voice actor Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI’s thinking skills.<\/p>\n","protected":false},"author":55,"featured_media":47288,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_gspb_post_css":"","footnotes":""},"categories":[3],"tags":[67,17],"class_list":["post-47286","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-ai","tag-japan-related-news"],"blocksy_meta":[],"yoast_head":"\nAce Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0 - AUTOMATON WEST<\/title>\n<meta name=\"description\" content=\"Ace Attorney dev Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI's thinking skills.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0 - AUTOMATON WEST\" \/>\n<meta property=\"og:description\" content=\"Ace Attorney dev Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI's thinking skills.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/\" \/>\n<meta property=\"og:site_name\" content=\"AUTOMATON WEST\" \/>\n<meta property=\"article:published_time\" content=\"2025-04-22T07:30:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-22T07:30:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1090\" \/>\n\t<meta property=\"og:image:height\" content=\"612\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Amber V\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AUTOMATON_ENG\" \/>\n<meta name=\"twitter:site\" content=\"@AUTOMATON_ENG\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Amber V\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0 - AUTOMATON WEST","description":"Ace Attorney dev Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI's thinking skills.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/","og_locale":"en_US","og_type":"article","og_title":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0 - AUTOMATON WEST","og_description":"Ace Attorney dev Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI's thinking skills.","og_url":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/","og_site_name":"AUTOMATON WEST","article_published_time":"2025-04-22T07:30:46+00:00","article_modified_time":"2025-04-22T07:30:48+00:00","og_image":[{"width":1090,"height":612,"url":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg","type":"image\/jpeg"}],"author":"Amber V","twitter_card":"summary_large_image","twitter_creator":"@AUTOMATON_ENG","twitter_site":"@AUTOMATON_ENG","twitter_misc":{"Written by":"Amber V","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#article","isPartOf":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/"},"author":{"name":"Amber V","@id":"https:\/\/automaton-media.com\/en\/#\/schema\/person\/fdc6517a00e35b6917ffdb02e8b574d9"},"headline":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0","datePublished":"2025-04-22T07:30:46+00:00","dateModified":"2025-04-22T07:30:48+00:00","mainEntityOfPage":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/"},"wordCount":579,"commentCount":0,"publisher":{"@id":"https:\/\/automaton-media.com\/en\/#organization"},"image":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#primaryimage"},"thumbnailUrl":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg","keywords":["AI","News (Japan-related)"],"articleSection":["News"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/","url":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/","name":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0 - AUTOMATON WEST","isPartOf":{"@id":"https:\/\/automaton-media.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#primaryimage"},"image":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#primaryimage"},"thumbnailUrl":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg","datePublished":"2025-04-22T07:30:46+00:00","dateModified":"2025-04-22T07:30:48+00:00","description":"Ace Attorney dev Masakazu Sugimori was surprised to see the first game in the series used as a benchmark for testing AI's thinking skills.","breadcrumb":{"@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#primaryimage","url":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg","contentUrl":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2025\/04\/20250422-47286-header.jpg","width":1090,"height":612,"caption":"Miles Edgeworth in Ace Attorney"},{"@type":"BreadcrumbList","@id":"https:\/\/automaton-media.com\/en\/news\/ace-attorney-dev-reacts-to-the-game-being-used-to-test-how-smart-ai-models-are-maybe-this-kind-of-deductive-power-is-the-strength-of-humans\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u30db\u30fc\u30e0","item":"https:\/\/automaton-media.com\/en\/"},{"@type":"ListItem","position":2,"name":"Ace Attorney dev reacts to the game being used to test how smart AI models are \u2013 \u201cmaybe this kind of deductive power is the strength of humans\u201d\u00a0"}]},{"@type":"WebSite","@id":"https:\/\/automaton-media.com\/en\/#website","url":"https:\/\/automaton-media.com\/en\/","name":"AUTOMATON WEST","description":"AUTOMATON is a website that covers the Japanese gaming world. We bring you the news on video games from Osaka and Tokyo.","publisher":{"@id":"https:\/\/automaton-media.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/automaton-media.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/automaton-media.com\/en\/#organization","name":"\u682a\u5f0f\u4f1a\u793e\u30a2\u30af\u30c6\u30a3\u30d6\u30b2\u30fc\u30df\u30f3\u30b0\u30e1\u30c7\u30a3\u30a2","url":"https:\/\/automaton-media.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/automaton-media.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2021\/04\/activegamingmedia_logo.png","contentUrl":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2021\/04\/activegamingmedia_logo.png","width":374,"height":190,"caption":"\u682a\u5f0f\u4f1a\u793e\u30a2\u30af\u30c6\u30a3\u30d6\u30b2\u30fc\u30df\u30f3\u30b0\u30e1\u30c7\u30a3\u30a2"},"image":{"@id":"https:\/\/automaton-media.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/AUTOMATON_ENG","https:\/\/www.youtube.com\/channel\/UCabvYnvuUUbbGUrxkaFRgSA"]},{"@type":"Person","@id":"https:\/\/automaton-media.com\/en\/#\/schema\/person\/fdc6517a00e35b6917ffdb02e8b574d9","name":"Amber V","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2024\/06\/12252024-100x100.jpg","url":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2024\/06\/12252024-100x100.jpg","contentUrl":"https:\/\/automaton-media.com\/en\/wp-content\/uploads\/2024\/06\/12252024-100x100.jpg","caption":"Amber V"},"description":"Editor-in-Chief since October 2023. She grew up playing Duke Nukem and Wolfenstein with her dad, and is now enamored with obscure Japanese video games and internet culture. Currently devoted to growing Automaton West to the size of its Japanese sister-site, while making sure to keep news concise and developer stories deep and stimulating.","url":"https:\/\/automaton-media.com\/en\/author\/amber-vjestica\/"}]}},"_links":{"self":[{"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/posts\/47286","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/users\/55"}],"replies":[{"embeddable":true,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/comments?post=47286"}],"version-history":[{"count":3,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/posts\/47286\/revisions"}],"predecessor-version":[{"id":47293,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/posts\/47286\/revisions\/47293"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/media\/47288"}],"wp:attachment":[{"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/media?parent=47286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/categories?post=47286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/automaton-media.com\/en\/wp-json\/wp\/v2\/tags?post=47286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}