A Comparative Study of ChatGPT and DeepSeek on Syntactic Ambiguity

Abstract

Linguistic ambiguity poses significant challenges to natural language processing (NLP) systems, particularly in context-dependent tasks such as machine translation and dialogue generation. While large language models (LLMs) like ChatGPT and DeepSeek exhibit advanced language capabilities, their efficacy in processing ambiguous structures remains understudied. This study constructs a systematically annotated corpus comprising 100 English sentences and two supplementary test sets, focusing primarily on three types of syntactic ambiguity: prepositional phrase (PP) attachment, attributive modification, and coordination scope. Adopting a dual-phase experimental design, we evaluate and compare the capabilities of ChatGPT-5 Mini and DeepSeek-R1 across two dimensions: (1) ambiguity detection accuracy tested through 100 separate dialogues to ensure controlled experimental conditions, and (2) disambiguation performance assessed by comparing model outputs across different contextual conditions. The results indicate that: (1) both models demonstrate strong competence in ambiguity detection, with DeepSeek-R1 achieving an accuracy of 85%, marginally outperforming ChatGPT-5 Mini (77%); (2) in context-dependent disambiguation tasks, neither model effectively utilizes contextual information to resolve PP attachment ambiguities; (3) similarly, both models fail to leverage contextual cues to disambiguate attributive modification structures. These findings suggest that LLM performance may rely more on surface-level statistical patterns than on deep syntactic reasoning, particularly struggling with non-local dependency resolution and probabilistic attachment interpretation. Furthermore, this study contributes to LLM interpretability research and provides implications for developing ambiguity-aware NLP applications.