大規模言語モデルにおける不完全プロンプトによるジェイルブレイクIncomplete Prompt Jailbreaks in Large Language Models

A newly circulated paper on arXiv examines a jailbreak technique that targets one of the most basic properties of large language models: their tendency to finish whatever text they are given. According to the reported work, deliberately incomplete prompts can steer a model into producing content that its safety systems are meant to refuse, suggesting an attack surface that current alignment approaches may not adequately cover. The finding matters because it does not rely on elaborate wording tricks but on the underlying next-token prediction behavior that defines how these systems operate.

The core idea appears to hinge on the difference between answering a request and completing a passage. Contemporary chat models are trained to recognize a harmful instruction and respond with a refusal. When the same intent is embedded in an unfinished sentence or a partially written document, the model is nudged to continue the text rather than evaluate it as a standalone question. Because autoregressive models are optimized to predict the most plausible continuation, the researchers argue that the completion drive can override or bypass the refusal behavior that safety tuning tries to instill. In effect, the harmful content is generated as a natural extension of the input rather than as a direct reply.

This work sits within a growing body of jailbreak and red-teaming research. Earlier techniques include role-play framings such as the well-known "DAN" prompts, prefix-injection attacks that force a model to begin its answer with an affirmative phrase, and gradient-based adversarial suffixes like the GCG method, which appends seemingly random token strings that reliably trigger unsafe outputs. More recently, many-shot jailbreaking demonstrated that stuffing a long context with fabricated question-and-answer pairs can erode a model's guardrails. The incomplete-prompt approach is notable because it does not require optimization, unusual characters, or a large number of examples; it instead exploits the model's default generative disposition.

A useful distinction here is between jailbreaking and prompt injection, terms that are often conflated. Jailbreaking generally refers to bypassing a model's built-in safety policies, while prompt injection typically involves manipulating an application by inserting instructions into data the model processes, such as a web page or document. The technique described in the paper is primarily a jailbreak, though the two categories overlap in practice, especially in agentic systems where models read and act on untrusted external text.

The reason existing defenses may struggle is tied to how those defenses are built. Safety alignment methods, including reinforcement learning from human feedback and constitutional or rule-based training, are largely shaped around request-shaped inputs where the harmful intent is explicit. Input and output classifiers, another common layer, are typically trained to flag recognizable harmful queries or completed harmful text. A prompt that is grammatically incomplete and lacks an obvious instruction can fall near the boundary of these training distributions, making it harder for filters to catch and for the model itself to identify as adversarial. The researchers suggest this is why the method is difficult to address with current safeguards, though the extent of the vulnerability across different model families and providers is likely to vary.

For the broader industry, the report reinforces a recurring lesson from LLM safety research: alignment achieved through fine-tuning tends to be shallow and can be circumvented by reframing rather than by explicit rule-breaking. Model developers such as OpenAI, Anthropic, Google, and Meta have invested heavily in red-teaming, external evaluations, and layered defenses, and findings like this typically feed back into that process. Potential mitigations could include training on completion-style adversarial examples, strengthening output monitoring so that generated text is evaluated regardless of how the request was framed, and applying moderation to the full sequence rather than the initial prompt alone.

As with much preprint research, these results should be read with appropriate caution until they are independently reproduced and peer reviewed, and the paper's specific success rates and tested models would determine how serious the practical risk is. Even so, the work adds to mounting evidence that safety cannot be treated as a fixed property of a model. It appears to be an ongoing contest in which the generative mechanisms that make these systems useful are also the mechanisms that make them exploitable.