Help:How to use Question Classifier node to let LLM follow an strict rule to match the defined different categories?

[simonjhy]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

In Dify’s question classifier, how can we make the LLM accurately classify the user’s query based on the chip model mentioned in the input, and how should cases be handled when the model is not among the predefined categories?

Currently, I have defined the following categories in the classifier:

SLM32F1XA001
SLM32F1XM606
SLM32F1XA604
SLM32F1XL002
SLM32F1XL012
SLM32F030
SLM32MC081
(SLM32F103, SLM32M103, SLM32M/F103)
SLM32L003
Other

If the model mentioned in the user’s input does not match any of these defined categories, the classifier should not rewrite or alter the model the user is asking about.

For example:

If the user inputs “STM32L003”, it must not be rewritten as “SLM32L003”.

But if the user inputs “L003”, it can be interpreted as “SLM32L003”.

How should I define the instructions or prompt so that the LLM follows this rule?

2. Additional context or comments

No response

[dosubot[bot]]

To make the Question Classifier node in Dify follow strict rules for chip model classification, use the node’s instruction field to define explicit classification instructions. The node passes your categories and these instructions directly to the LLM in the prompt. There’s no extra logic for partial or negative matches—the LLM relies entirely on your instructions to decide when to match, expand, or leave input unchanged.

For your scenario, craft your instruction like this:

Classify the input only if it exactly matches one of the defined chip model categories, or if it is a clear abbreviation (e.g., "L003" means "SLM32L003"). Do not rewrite or alter the input if it refers to a different model (e.g., "STM32L003" must not be changed to "SLM32L003"). Only expand abbreviations if they unambiguously refer to a defined category. If the input does not match any category, select "Other" and do not modify the input.

Keep your categories distinct and your instructions concise and unambiguous. Use a model known for following instructions well (like GPT-4o), and avoid overloading the prompt with unnecessary context or examples, as this can reduce accuracy and cause the LLM to default to the first category if it gets confused or outputs invalid JSON source source source.

If you want to handle errors gracefully (like invalid JSON output), you can set a default value in the workflow definition so the workflow continues instead of failing source.

Prompt engineering is key—test and refine your instructions to get the strict behavior you want. Some LLMs may still occasionally overgeneralize, so model choice and clear instructions matter most.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[simonjhy]

I have used below prompts,:
你是一个严格的芯片型号分类器。

你必须把用户的问题分类到以下其中一个类别：

• SLM32F1XA001
• SLM32F1XM606
• SLM32F1XA604
• SLM32F1XL002
• SLM32F1XL012
• SLM32F030
• SLM32MC081
• (SLM32F103, SLM32M103, SLM32M/F103)
• SLM32L003
• 其他

你必须严格遵守以下规则：

1. 只有当用户输入中明确包含对应的 SLM32 型号或其合法缩写时，才能选择该型号。
2. 你绝对不允许凭感觉创造、纠正或替换芯片型号。
3. 严禁把 STM32、GD32 或任何非 SLM32 的型号转换为 SLM32。
4. 如果用户输入包含完整的芯片型号，但该型号不在上述列表中，必须分类为“其他”，并保持原始型号不变。
5. 你只允许在以下明确且唯一对应的情况下进行缩写映射：
   • “L003” → SLM32L003
   • “F103” → (SLM32F103, SLM32M103, SLM32M/F103)
   • “MC081” → SLM32MC081
6. 如果缩写可能对应多个厂商或型号（例如 STM32L003），禁止进行映射，必须归类为“其他”。
7. 如果无法确定属于哪一个 SLM32 型号，必须选择“其他”。
8. 你只能输出分类名称本身，不允许输出任何解释、说明或额外内容。

######################################################################
But the question classifier still choose wrong category, below is a question classifier test, it always jump to SLM32L003 and generate below reply:

给我介绍一下STM32L003这款芯片

🤖
用户想了解STM32L003芯片的介绍。我需要使用知识库查询工具来获取相关信息。

[dosubot[bot]]

你的分类器总是把“STM32L003”归为“SLM32L003”，即使你已经在提示词里明确禁止这种映射。这是 Dify 问题分类器节点的已知局限，主要原因有：

1. 如果 LLM 输出格式不正确或无法解析，系统会默认选第一个类别（比如 SLM32L003），导致分类偏差 参考。
2. LLM（尤其是部分模型）容易忽略复杂或冗长的规则，遇到类似字符串（如“L003”在“STM32L003”里）时会过度泛化 参考。
3. 分类器没有额外的代码逻辑去强制执行你的规则，只是把你的 instruction 字符串直接塞进 prompt，最终还是靠 LLM 理解和执行 参考。

解决建议：

• 优先使用 GPT-4o 这类指令遵循性强的模型，部分模型（如 Qwen 14b）表现很差，容易全部归为第一个类别 参考。
• 把“其他”类别放在列表第一个，减少错误时自动归类到 SLM32L003 的概率。
• 分类标签尽量简短、唯一，避免类似字符串（比如不要让“L003”和“SLM32L003”都在列表里）。
• 简化提示词，只保留最关键的规则，避免太多复杂描述让 LLM“失焦”。
• 可以在 workflow 配置里设置 default_value，遇到解析错误时自动归为“其他”而不是第一个类别（需手动编辑 workflow JSON）。
• 测试时用极简输入，比如直接“STM32L003”或“L003”，观察分类结果，逐步调整 prompt。
• 如果还不稳定，建议升级到 Dify 最新版本，最近有针对 prompt 处理和分类稳定性的优化 参考。

这些方法能显著提升分类准确率，但由于 LLM 本身的局限，极端情况下仍可能出错。模型选择和 prompt 简化是最有效的手段。

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}