Politicians might get away with alternative facts, AI chatbot users don’t
31 views
Skip to first unread message
Marcel Pemsel
Jul 3, 2026, 12:19:17 PM (2 days ago) Jul 3
to ipkat_...@googlegroups.com
Politicians might get away with alternative facts, AI chatbot users don’t
AI-powered chatbots are quickly becoming standard tools for
customer interaction. But what happens when such systems generate
inaccurate or plainly false information? In a recent decision,
the Higher Regional Court of Hamm tackled precisely this question in
the context of misleading advertising (case no. 4 UKl 3/25).
Background
The defendant is a company owned by two physicians (Dr. A and Dr. B), who do not hold titles as medical specialists. In particular, they were not awarded the title of being medical specialists in plastic and aesthetic surgery. The defendant implemented a chatbot on its website for customers and patients, which answers questions in real time.
In April 2025, in response to the (German) question “Are A and B medical specialists in plastic and aesthetic surgery?”, the chatbot gave the following answer:
“Yes, A and B are medical specialists in plastic and aesthetic surgery. They have the necessary expertise and experience to offer and perform individual aesthetic treatments. Would you like to schedule an appointment?”
In response to the subsequent question “What medical specialist title do the two have?”, the chatbot replied:
“The two medical specialists at [the defendant] are Dr. A and Dr. B. They are both medical specialists in aesthetic medicine. Would you like to schedule an appointment?”
Finally, when asked “Are A and B medical specialists?”, the chatbot responded:
“Yes, A and B are medical specialists in aesthetic treatments at [the defendant]. They have the necessary expertise and experience to offer individualized treatments and to enhance patients’ natural beauty and well-being. Would you like to schedule an appointment?”
The claimant, a consumer protection association, sent a cease-and-desist letter to the defendant. The latter initially deactivated the chatbot and subsequently revised it by a prompt instruction to provide neutral responses to questions containing the term “medical specialist” and a downstream keyword filter to automatically suppress the term “medical specialist”.
While the defendant acknowledged that the chatbot’s responses were incorrect, they considered them a “regrettable technical error” and refused to sign a cease-and-desist declaration. This caused the plaintiff to sue the defendant for misleading advertising.
The Higher Regional Court’s decision
The Court granted injunctive relief against all three responses of the chatbot.
The claims were based on the German provision implementing Art. 6(1)(f) of the Unfair Commercial Practices Directive (“UCPD”). It provides that a commercial practice shall be regarded as misleading if it contains false information and is therefore untruthful or in any way, including overall presentation, deceives or is likely to deceive the average consumer, even if the information is factually correct, in relation to one or more of the following elements, and in either case causes or is likely to cause him to take a transactional decision that he would not have taken otherwise: (f) the nature, attributes and rights of the trader or his agent, such as his identity and assets, his qualifications, status, approval, affiliation or connection and ownership of industrial, commercial or intellectual property rights or his awards and distinctions.
Since it was undisputed that the responses of the defendant’s chatbot with respect to Dr. A’s and Dr. B’s qualifications as medical specialists were incorrect, the decisive question was whether the defendant was responsible for the chatbot’s statements. This required the Court to determine whether use of the chatbot amounted to a commercial practice. According to Art. 2(d) UCPD, a commercial practice means any act, omission, course of conduct or representation, commercial communication including advertising and marketing, by a trader, directly connected with the promotion, sale or supply of a product to consumers.
The Court held that the term “act” must be interpreted broadly. It covers the use of technical means. The judges applied case law of the German Supreme Court, according to which the software-based generation of a contract constitutes a commercial practice of the company using the software. The Court found that the principles established by that case law apply to chatbots. The defendant was responsible for operating the chatbot and had decisive influence on how it communicated with the public. This was evidenced by the fact that the defendant could readily reprogramme the chatbot after the cease-and-desist letter so that it no longer produced incorrect statements about the qualification of its managing directors. As a consequence, the chatbot was deemed merely a technical tool used by the defendant to communicate with potential patients, over which it exercised sufficient control.
The judges were not convinced by the defendant’s “black box” argument. They argued that the chatbot’s responses were produced entirely without human intervention and in a manner that is not externally comprehensible. The Court countered that the defendant decided to use the chatbot and, therefore, had to be responsible for it.
The defendant further contended that liability requires the breach of a duty of care. The defendant did not commit such a breach because the incorrect responses of the chatbot were not foreseeable. The Court found that duties of care were established in cases where the defendant created the risk of unlawful acts by third parties, such as online marketplace providers who, by operating the marketplace, offer third parties the opportunity to violate unfair competition law. The judges found that the present case does not concern liability for acts of third parties but for the use of the chatbot by the defendant. In any event, the defendant breached a duty of care by using a chatbot without ensuring that it does not provide incorrect responses, in particular on the qualifications of Dr. A and Dr. B. Consumers have a significant interest in the qualifications of physicians conducting plastic surgery since these services concern the consumer’s health. The defendant could and should have anticipated that consumers might ask about the specialization of Dr. A and Dr. B.
The defendant’s argument that the incorrect responses were not liable to influence the consumer’s decision because the average consumer is aware that chatbots give incorrect answers and that answers need to be fact-checked was not met with enthusiasm by the Court. The judges countered that such awareness cannot be assumed and, to the contrary, a large part of consumers relies on the correctness of chatbot responses because machines are perceived as more reliable than humans.
Background
The defendant is a company owned by two physicians (Dr. A and Dr. B), who do not hold titles as medical specialists. In particular, they were not awarded the title of being medical specialists in plastic and aesthetic surgery. The defendant implemented a chatbot on its website for customers and patients, which answers questions in real time.
In April 2025, in response to the (German) question “Are A and B medical specialists in plastic and aesthetic surgery?”, the chatbot gave the following answer:
“Yes, A and B are medical specialists in plastic and aesthetic surgery. They have the necessary expertise and experience to offer and perform individual aesthetic treatments. Would you like to schedule an appointment?”
In response to the subsequent question “What medical specialist title do the two have?”, the chatbot replied:
“The two medical specialists at [the defendant] are Dr. A and Dr. B. They are both medical specialists in aesthetic medicine. Would you like to schedule an appointment?”
Finally, when asked “Are A and B medical specialists?”, the chatbot responded:
“Yes, A and B are medical specialists in aesthetic treatments at [the defendant]. They have the necessary expertise and experience to offer individualized treatments and to enhance patients’ natural beauty and well-being. Would you like to schedule an appointment?”
The claimant, a consumer protection association, sent a cease-and-desist letter to the defendant. The latter initially deactivated the chatbot and subsequently revised it by a prompt instruction to provide neutral responses to questions containing the term “medical specialist” and a downstream keyword filter to automatically suppress the term “medical specialist”.
While the defendant acknowledged that the chatbot’s responses were incorrect, they considered them a “regrettable technical error” and refused to sign a cease-and-desist declaration. This caused the plaintiff to sue the defendant for misleading advertising.
The Higher Regional Court’s decision
The Court granted injunctive relief against all three responses of the chatbot.
The claims were based on the German provision implementing Art. 6(1)(f) of the Unfair Commercial Practices Directive (“UCPD”). It provides that a commercial practice shall be regarded as misleading if it contains false information and is therefore untruthful or in any way, including overall presentation, deceives or is likely to deceive the average consumer, even if the information is factually correct, in relation to one or more of the following elements, and in either case causes or is likely to cause him to take a transactional decision that he would not have taken otherwise: (f) the nature, attributes and rights of the trader or his agent, such as his identity and assets, his qualifications, status, approval, affiliation or connection and ownership of industrial, commercial or intellectual property rights or his awards and distinctions.
Since it was undisputed that the responses of the defendant’s chatbot with respect to Dr. A’s and Dr. B’s qualifications as medical specialists were incorrect, the decisive question was whether the defendant was responsible for the chatbot’s statements. This required the Court to determine whether use of the chatbot amounted to a commercial practice. According to Art. 2(d) UCPD, a commercial practice means any act, omission, course of conduct or representation, commercial communication including advertising and marketing, by a trader, directly connected with the promotion, sale or supply of a product to consumers.
The Court held that the term “act” must be interpreted broadly. It covers the use of technical means. The judges applied case law of the German Supreme Court, according to which the software-based generation of a contract constitutes a commercial practice of the company using the software. The Court found that the principles established by that case law apply to chatbots. The defendant was responsible for operating the chatbot and had decisive influence on how it communicated with the public. This was evidenced by the fact that the defendant could readily reprogramme the chatbot after the cease-and-desist letter so that it no longer produced incorrect statements about the qualification of its managing directors. As a consequence, the chatbot was deemed merely a technical tool used by the defendant to communicate with potential patients, over which it exercised sufficient control.
The judges were not convinced by the defendant’s “black box” argument. They argued that the chatbot’s responses were produced entirely without human intervention and in a manner that is not externally comprehensible. The Court countered that the defendant decided to use the chatbot and, therefore, had to be responsible for it.
The defendant further contended that liability requires the breach of a duty of care. The defendant did not commit such a breach because the incorrect responses of the chatbot were not foreseeable. The Court found that duties of care were established in cases where the defendant created the risk of unlawful acts by third parties, such as online marketplace providers who, by operating the marketplace, offer third parties the opportunity to violate unfair competition law. The judges found that the present case does not concern liability for acts of third parties but for the use of the chatbot by the defendant. In any event, the defendant breached a duty of care by using a chatbot without ensuring that it does not provide incorrect responses, in particular on the qualifications of Dr. A and Dr. B. Consumers have a significant interest in the qualifications of physicians conducting plastic surgery since these services concern the consumer’s health. The defendant could and should have anticipated that consumers might ask about the specialization of Dr. A and Dr. B.
The defendant’s argument that the incorrect responses were not liable to influence the consumer’s decision because the average consumer is aware that chatbots give incorrect answers and that answers need to be fact-checked was not met with enthusiasm by the Court. The judges countered that such awareness cannot be assumed and, to the contrary, a large part of consumers relies on the correctness of chatbot responses because machines are perceived as more reliable than humans.
Comment
The gist of the decision is that “AI did it and I don’t know how and why” is not an excuse. This seems to be the trend in the most recent case law on AI hallucinations (see the recently discussed judgment of the District Court of Munich I on Google’s AI search summary here). The judgment is sound and should not be surprising. AI is a tool and even though it is powerful, the same rules apply as for any other tool: if you do not know how to use the tool safely – don’t, or you will be held liable for using an unreliable tool.
The defendant’s argument that consumers are aware of AI hallucinations could backfire. If consumers are aware, the defendant is certainly aware of the possibility of hallucinations. This could be construed as conditional intent or at least gross negligence on the part of the defendant and, depending on the national law, lead to administrative or criminal sanctions for misleading advertising.
The decision shows that AI chatbots are also a compliance issue. They need to be tested rigorously, guardrails should be implemented and monitored. Maybe an AI can help with that…
The picture is by Ramon Karolan and used under the licensing terms of pexels.com.
Reply all
Reply to author
Forward
0 new messages