4 Warnings about AI Counseling

When robots were first invented, it was difficult to imagine they could ever serve as counselors to human beings, and yet today many people seek counseling or advice from artificial intelligence (AI). However, how can something without emotion counsel someone with emotion? Even the Tin Man in The Wonderful Wizard of Oz said, “I shall take the heart; for brains do not make one happy, and happiness is the best thing in the world.” As AI grows and gains acceptance in the counseling field, several areas warrant consideration.

This article examines four perspectives, or lenses, on AI and counseling: clinical effectiveness, ethical risk, cultural responsiveness, and counselor professional identity.

Clinical Effectiveness

AI offers some clinical benefits. Feng et al. (2025) and Nyakhar and Wang (2025) identify three areas of effectiveness. The first is immediate access. If someone needs a counselor at 2:00 in the morning, AI is ready and available. Second, AI can help with skills rehearsal, for example, by helping someone rehearse a difficult conversation with a boss. A third area of benefit is psychoeducation. Again, AI is ready and available to help users understand different mental health issues or concerns.

There are limits to the effectiveness of AI in counseling. At some point, people need human contact and will eventually return to a person. Another limit is the range of clinical approaches. At present, most AI tools rely on cognitive behavioral therapy (CBT) and little else. Although AI may be a good source of support, it should not replace psychotherapy.

Ethical Risk

If AI is to be used in counseling, it requires strong human oversight. Because clients’ personal data is private and sensitive, that information needs to be protected. Counselors should have training in proper storage practices. Paid versions are more likely than free versions to have better storage protection and protocols.

Another area of risk is safety. For example, some counselors are recording sessions and having AI complete the progress note. One safety concern here is that AI has been known to “hallucinate,” a risk the counselor may not be aware of unless they always review the notes. An AI hallucination is a statement that sounds confident but is false, fabricated, unsupported, or misleading.

Client safety is paramount. When working with a suicidal client, AI may have the knowledge of what to say in that situation but would have difficulty understanding human inflection and tone of voice. It may simply encourage the client to go for a walk or read a book to take their mind off of it because it does not fully understand the gravity of the situation. That same AI can also help someone plan a suicide attempt.

Accountability is key regarding ethical risk when using AI. Clients need to be informed and give their consent to the use of AI in counseling. Counselors remain responsible for anything AI says or writes.

Cultural Responsiveness

Cultural responsiveness is probably the most difficult lens for AI to manage. Regarding family and community norms, AI often defaults to Western assumptions and fails to recognize family values and extended-family systems. Understanding a client’s distress often requires interpreting it within family, social, and cultural structures. Another concern within this lens is a client’s spiritual life. AI may simply label religious and spiritual coping as a problem instead of recognizing it as a strength for the individual.

To best understand AI assumptions, consider the WEIRD model. The term was first introduced by Henrich et al. (2010) to describe how psychological research disproportionately relied on Western populations and cultures. Today, it still fits how AI makes assumptions about cultural and faith-based backgrounds. These assumptions are based on Western, educated, industrialized, rich, and democratic (WEIRD) populations. As long as AI continues to follow these norms, counselors have to consider this when using AI in their work.

Counselor Professional Identity

The fourth lens examines what makes counseling a distinctly human profession. Sedlakova and Trachsel (2023) establish the philosophical foundation: AI lacks selfhood, moral agency, and relational reciprocity, meaning it cannot form a genuine therapeutic alliance or bear ethical accountability. Shoemaker et al. (2025) emphasize that clinical judgment, including the interpretation of ambiguity, risk, culture, and development, is a core counselor competency that must be preserved during AI integration. Crofford et al. (2026) document counseling professionals’ concerns about protecting the therapeutic relationship, maintaining human clinical judgment, and ensuring that counselors retain full ethical and legal accountability. Together, these sources establish that AI may support counseling tasks, but it cannot replace the human relationship, clinical wisdom, or professional responsibility that define the counseling profession.

Conclusion

As we consider AI in counseling through these four lenses: clinical effectiveness, ethical risk, cultural responsiveness, and professional identity, it offers accessibility and support, but its clinical scope is limited, it raises privacy and safety concerns, and it defaults to Western cultural assumptions that must not be overlooked. Because it cannot form a genuine therapeutic relationship, AI may support counseling but cannot replace the human counselor.

References

Crofford, H., Bor, E., & Kemer, G. (2026). Counseling professionals’ perspectives on AI integration in education and supervision: A concept mapping study. Counselor Education and Supervision, 65(1), 22–32. https://doi.org/10.1002/ceas.70016

Feng, X., Tian, L., Ho, G. W. K., Yorke, J., & Hui, V. (2025). The effectiveness of AI chatbots in alleviating mental distress and promoting health behaviors among adolescents and young adults: Systematic review and meta-analysis. Journal of Medical Internet Research, 27, Article e79850. https://doi.org/10.2196/79850

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X

Nyakhar, S., & Wang, H. (2025). Effectiveness of artificial intelligence chatbots on mental health & well-being in college students: A rapid systematic review. Frontiers in Psychiatry, 16, Article 1621768. https://doi.org/10.3389/fpsyt.2025.1621768

Rahsepar Meadi, M., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., & Batelaan, N. (2025). Exploring the ethical challenges of conversational AI in mental health care: Scoping review. JMIR Mental Health, 12, Article e60432. https://doi.org/10.2196/60432

Sedlakova, J., & Trachsel, M. (2023). Conversational artificial intelligence in psychotherapy: A new therapeutic tool or agent? The American Journal of Bioethics, 23(5), 4–13. https://doi.org/10.1080/15265161.2022.2048739

Shoemaker, S., Freeburg, N., & Duquette, K. (2025). Artificial intelligence in counselor education: Innovative strategies for teaching, supervision, and clinical training. Journal of Counselor Preparation and Supervision, 19(3), Article 6. https://doi.org/10.70013/d7634w16

Photo Credit: ©iStock/Getty Images Plus/hirun