Conversations with AI May Not Be Confidential

Here’s What Business Owners Should Know:

Generative AI tools like ChatGPT, Claude, and others are quickly becoming part of everyday business operations. Business owners are using them to draft contracts, analyze legal questions, brainstorm strategy, and summarize complex issues.

But a recent federal court decision highlights an important risk that many business owners may not realize: Your conversations with AI tools may not be confidential. And in some situations, they could even become discoverable in litigation.

Let’s walk through what happened and why it matters.

A federal judge in the Southern District of New York recently considered whether communications with a generative AI platform were protected by attorney-client privilege.

Attorney-client privilege is one of the strongest confidentiality protections in the law. It generally protects private communications between a lawyer and client made for the purpose of seeking legal advice.

But in this case, the communications at issue were not between a client and a lawyer. Instead, the user had input information into a consumer generative AI platform while seeking guidance. The key question for the court was simple: Were those communications confidential enough to be protected?

The court concluded they were not. Because the AI tool’s terms of service allowed the company to store and potentially use user inputs to train its models, the court found there was no reasonable expectation of confidentiality. Without confidentiality, attorney-client privilege cannot apply.

In other words: Using a consumer AI tool meant the communications were not legally protected.

Many business owners are already turning to generative AI tools for help with legal and operational questions. For example:

• “Draft a non-compete agreement.”

• “How do I terminate an employee?”

• “What legal risks should I consider before buying this business?”

• “Analyze this contract and tell me the risks.”

These prompts often contain sensitive business information, including:

• internal strategies

• confidential financial data

• potential disputes

• legal vulnerabilities

If that information is entered into consumer AI tools, two important things may happen:

1. The information may not be confidential because most consumer AI tools operate under terms that allow the provider to store and potentially review user inputs. Some platforms may also use data to improve or train models. That means the communication may not meet the legal standard for confidentiality.

2. The communication may become discoverable in litigation. If a dispute arises later, opposing counsel could potentially argue that those AI conversations are discoverable evidence. Imagine a prompt like this:

“We think our independent contractors might actually qualify as employees. How big of a legal risk is this?”

That kind of conversation could become problematic if it appears later in a lawsuit.

However, it is important to note that the court’s decision focused on consumer-grade AI platforms, not enterprise tools designed for professional use. Enterprise AI services often include:

• contractual confidentiality protections

• data-use limitations

• commitments that user inputs are not used for model training

• stronger privacy and security controls

Those protections can change the legal analysis. But the reality is that many business owners and employees are still using free or consumer versions of AI tools, which may not include those safeguards.

Generative AI is an incredible tool. It can help business owners understand legal concepts, brainstorm solutions, draft first versions of documents, and organize complex information, but it does not replace legal counsel, and it does not provide the same confidentiality protections.

When business owners rely on AI alone to analyze legal risks, they miss important benefits that lawyers provide, including strategic judgment based on real facts, advice tailored to specific jurisdictions, negotiation experience, professional accountability, and most importantly, attorney-client privilege.

The bottom line for business owners is that AI can be a helpful tool for learning and brainstorming. But when it comes to sensitive legal questions, assume that if you put a prompt into a consumer version of an AI tool, your competitors, regulators, and opposing counsel will be able to get at it somehow.

The recent SDNY decision is one of the first court rulings to address generative AI communications in the context of confidentiality, and it likely won’t be the last. As AI becomes more integrated into business operations, courts will continue to define how traditional legal doctrines apply to these new technologies. For now, the takeaway for business owners is straightforward:

AI can be a powerful assistant, but it does not provide the legal protections that come with speaking to a lawyer. And when it comes to sensitive legal matters, those protections still matter.

About Fourscore Business Law

Whether you're launching a startup, raising capital, or navigating M&A, our experienced team provides strategic, world-class legal counsel that supports your vision and enables confident decision-making at every stage of growth. Fourscore Business Law empowers forward-thinking entrepreneurs and business leaders by simplifying complex legal transactions. With a focus on efficiency and personalized service, we help you overcome legal challenges so you can focus on innovation and scaling your business. Ready to secure your startup's future? Take our free legal assessment for founders today to better understand your company’s legal health.

Start Your Free Legal Review