AI & Technology

Why relying on ChatGPT isn’t enough — and why accountants need a tool built for us

In the wake of rapid advances in generative AI, many accounting firms and advisory practices are experimenting with chat-based tools to speed up research,...

September 15, 2025
5 min read
Tax Stack AI Team
ChatGPT for accounting
AI for accountants
tax AI assistant
accounting AI software
Why relying on ChatGPT isn’t enough — and why accountants need a tool built for us

In the wake of rapid advances in generative AI, many accounting firms and advisory practices are experimenting with chat-based tools to speed up research, draft memos, summarise regulations, etc. On its face, it seems like a win: ask ChatGPT a tax-question, get a quick answer. But what happens when the “quick answer” is wrong? When the AI invents a source, misstates a regulation, or provides plausible but inaccurate guidance?

That risk isn’t hypothetical. Consider the high-profile case of Deloitte Australia: the firm delivered a government-commissioned report (for roughly A$440,000 / US$290,000) that contained multiple errors—including fake citations, invented case law quotes, and references to academic papers that apparently don’t exist. The consulting firm ultimately agreed to partially refund the government contract.

What does that tell us, in the context of accounting and advisory work? A lot.

  1. The risk of “hallucinations” in generative AI

Generative AI tools—like ChatGPT and others—work by predicting plausible completions of text, based on enormous language-model training data. They do not guarantee factual accuracy, legal compliance, or domain-specific expertise. In the Deloitte case, one researcher observed:

“…the report was quoting academics and legal experts who didn’t exist.”

In the accounting/ tax world, that kind of error can be disastrous—not just embarrassing. Fabricated references, mis-quoted statutes or regulations, or missing disclosures can expose firms to compliance, audit, or reputational risk.

  1. Why accountants can’t treat ChatGPT like an encyclopedia

The Deloitte situation highlights a core lesson for accounting professionals: you are still accountable for the work, you still need to apply professional judgment, and you still need to verify. Even if you use ChatGPT to draft something, you must review, validate, and document. As one expert put it:

“Accountants have to own the work, check the output, and apply their judgment rather than copy and paste whatever the system produces.”

If your firm is using a generic chat tool as a research shortcut, you risk errors creeping in—especially with complex tax issues, evolving regulations, international matters (e.g., business relocation, cross-border income) or advisory work where accuracy is non-negotiable.

  1. The accounting-specific problem: tax, compliance, context

For a CPA firm (such as yours) engaged in advisory work, tax planning, relocations, and compliance (particularly with the niche of Act 60 and relocations to Puerto Rico), the stakes are even higher. You are dealing with:

Highly specific statutory language

Regulatory filings and reports

Data required by CPAs, attorneys, realtors in collaboration

Ready to transform your practice with AI? Talk to Tax Stack AI about a custom system built for your firm.

Real-world consequences for clients (e.g., tax incentives, relocation costs, residency rules)

In that environment, a generic chat-AI tool may produce what looks like a credible answer—but without full domain awareness you risk missing key exceptions, mis-interpreting residency rules, or worse: giving guidance that could lead to audit exposure or client mis-steps.

  1. Why you need a tool built for accountants — and how Tax Stack AI fits

Given the realities above, here’s how your firm’s offering (Tax Stack AI) can position itself:

It’s built specifically for accounting/tax/advisory workflows (not a general-purpose chat tool).

It incorporates professional-grade controls: verified data sources, domain-specific tax logic, audit-trail, versioning, and review workflows.

It is designed for compliance: a system where outputs can be documented, source-checked, linked to real regulation, and flagged if something doesn’t pass muster.

It supports your advisory niche rather than vanilla chat conversations.

It helps your team maintain oversight and professional judgment rather than outsource that to an imperfect model.

By positioning the conversation this way, you’re not rejecting AI altogether—you’re saying: “Yes, we use modern tools—but we use tools purpose-built for accounting risk/controls, not repurposed chat bots.”

  1. Key messages for your accountants/clients

When your firm communicates this internally or externally, consider these reframed points:

“Faster” isn’t enough; accuracy + documentation matter.

Generic chat tools may accelerate drafting—but they don’t replace domain expertise or review controls.

Errors in financial/tax guidance damage not just reputation—they potentially cost money, regulatory exposure or client loss.

A purpose-built tool ensures your firm remains the professional reviewing and owning the output—not losing control to a black-box model.

Clients want assurance: that you’re using tools that are tailored to CPA standards, audit-readiness, and client-risk mitigation.

  1. Call to action

If you’re an accounting professional, firm owner or advisor:

Review your current AI-tool strategy: what chat-bots are you using, how do you control their output, how do you document review/oversight?

Consider adopting a domain-specific tool (like Tax Stack AI) designed for accounting workflows, tax compliance and advisory work.

Implement internal controls: reviewer checklists, version histories, source-verification, disclaimers, audit-trail.

Educate your team: AI is a tool, not a substitute for your professional judgment.

Conclusion

The Deloitte Australia case is a cautionary tale for all professionals using generative AI: even major firms make mistakes when relying on chat-based or generative tools without proper oversight. In the world of accounting, tax and advisory services—which demand high accuracy, regulatory compliance and professional accountability—this risk is magnified.

At Tax Stack AI, we believe in embracing technology—but doing so intelligently: deploying tools built for accounting, backed by domain expertise, review controls and audit-ready documentation. Because your reputation, your client outcomes and your firm’s liability depend on more than a quick prompt and an AI-answer.

Transform Your Accounting Practice Today

We build custom AI systems that help tax professionals streamline workflows, reduce costs, and deliver better client service.

Complimentary intro consultation for accounting firms