Every tool now claims to be powered by artificial intelligence (AI), which means the phrase itself tells you almost nothing. So what should marketers actually look for when they’re digging through the myriad AI options available?
It starts with what matters most: finding a tool that solves your specific problem, works with your data (never easy), and can fit into your workflows without major changes.
Too often, teams skip those basics and jump straight into pilots. The results are predictable: 95% of AI pilots fail. The problem usually starts upstream in how we evaluate AI tools. Our focus drifts from outcomes to features and buzzwords, and pilots lose direction before they even begin.
AI isn’t the issue. We just need to ask better questions.
Start with the problem you’re trying to solve
We’ve fallen into the trap of evaluating AI tools backwards. Teams either get stuck considering every new product on the market or hand the task to an innovation team to pilot in isolation. Both approaches fail for the same reason: They start with the technology instead of the business problem.
Start by defining your most important use cases. Do you need to attribute revenue to campaigns? Improve email open rates? Cut customer acquisition costs? Once that’s clear, you can narrow the field.
Skipping this step often leads to drowning in an endless list of options. While your team spends months debating between three platforms, your competitor picks a direction and starts learning. Even if the outcome is “this doesn’t work for us,” they’re ahead. Progress beats indecision every time.
The right sequence is simple: Identify the problem, prioritize the use cases where AI can strengthen or speed up the solution, then move forward.
Most tools will fail the data test
Once you know what you’re solving for, it’s time to tackle the harder question: Will this work with your own data?
Vendor demos run on clean, ideal data sets. In practice, data availability and quality remain some of the biggest hurdles in AI adoption. Most enterprise data is fragmented, siloed, outdated, or constrained by privacy regulations. That’s where most pilots start to fall apart.
Take an enterprise search tool. It promises to surface the right information at the right time, but it’s only as good as the data underneath. How does it know which document is the most recent when you have three conflicting versions? How does it handle incomplete metadata?
That foundation is often the missing link. As Branch’s chief product officer, Irina Bukatik, explains it: “Everyone wants a model, but building one requires infrastructure most simply don’t have. It’s understandable; we didn’t need sophisticated data pipelines five or ten years ago. Now, how you save data, where it goes, how it’s structured, and who has access are all mission-critical.”
Branch’s own AI is no exception. It can analyze campaign data in real time, visualize results, and surface action items, but it still relies on consistent tagging, link data, and well-defined event naming to work. (The good news is that AI can help improve that consistency, too). Still, no matter how advanced the model, inconsistent data inputs always lead to incomplete answers.
This becomes even harder for organizations with legacy systems. Tools like Lovable, a website builder that lets you create products without code, are incredibly cool. But if you have complex backends, existing integrations, and years of technical debt, you can’t just rip and replace. Enterprise complexity matters, and most AI tools aren’t built for it (yet).
Before committing to any tool, pressure-test it against your reality:
- Will it work with fragmented data across multiple systems?
- What happens at the edges, not just the happy path shown in demos?
- What infrastructure changes do we need to make this work?
- Is this built for our enterprise complexity, or for a startup with a clean slate?
Understanding probabilistic vs. deterministic outputs
There’s also a fundamental mindset shift that most teams overlook: moving from deterministic to probabilistic thinking.
Traditional software is deterministic. Create a Salesforce report that filters for accounts over $100K, and you’ll get the same list every time. If something looks weird, you know there’s a bug. The system is wrong, not your expectation.
AI doesn’t work that way. Ask Movable Ink’s Da Vinci to generate subject lines for your campaign and run it twice. You’ll get different variations each time. The same input can generate different outputs. That’s not broken; it’s how AI works.
If we evaluate AI tools expecting the same black-and-white consistency as traditional software, every pilot will feel broken. The goal should be meaningful improvement, not perfection. That means adjusting our expectations about what success looks like when outputs vary by design.
How to evaluate AI tools (like any other tool)
In six months, we’ll all stop saying “AI tools.” Most of us have been using AI and machine learning technology for over a decade, whether we realized it or not. Generative AI opened new doors, but it didn’t fundamentally change how we should evaluate technology.
1. Ask the right questions
Think about email service providers. You need one to do email marketing. Does it matter if OpenAI or Azure powers its send-time optimization? Not really. What matters is whether it solves your problem: Does it keep you out of spam folders? Does it show which segments are actually engaged? Does it automate the testing you’d otherwise do manually?
When assessing any AI tool, focus on its value and potential blockers:
- Does it solve a real business problem?
- Can it work with the data and systems we already have?
- How easily can it fit into current workflows?
- What tangible improvement would signal success?
2. Pilot with discipline, not desperation
There are hundreds of tools promising to make marketing faster, smarter, and more creative. It’s worth experimenting. But only with purpose.
Every time I open LinkedIn, someone is posting about their custom GPT that just analyzed 500 sales calls or built an entire content calendar. The pressure to adopt the latest is everywhere. But the race to have the newest tools is fruitless.
The right approach to AI tools is the same as any other enterprise technology:
- Experiment, but with clear success criteria.
- Pilot quickly, and fail fast when something doesn’t work.
- Don’t rip out what’s already working just because a new model dropped.
The fundamentals haven’t changed. The only way to know if a tool is right for your business needs is through disciplined evaluation.
Want to chat about AI in mobile marketing? Let’s talk.
