In our recently published white paper, “The human touch in AI: A human-centric approach for better AI and data product development,” Sanja Bogdanovic, head of data solutions at HTEC, explores key challenges in data and AI projects and the benefit of a human-centric approach. She addresses issues like misaligned objectives, unrealistic stakeholder expectations, technology-driven product development, and the critical importance of expert engagement from the earliest stages. In this three-part series, Sanja expands on these insights and shares personal experiences that underscore why early-stage strategies are essential for long-term project success. [Read the full white paper here].
When embarking on any data or AI project, there are often hidden obstacles like the ones mentioned above that, if not addressed early, can derail the entire solution. To ensure success, the first stages of the project — understanding the problem and aligning on objectives — are critical. Research by The Centre for Business Analytics shows that 80% of all data science projects still fail in 2024, many due to stakeholder misalignment and a lack of problem understanding.
To illustrate how this looks in real life, I’ll share an episode from a crucial meeting with our internal company stakeholders. It was one of those meetings where everyone knew the stakes were high. The possibility of losing a client was very real, and we met to understand the root cause of the issue and discuss a possible turnaround strategy.
The why behind our customer’s disappointment
We started by systematically dissecting every aspect of the engagement: the requirements, the business objectives, the delivery process, the composition of the team, and even leadership decisions. The goal was to identify any missteps that could have contributed to the client’s dissatisfaction. However, despite our exhaustive review, there weren’t any obvious flaws in the technical solution itself. The real issue was something less tangible but equally impactful — our client’s lack of understanding of how to use the final product.
From their perspective, the product failed to align with their needs. It wasn’t about a few missing features or technical hiccups; there was a fundamental disconnect. They couldn’t see how the solution was supposed to solve their problems because they never fully grasped what it was designed to do.
At first, it was tempting to label this as a communication problem — and yes, there were certainly communication challenges along the way. Expectations weren’t always clear, and deliverables sometimes seemed disconnected from the desired outcomes. But blaming it all on communication felt like taking the easy way out. The root of the issue was deeper: it lay in the people who were involved on both the client’s and our side — or rather, in the lack of the right expertise at critical decision points. Throughout the project, we were bridging the gap in data literacy with our client’s stakeholders, which sometimes made it difficult to convey various approaches and gain alignment on the proposed solutions. Additionally, many of our team members were new to data-driven projects, and, in hindsight, our approach could have benefited from stronger guidance and leadership. This experience served as a valuable learning moment for both teams, highlighting areas for growth in collaboration and project planning.
When does a stakeholder need help to understand the product? What does it mean to deliver a confusing result, and how can you avoid getting yourself into this uncomfortable situation?
The right way to approach your data and AI product journey
No journey comes without challenges, but the unpredictability of the road ahead makes it exciting. Being aware of the unknowns, understanding their potential impact, and transforming that awareness into safeguards can make the data and AI project journey an adventure.
Here’s my advice, based on personal experience (and a few failed AI projects), on how to make your AI journey fun and steer clear of common pitfalls in the first stage of the product lifecycle: problem definition. For additional clarity, I’ve divided problem definition into two sub-stages: sparking interest and securing buy-in. Now, let’s dive into each — examining the obstacles that may come your way and what you can do to avoid them.
Stage 1: Sparking interest — stakeholder alignment and collaboration between the experts
In this early phase, you’re establishing a relationship with your stakeholders and sparking interest in the potential of data and AI. The biggest obstacles during this phase are often stakeholder alignment, expert collaboration, and a technology-driven design approach. Inspired by the excitement of “doing something with AI,” technical teams and clients often neglect the stakeholder problems that need to be solved.
Obstacle: Technology-driven enthusiasm over problem-driven focus
With the popularity of AI, most companies want to build a solution using data and AI. You might find that your stakeholders are blinded by the promise of AI, while you’re eager to prove the value of the technology and your expertise. This shared enthusiasm can unintentionally lead to a technology-first mindset, where the solution is driven by what data and AI can do rather than the actual problems the stakeholders need to solve.
Solution: Shift the focus to problem-driven design
Instead of rushing into building a solution around the latest AI capabilities, focus on deeply understanding the problem your stakeholders face. Initiate early discussions and focus on understanding your stakeholders’ context and objectives. Do you and your team feel comfortable with the project’s direction and level of expertise? It’s important to determine both early and identify any potential risks. Most importantly, it’s critical to shift the focus from what we can do with data and AI to how we can solve this specific problem. As much as this step is important for your stakeholders to recognize you as a reliable partner, it is equally important for you to identify the environment in which you and your team can thrive. Don’t ignore your findings in this stage, instead, aim to align you and your client’s perspectives.
Stage 2: Securing buy-in without overcommitting
Once you’ve sparked interest and aligned mindsets, the next stage is critical for defining the scope and securing buy-in. This is where many projects stumble, often due to the pressure to close deals quickly or incorrect assumptions about the problem’s complexity.
Obstacle: Rushing through problem definition
At this stage, it’s crucial to gain a thorough understanding of the client’s problem. Without this, initial misunderstandings can lead to poorly aligned objectives, misallocated resources, and a solution that ultimately fails to address the client’s real needs. For example, if the problem is data-driven, but you don’t have a grasp on how to leverage data in the solution, or apply AI, then you may make a critical error. This could look like building a custom problem blueprint or relying on an existing one based on incomplete information. Also, without a complete understanding of the project’s scope, you may presume that your current team is technically competent and miss the opportunity to include additional experts with relevant domain experience.
The scariest part? In most cases, when this occurs, teams are unaware of the early obstacles they’ve placed on their way.
Solution: Involve the right experts early and take time to unravel the problem
To avoid underestimating the problem’s complexity, you need to involve a diverse set of experts from the start. These should be high-level thinkers — think technology, data, domain, and business strategists — or people who can help you thoroughly assess the project’s scope and potential challenges. You don’t have to assemble a large team at this stage; a couple of domain experts and data specialists can provide crucial insights that prevent missteps later on.
Additionally, open discussions about your stakeholders’ data maturity and AI readiness are essential. The Boston Consulting Group’s Digital Acceleration Index (DAI) is a tool used to measure an organization’s digital maturity across 42 aspects in seven categories. It categorizes them from data champions (top 25%) to data laggards (bottom 25%), providing a clear benchmark for digital progress and readiness. A recent BCG report found that data champions outperform laggards with four times more scaled AI use cases and five times the financial impact.
Assessing the client’s digital maturity
Assessing the clients’ digital maturity is a collaborative effort, representing the first step towards establishing mutual alignment between your team and theirs. Together, you’ll select participants for the maturity and readiness survey, run the survey, collect results, and conduct additional interviews to resolve any ambiguity in your insights. Use this time to have open conversations with the stakeholders and consider multiple approaches to the project’s problem. This will help identify potential gaps in the client’s capabilities that might slow or derail the project’s progress. Use this stage to ensure both sides are aligned and avoid the trap of overcommitting based on incomplete information.
Knowing when to walk away
It is perfectly fine to walk away from a potential project if your evaluation and assessments imply that the stakeholders’ organization cannot support the solution they’re requesting. Gartner predicts that by 2025, 30% of generative AI (GenAI) projects will be abandoned after proof of concept, due to low data quality, questionable risk controls, escalating costs, or vague business value. Leaving a project at this stage is not a loss — it’s a victory for both you and your stakeholders. It is a bold decision that saves time, money, and the reputation of all parties involved.
However, if data maturity and organizational readiness indicate the potential for success, you and the stakeholders will update your scope accordingly so that you can achieve the desired outcome with a greater level of predictability.
Takeaway: build a foundation for success
In the early stages of a data and AI project, it’s easy to let enthusiasm and pressure push you toward rushed decisions. However, with a focus on problem-driven design, early expert involvement, and proper attention to data maturity and AI readiness, you can avoid many of the common pitfalls that stall, derail, or doom these projects. Remember, the foundation you lay in these first stages will dictate the success of the entire journey.
Understanding the early pitfalls is just the beginning. In the next part, Sanja explores the discovery stage, uncovering more obstacles and how to overcome them. Stay tuned for Part 2 of the data and AI solutions pitfalls and our advice on how to overcome them.
