In last week’s issue, we brought you a decision journey on how best to apply artificial intelligence (AI) and data-driven analytics to your business. But deployment isn’t enough. How confident are you that your analytics program is delivering?

Only a small fraction of the technology’s potential value has been realized, McKinsey research has found—as little as 10 percent in some sectors. There’s already a performance gap between leaders and laggards in AI and analytics adoption, and that gap is expected to grow.

Frustrations are beginning to surface as many executives realize they’ve failed to convert their analytics pilots into scalable solutions. Some have spent millions of dollars on advanced analytics but can point to almost nil in impact on the bottom line. If you’re getting into the game, consider these ten red flags that signal an analytics program is in danger, as well as how to avoid them.

In one large financial-services firm, the CEO was an enthusiastic supporter of advanced analytics. He was especially proud that his firm had hired 1,000 data scientists, each costing about $250,000 a year. Later, as the new hires began to fall short of expectations, it was discovered that many of them were not, by strict definition, data scientists at all. Neither the CEO nor the firm’s human-resources group had a clear understanding of the data-scientist role—nor of other data-centric roles, for that matter.

Meanwhile, some companies have been so focused on hardcore data experts that they’ve neglected a role that can genuinely make or break performance: the analytics translator. This position is best filled by someone on the business side who can help leaders identify high-impact analytics use cases and then “translate” business needs to data scientists, data engineers, and other tech experts, who then build actionable analytics solutions.

Devising these digital use cases also means thinking through potential ethical and regulatory issues, which can prove unexpectedly tricky. One large industrial manufacturer, for instance, set out to understand the links between job conditions and absenteeism, in an attempt to identify and redesign processes apt to lead to injuries or illnesses. The company meant well, but the algorithms it developed looked for correlations with factors like ethnicity, religion, and gender, although these demographic data fields were switched off. Fortunately, the company caught and corrected the problem before incurring steep regulatory fines—and turning an effort meant to protect employees’ health into one subject to algorithmic bias.

The takeaway: working with data, particularly personnel data, introduces a host of risks. Significant supervision, risk management, and mitigation efforts are required to apply the appropriate human judgment to the analytics realm.

Finally, it’s useful to remember that applying AI and other digital solutions to business tests even the most agile leader. Rapid change is part and parcel of today’s business environment, challenging well-grooved leadership approaches.