Identifying if AI is useful for your team and getting started: A practical guide for Engineering Leaders
“What's your plan for AI for your team?”, “what are we doing about AI?”, or “what’s your AI strategy?”, are questions that many engineering leaders are getting from their bosses. I’ve also heard from many teams whose product roadmap says, “something with AI.”
So, what do you do when you get those questions? And how do you cut through the hype around AI to figure out if there's something that's actually useful for your business, as well as your users? As engineering leaders face mounting pressure to "do something with AI" with their team(s), the challenge isn't just about adopting a new technology—it's about doing so thoughtfully, deliberately, and pragmatically to create real value and avoid hype-driven pitfalls. And in this guide, you’ll learn how to do it.
The gist:
Exploring AI should be done like you would with any other new technology:
Why thoughtfulness, deliberation, and pragmatism are so important when assessing AI for your team
The tech industry prides itself on innovation, but too often, it means that we follow hypes, trends, patterns, and go into extremes. The herd mentality in our industry is strong, because the concern about missing out on the next big thing is real. Currently, the “(next) big thing” is AI, and while the technology presents interesting opportunities with applications to improve identification and treatment of diseases, much of the broad discourse about it is currently also full of big promises without much substance. And AI is already expected to have vast impact on the global climate crisis, with Datacenters expected to emit 3x more carbon dioxide because of generative AI. Over the last decade, I've also spoken a lot about baked-in issues like data biases and ethical concerns, as well as the disastrous consequences on users and humanity at large, such as in A Talk about Nothing and Debugging the Tech Industry: A Talk about you and me.
Meanwhile, the pressure leaders are getting to use this technology is vast, not just for internal uses like with coding assistants, but also for external users. I've worked with many leaders over the last years to help them identify if there's substance to AI for their business and how to assess and use it in ways that actually make sense (or realize that AI isn’t the right solution for their problems). The “actually make sense”-part is crucial.
I also covered this topic in my latest Leadership Confidential podcast episode in a conversation with Daniel Paulus, VP of Engineering at Checkly, you can listen to it here:
A Step-by-Step Guide to Building Your AI Strategy
1. Start with the Problem, Not the Solution
The tech industry loves AI hype right now. But building something with AI just because everyone else is doing it? That's exactly how you end up with expensive solutions to non-existent problems.
Before diving into implementation, like with any other new and exciting technology, the first and most crucial step is identifying genuine problems that AI might help you solve. After all, AI is a tool to solve a problem, not an end in itself. Don’t chase technology for technology’s sake, but understand business problems and what tools could help you solve them.
Start by looking at your actual challenges:
What problems do your internal or external customers keep bringing up? (This Forbes article has some good pointers on identifying issues.)
What complaints show up in your issue trackers or support tickets regularly?
Which ideas have your team members or customers championed that got buried in the backlog?
Where does your strategy still have gaps that need addressing?
You might be looking at areas like:
Internal process improvements: Identify repetitive or time-consuming tasks within your organization that could benefit from automation, such as document processing, meeting summaries, or internal knowledge management.
Engineering efficiency enhancements: Look at areas like code reviews, testing, or debugging support, as well as recurring toil work on your teams.
Product feature development: Rather than bolting AI onto existing features, consider how it could genuinely expand what your product can do for users.
Cross-departmental productivity gains: Knowledge sharing between teams is often painful. Explore how AI tools could improve collaboration and efficiency across different teams.
The key is starting with a business need rather than chasing technology for technology's sake.
Watch out for red flags like:
Solutions hunting for problems: If you start with "we need AI" rather than "we need to solve X", you're probably heading down the wrong path.
Overambitious scope: Be very skeptical of any plan that promises to fully automate complex processes or replace entire teams. Start small, prove value, then expand.
Technology FOMO: Don’t add AI features just because your competitors are doing it. Understand what problem you're actually solving.
Misaligned technology choices: Sometimes a simple script or straightforward algorithm is more reliable, cheaper, and easier to maintain than a complex AI solution. Don’t overcomplicate what could be simple.
2. Assess Feasibility and Technical Fit
Before you invest serious time and resources, you need to figure out if AI is actually the right tool for your problem. This isn't just about technical feasibility – it’s about whether AI is really the best approach.
First, do your homework:
Understand current AI capabilities: AI is advancing quickly, but it also has clear limitations. Study similar implementations in your space. What worked? What failed? What surprised people?
Dive into open-source projects: The AI community is remarkably open about sharing both successes and failures. Look at how others have tackled similar problems. Their challenges can help you avoid the same pitfalls.
Map out technical requirements: What infrastructure will you need? What kind of data? What expertise? Be brutally honest about what you have versus what you'll need.
Compare with conventional approaches: Get your team members and technical leaders involved. Have them propose non-AI solutions to the same problem. Sometimes the “boring” solution is actually the best one.
Resources To Dig Into
The following resources, recommended by my podcast guest Daniel Paulus, will help you dig into research, frameworks, as well as a variety of practical examples and open-source projects:
Research & Studies:
Frameworks & Tools:
Technical Resources:
GitHub Projects:
Additional Resources:
3. Build to Learn: A Proof Of Concept (PoC)
Let’s be real: diving straight into building AI features for production is likely going to cost you dearly in both time and resources. Instead, start with a focused experiment that shows clear value while keeping risks contained.
Strong proof of concepts share a few key characteristics:
Specific scope: Pick one concrete problem to solve, not three loosely related ones
Clear timeline: 4–8 weeks is usually enough to learn if you’re onto something valuable
Defined success metrics: You need to know if it’s actually working
Real user involvement: Get feedback from day one, not just at the end
Here's how to run your PoC effectively:
Step 1: Choose Your Focus
Pick the problem: Something significant enough to matter but contained enough to tackle quickly
Define success: Set specific, measurable goals. "Better code reviews" is too vague; "50% faster security checks" is concrete
Map dependencies: What data, tools, and access will you need?
Identify stakeholders: Who needs to be involved and informed?
Step 2: Structure Your Team
Assign clear ownership: One person needs to drive this
Keep the team small: 2–3 people maximum for most PoCs
Consider external help: Don’t disrupt your core teams if you can avoid it
Set firm boundaries: Be clear about time commitments and scope
Step 3: Build Your Learning Loop
Start with assumptions: Write down what you think will work (and what won’t)
Test quickly: Get something in front of users within days, not weeks
Document everything: Capture technical findings, user feedback, and surprises
Review regularly: Schedule explicit checkpoints to assess progress
The key is treating your PoC as a learning tool first and foremost. You’re not trying to build the perfect solution – you're trying to figure out if AI can actually solve your problem in a meaningful way.
Don't wait until you think everything’s perfect. A partially working prototype that solves a real pain point will teach you more than a polished system that misses the mark. Your goal is to gather enough concrete evidence to make informed decisions about whether and how to scale up.
Remember: A successful PoC isn’t about proving the technology works – it’s about proving it solves real problems for real people in your organization.
4. Managing Resources and Budget
When it comes to AI initiatives, costs add up fast – and in ways you might not expect. The obvious costs like licenses and infrastructure are just the beginning. You’ll need to factor in growing API usage as your system scales, computing resources for hosting models, and data storage that expands with your training sets. Those startup credits various providers offer? They’re great for experiments, but make sure you budget for real production usage patterns.
The engineering investment goes far beyond initial development time. Your teams will need space to learn new tools and approaches, often while maintaining existing systems. They’ll need to build monitoring for your AI systems, handle regular model retraining, and deal with edge cases that only show up in production. Integration work often proves more complex than expected – connecting AI systems with your existing tools takes time and careful planning. And all of this needs to be documented so knowledge doesn’t stay locked in one person’s head.
Then there’s the opportunity cost to consider. Every hour your team spends on AI initiatives is an hour not spent elsewhere. What other projects get delayed? How does this affect your regular roadmap? Teams need time to get comfortable with new tools and approaches, and quick experiments can create long-term maintenance needs that you’ll have to factor into future planning.
Building expertise is another crucial investment. Your teams will need training to work effectively with AI tools, and you'll want to create internal documentation and playbooks to share knowledge efficiently. You might need consultants or specialists to fill critical gaps, especially early on. And since AI is evolving rapidly, budget for ongoing learning and development to keep your team’s skills current.
A common trap is underestimating maintenance costs. AI projects often look deceptively simple at first glance. But maintaining, monitoring, and evolving these systems takes sustained investment. Budget not just for building, but for keeping these systems healthy and effective over time.
5. Address Compliance and Security Early
Security and compliance needs to be part of your AI strategy from day one – not bolted on later when problems emerge. This is especially crucial as AI systems often handle sensitive data in new ways.
Data Privacy and Protection: AI systems are data-hungry by nature. You’ll need clear policies about what data they can access, how they process it, and where that data ends up. This gets particularly complex under GDPR and similar regulations. Pay special attention to how customer data flows through your system, and be explicit about what can and cannot be used for training or testing purposes.
Employee Data Handling: When building internal tools, you’re likely dealing with sensitive employee data – performance metrics, communication records, or project details. You need clear guidelines for how this data gets used, who can access it, and how you're protecting everyone's privacy.
Security Vulnerabilities: AI systems bring their own security challenges. Prompt injection attacks can expose sensitive data or manipulate model outputs. Model poisoning could corrupt your training data. Build security reviews into your development process and have clear mitigation strategies before deployment.
Compliance Requirements: Different industries have different rules about AI usage. Healthcare has strict patient data protection requirements. Financial services have audit trail needs. Map out your compliance landscape early and build it into your development process.
Remember: The time to think about security is not after you’ve built something cool – it’s before you write the first line of code. Getting this wrong can be extremely costly, both financially and in terms of trust.
Common Pitfalls to Avoid
AI gets people excited – but it’s easy for projects to go sideways. Here’s some traps to watch out for:
Overenthusiasm: It’s easy to get caught in the perceived potential of AI. But don’t let that excitement derail your regular roadmap. Having motivated teams is fantastic, but that energy needs to be channeled productively and balanced against your existing commitments and priorities.
Scope creep: All projects are prone to expanding beyond their initial scope, but AI projects are particularly prone to scope creep. There’s always one more feature to add, one more use case to cover. But resist that temptation – instead, start small, prove value, then expand thoughtfully. Scope creep in AI projects isn’t just annoying, it’s also often very expensive.
Ignoring alternatives: Sometimes, AI might not actually be your best bet. A traditional software solution or straightforward automation might be more reliable, easier to maintain, and more cost-effective. Don’t get so caught up in AI possibilities that you overlook simpler solutions.
Underestimating risks: AI systems can introduce risks that aren’t obvious at first glance. Security vulnerabilities, privacy concerns, and ethical implications may not all be immediately apparent during the initial planning. Think of higher-order effects, and consider that fixing issues later is usually much harder. Also, make sure you have a plan for dealing with additional issues that come up during the implementation phase or during productive use!
Lack of expertise: Building effective AI solutions requires specialized knowledge that should not be underestimated. Whether through training your team, hiring specialists, or partnering with experts, make sure you have the right skills available. Do not think that you can “just wing it” with AI and expect to achieve good results.
Many of these pitfalls aren’t obvious until you’re already dealing with their consequences. Taking time to think them through early can save you significant headaches later.
Making It Work: Practical Tips
When it comes to putting all of this into practice, there are two main challenges to tackle: finding the right expertise and managing team interest effectively. Let's look at both.
Finding the Right Expertise
Connect with local AI communities: Build relationships with practitioners and researchers through meetups and conferences – their war stories will teach you more than theoretical discussions.
Focus on practical events: Look for events that focus on practical AI implementation rather than just discussing theory, and seek out speakers and attendees who have actually shipped AI to production.
Consider freelancers: Use external expertise for initial projects, to validate concepts and create prototypes, while maintaining flexibility to scale.
Find and grow internal champions: Identify and support team members who show both interest and aptitude in AI – they'll become your future experts guiding your AI initiatives.
Managing Team Interest
Communicate thoughtfully: Frame AI projects around specific problems being solved, not just general technology adoption a.k.a. “doing AI”.
Set clear boundaries: Be explicit about who works on AI projects and when, to prevent diverting attention from core business objectives.
Balance priorities: Create structured opportunities for teams to explore AI without compromising their primary responsibilities.
Establish guidelines: Create clear policies for AI tool usage, including appropriate use cases, security and privacy considerations.
Building an AI strategy isn't about revolutionizing everything at once or chasing the latest trend. It's about thoughtfully finding where AI can actually help your team and organization, and implementing it in ways that make sense for your context and resources.
Whether you’re feeling pressure to “do AI” or are genuinely curious about its potential, keep your focus on solving real problems. Start small, validate early, and always keep your business objectives in focus. Remember: the goal is not “having AI” – it’s to solve real problems and create tangible value for your organization.
