Being a data super geek, I absolutely love defining KPIs (Key Performance Indicators). I use them for everything in life. I want to lose 15lbs in the next three months – then I need to know what KPIs to monitor over the next three months in order to know if I’m on track to meeting this goal. A logical KPI would be to monitor what my scale tells me I weigh every morning. Another KPI might be tracking how much energy I have everyday hoping to see an incremental improvement. In this article we will define six of the most important KPIs you should be monitoring within your Developer API Platform.
Before we begin… A Quick Primer on KPIs and OKRs
KPIs are not the methods used to achieve the specific goals you’ve set, nor are they a definition of what you’re trying to achieve. Those are OKRs “Objective & Key Result”. KPIs are simply what metrics are absolutely important to monitor in order to have a clear understanding that you are on track in meeting the goal (OKR) you’ve set for yourself. You begin by setting OKRs for your Developer Experience. An OKR’s “Objective” needs to be measurable. “Making developers happy” is NOT a measurable Objective. A measurable Objective might be “To reduce Developer Support Tickets by 50% this quarter” with a Key Result being “Faster issue resolution for Developers using less Developer Support resources”.
Now that you’ve defined which OKRs are most important to focus on at the moment, now comes the fine art of defining what data points to be monitored in order to clearly see how your actions are moving the needle on archiving your Objective. These are your KPIs!
These are 6 of the most important KPIs all Developer Experiences PMs should be tracking no matter what OKRs they’ve set for themselves.
1. API Uptime
Let us dive right into the most important KPI you should be monitoring – The overall health of your APIs. Back in 2020, the world learned just how much of the web services everyone uses relied on Amazon’s AWS APIs when an outage that lasted three hours took down 30% of the internet. An API outage at Twilio caused businesses that rely on their communication APIs to lose money every minute it was down. Even if the APIs you provide developers aren’t mission critical services, they are still providing some kind of value to those developers that trusted your API services wouldn’t cause their own solutions to suffer. Reliability is paramount. Make sure you are monitoring your API’s health in order to solve for potential infrastructure upgrades your APIs require to remain stabile as more and more Developers are accessing them.
2. Developer Funnel Analytics
Developer Experiences follow the same type of funnel often used by SaaS sales teams. Tracking metrics at each step developers take will show you where to focus your efforts in order for your Developer Experiences to grow. The following steps in the Developer funnel highlights the data trail you should be following.
- Acquisition — Incoming traffic to your Developer Experience
- Registration — Number of Developer registrations from incoming traffic
- Retention — Number of API test calls made per registered developer (See #3 below)
- Revenue — Number of successfully launched API integrations
- Referral — Ratio of new incoming traffic growth per launched integration
3. TTFHW – Time To First Hello World
It doesn’t matter how many Developers your API Platform might be attracting if none of them are actually launching integrations. In the universe of Developer Experiences, the term TTFHW or “Time To First Hello World” is used to define the measurement of time Developers take from first Registering to actually launching their integration using your APIs. Continually working on your Developer Experience in order to shorten the time it takes for Developers to launch is key to its success and growth. How fast TTFHW is happening is an important KPI to monitor as you’re developing better tools and documentation to increase your number of successful integrations.
4. SDK Usage
SDKs enable Developers to integrate your APIs faster and easier. Your Developer Experience should be offering SDK downloads in all of the most popular programming languages such as PHP, iOS, Android, Ruby, Java, etc. Tracking which SDKs your developers are downloading the most provides valuable information on what programming languages you should focus the most on in your documentation and code examples.
5. Ratio of Success and Failed API Calls
Just as important as measuring the overall health of your APIs, a solid KPI for indicating if your Developers are having issues understanding how your APIs work is to monitor failed API calls. This could highlight a potential error in your API documentation or even worse, an API not functioning as it should.
6. Support Tickets
The amount of Developers requiring help from your support team is a key metric for how well your Developer Experience is performing. This KPI reveals areas of trouble such as confusing API documentation, or the need for specific tutorial articles and sample code for widely misunderstood problems your Developers are facing when trying to integrate your APIs.
Conclusion
Choosing what KPIs will give you a clear understanding of the health and growth of your Developer Experience is one of the most important skills a DX Product Manager can learn. Sometimes one of the best KPIs to monitor is the feedback you get from your Developers directly. Just ask them! For a deeper dive into OKRs and KPIs, check out the book “Measure What Matters” by John Doerr. Also, check out our article “7 Mistakes Companies Make Launching an API Developer Experience” for more advice on developing amazing Developer Experiences.
