Writing OKRs For Data Science

OKRs – Objectives and Key Results – encourage you to align departmental and team Objectives with what the Company wants to achieve. The measurable part of an OKR comes in the form of 1 – 4 Key Results. These numerically describe the desirable end-state you want to reach for.

For example, let’s imagine a bank that wants to have a market leading Customer Experience has a growing issue with fraud and has this OKR.

OBJECTIVE – Spot fraud most of the time and reduce customer stress
KEY RESULT – Reduce the amount of fraud going undetected and reported by customers by 30%

For the coming 3 months a big drive to solve and help achieve this has been given to the data science team.

The outcome you want is well defined but the OKRs that would help support this are often where challenges appear. OKRs are best when they are written alongside OKR consultants and coaching sessions.

Data science, like most science, is a process of experimentation. As such the improvements you want to make are in the process itself.

Data Science OKRs

With OKRs it’s necessary to have in-teams and cross-functional team conversations about goals. The conversations are all geared towards where improvements can be made and collaboration and alignment optimization.

If Objectives are where you describe the ‘what’ and ‘why’, and your Key Results are what success looks like with a degree of stretch built in to stimulate innovation, increase effort and prolong focus.

For Data Science teams, measurable improvements in the form of Key Results might look like:

• Improve the efficiency of doing X by Y
• Accurately detect X Y% of the time
• Increase the data accuracy from X to Y
• Reduce our Log Loss from X to Y
• Improve our F Score from X to Y
• Increase data completeness from X to Y
• Reduce the time it takes to use a new data source from X to Y
• Get data from X into Y in Z time
• Increase the number of data sources used from X to Y
• Increases the number of models approved from X to Y
• Reduce the time it takes to create and validate a model from X to Y
• Reduce the time it takes to deploy a model from X to Y
• Reduce defects detected by data science team from X to Y

In the above fraud example, a supporting OKR might be:

OBJECTIVE – Improve the fraud detection algorithm significantly
KEY RESULT – Accurately detect fraud 98% of the time

After discussion in the team it was decided that the current log-loss method punishes false negatives and false positives equally and it’s proposed to discover whether punishing false negative and false positives results differently might help.

With OKR the activities that are done, which in this case is being labelled ‘False Negative / Positive Differential Treatment’, are called Initiatives. This is one of the Initiatives the team is doing.

Initiatives are no different to most agile processes. There’s a backlog of things that could be done to achieve specific, desirable and measurable outcomes. The obvious aim being to progress the ones most likely to have the desired impact and close the loop on learning whether it did or not.

This agile way of working is central to the OKR methodology which is why it’s so easily adopted by engineering teams. The only real leap is to embrace being connected to hard and measurable value orientated outcomes.

The good news is that having your work connected with business value is welcome by most. Being disconnected is a state that is less fulfilling as it offers the possibility of your work being less valuable or undervalued – it’s way easier to recognise and reward teams that achieve high value outcomes.

OKR Planning Cycles

All of the way through a typical quarterly OKR cycle there are weekly check-ins where progress is discussed, plans for the week, problems and successes. This gives everyone the opportunity to make any necessary changes.

At the end of what for most are quarterly planning cycles, the question asked are:

• Did you achieve the desired outcomes?
• What was learned?
• Should the OKR be continued?
• What new OKRs would continue to support the higher level Objectives?

What’s in the secret sauce

In essence then, OKRs are simple. An Objective and a few ways of measuring success. However there is something else going on. OKRs encourage you to optimise your data science processes around high value processes that create achievements and business outcomes that can be measured.

That said, what needs to be clear is that OKRs are not about achieving goals at 100% all of the time – as any OKR consultants would tell you. They’re a framework designed to make failure safe and develop learnings. Depending on how difficulty was calibrated it might be that 70% would be seen as an achievement.

The secret sauce that helped you achieve more came from having clarity on an outcome and the ability to imagine what an amazing level of success would look like, and then try to achieve it. Collaborating, innovating, learning and adapting as you go.

OKR Examples For Different Departments

Company OKR Examples
Finance OKR Examples
HR OKR Examples
Marketing OKR Examples
Sales OKR Examples
Customer Service Examples
Engineering OKR Examples
Product Marketing OKR Examples
Product Management OKR Examples
Data Science OKR Examples