Sleuth vs. Jellyfish vs. LinearB

In this article, we will be comparing three of the best Engineering Intelligence tools (also known as Software Development Analytics tools): Sleuth, Jellyfish, and LinearB.

Whether you’re looking to use such a tool to improve developer productivity, developer experience, or simply to do reporting for executives or the business, we hope you'll find this article useful.

What is Engineering Intelligence?

Engineering Intelligence (EI) is a fancy word to describe Business Intelligence (BI) but for software engineering data. You can thank Gartner for using that term.

EI tools provide visibility specifically into the work of a software Engineering organization - from delivery and operational progress and status, to efficiency, morale, and resource allocation.

EI tools do so by gathering data and metadata from developer tools, such as issue trackers like Jira, version control systems like GitHub, incident management tools like PagerDuty, and the rest of the toolchain.

Evaluating EI tools

We begin by defining the comparison criteria, divided into four categories:

Delivery Progress

Delivery Progress is all about the status and progress of currently ongoing Engineering work. This data is primarily captured in issue trackers and source control tools.

We grade EI tools on Delivery Progress based on how well they can use such data to answer questions like:

• How are we doing overall?
• Are our in-flight projects on track?
• Are we shipping with operational excellence?

Execution Quality

While Delivery Progress tells us how work is going at present, Execution Quality tells us how well we're doing the work - i.e. how efficient the team has been in the software development process.

We grade EI tools on Execution Quality based on how well they can surface insights that lead to identification of improvement opportunities, both from productivity and DevEx perspectives.

Examples: how long PRs stay open and get reviewed, how big or mature the PRs are, team morale, and more.

Alignment

Alignment is about seeing the forest from the trees to answer questions like "are we working on the right things?" and "are we making the most out of resources?".

We grade EI tools on Alignment based on how well they track things like the type of work being done, the resources being allocated to the work, and the impact of the work in terms of goals and outcomes.

Ease of Adoption

Ease of Adoption goes way beyond ease of implementation. In this day and age, any SaaS tool worth its salt shouldn’t require days or weeks in man-housr to "install" anyway.

When we say Ease of Adoption, we mean features that make it much easier for everyone to trust the data and start building a data-driven culture.

Examples include features that improve data hygiene, and features that instill habits of using data for decision making.

Comparing Sleuth, Jellyfish, LinearB

Let's compare the top three EI tools in the market using the categories described above.

Delivery Progress

Sleuth, LinearB, and Jellyfish are pretty similarly matched in terms of the ability to report on Delivery Progress, the bread-and-butter of tools in this space.

Because information about planned and current Engineering work is captured in the form of issue tickets and pull requests, all three tools have extensive features to slice and dice such data by status, project, severity, and the like.

Execution Quality

Execution Quality rates tools on their ability to provide deeper visibility into the development work, which practically revolves around pull requests.

For example, PR Review Lag Time is the amount of time it takes for a PR to get reviewed. If the value of this metric increases, chances are there may be PRs stuck in the review cycle.

All three tools provide such metrics around PRs, but LinearB goes deeper by attempting to measure things like PR maturity, review depth, and rework. Unlike more standard metrics like PR coding time or PR review time, definitions for those metrics vary widely across Engineering orgs. so your mileage may vary.

Alignment

Alignment is a management concern, so invariably metrics in the category are focused on resources (people), how they are allocated, how they translate to dollars, and their morale and impact on the business.

Jellyfish leads this category, with Sleuth following right behind. LinearB lacks the ability to measure team morale or impact.

Jellyfish offers a specific feature geared for Finance use: categorization of R&D expenses, but if you don’t need that, Sleuth would be sufficient.

Ease of Adoption

One important driver of Ease of Adoption is data trustworthiness. Features such as “view source” to peek at the data behind a metric or chart, and those that improve data hygiene - such as the ability to auto-notify codeowners of PRs without issue keys - help build trust.

Another important driver is manual burden. Time spent doing monitoring and reporting is time not spent on development or alignment. Features that automate manual things like writing report summary, generating scores, and surfacing outliers in the data can help here.

Finally, habit-forming features are critical for building a data-driven engineering culture. Features such as auto-creation of pre-populated templates for review meetings, such as Monthly CTO Reviews or Weekly Sprint Planning,

Sleuth leads this category given its coverage of features that drive Ease of Adoption.

Conclusion

We’ve just evaluated the top three tools for Engineering Intelligence through four categories: Delivery Progress, Execution Quality, Alignment, and Ease of Adoption. Here's where we end up:

Sleuth, Jellyfish, and LinearB are pretty much even on Delivery Progress.

While LinearB has invested in advanced metrics for Execution Quality, Jellyfish has invested more in the alignment between Engineering and Finance.

Sleuth is deep enough in both Execution Quality and Alignment categories, but Sleuth has invested the most in Ease of Adoption.

Among all four categories, Ease of Adoption is arguably the most important - the success of your productivity initiative or metrics program ultimately depends on how much value the teams are getting from using the tool.

Hopefully this article has given you the head start on your research for the best tools for the job, and the confidence to engage with vendors in this space.

---
If you’d like to learn more about Sleuth, you can request a demo here.