Engineering Productivity Module
Engineering Productivity Module
The Engineering Productivity Module helps engineering teams track velocity and progress towards current goals across every stage of the Software Development Life Cycle. It surfaces the unplanned work that may be competing for a team's attention and provides industry benchmarks to measure up against high-performing teams.
Accessing the Engineering Productivity Module
The Engineering Productivity Module is accessible under the Engineering Productivity folder within Faros Reports. It is a premium add-on in the Faros platform. Under Menu > Dashboards, select View All and Faros Reports.
Engineering Productivity Module Dashboards
The Engineering Productivity module is composed of four dashboards. Three of them are at the organizational level: Organization Engineering Productivity Summary, Organization Engineering Productivity Velocity and Organization Engineering Goal Detail and one is at the team level: the Team Engineering Productivity Summary. Each dashboard is detailed below.
Organization Engineering Productivity Summary Dashboard
The Organization Engineering Productivity Summary dashboard provides an organizational view where senior leaders can get a high level understanding of progress towards engineering goals (tasks required to complete for a given release, more information on that here, and overall engineering velocity across the organization.
The dashboard is structured into two primary sections: Progress Against Engineering Goals and Engineering Velocity.
All metrics can be segmented by team, release, and a specific time period, using filters at the top of the page.
- The desired team can be set using the Teams filter and defaults to all teams. You can also click on any of the team traces to drill down to a specific team.
- The desired time period is set by the Date filter and defaults to the previous 90 days.
- The release is set using the Release filter.
The Progress Against Engineering Goals section shows where an organization stands compared to its current goals and highlights other engineering tasks that may be causing delays. In Faros, a task corresponds to a ticket or work item (in the example of Jira, it covers stories, bugs and tasks). It is composed of the following metrics:
| Metric | Description |
| Progress to Goal | Number of tasks completed relative to the number of tasks that should have been completed at this point in a release, expressed in terms of expected workdays. A negative value indicates being behind schedule, while a positive value indicates being ahead of schedule. |
| Progress towards Release Plan by Sub-Org over Time | Number of tasks completed relative to the number of tasks that should have been completed at this point in a release, expressed in terms of expected workdays, broken down by sub-orgs. A negative value indicates being behind schedule whilst a positive value indicates being ahead of schedule. |
| Number of Tasks by Status | Number of tasks in each status, week-by-week. |
| Ratio of Unplanned to Total Work | Number of unplanned tasks as a percentage of the total number of tasks completed. Unplanned tasks correspond to anything that is not a task or a story (bugs and custom types). Planned work is composed of the story and task types. A large amount of unplanned work is likely to prevent achieving goals. |
| Ratio of Unplanned to Total Work by Sub-Org over Time | Number of unplanned tasks as a percentage of the total number of tasks completed, by sub-orgs and by week. |
| Work Breakdown by Category over Time | Ratio of unplanned tasks to the total number of tasks completed, by week. A large percentage of unplanned work week after week can be an indicator of poor planning. |
The Engineering Velocity section provides additional context in the form of velocity metrics. It is composed of the following metrics:
| Metric | Description |
| Avg Task Cycle Time | Average overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed. |
| Task Cycle Time by Sub-Org over Time | Average overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed, broken down by sub-orgs week-over-week. Helps compare teams and address hotspots. |
| Task Cycle Time Avg by Estimated Task Size and Sub-Org | Average overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed, broken by sub-orgs and by task size (points). Useful to identify outliers. |
| Avg Monthly Pull Request Merge Rate | Average number of pull requests merged per team member per month. |
| Pull Request Merge Rate by Sub-Org over Time | Average number of pull requests merged per team member in a week, broken down by sub-orgs week-over-week. Useful to compare between teams. |
| Monthly Pull Request Merge Rate vs Team Size (Avg) by Sub-Org | Average number of pull requests merged per team member in a month vs. the average size of the team in each sub-org. Helps identify high-performing teams and replicate best practices. |
Organization Engineering Productivity Goal Detail Dashboard
The Organization Engineering Productivity Goal Detail dashboard provides a closer look at engineering progress towards release goals, displaying potential factors such as ratio of unplanned to total work, open to in-progress tasks, and task time spent in different stages.
The dashboard is structured into two primary sections: Goal Progress Drivers and Goal Progress by Sub-Org.
The Goal Progress Drivers section analyzes the type and stage of the work in progress. It is composed of the following metrics:
| Metric | Description |
| Work Breakdown by Category over Time | Ratio of unplanned tasks to the total number of tasks completed, by week. A large percentage of unplanned work week after week can be an indicator of poor planning. |
| Number of Tasks by Status | Number of tasks in each status, week-by-week. |
| Progress towards Planned Releases vs Ratio of Unplanned to Total Work | Number of tasks completed relative to the number of tasks expected to have been completed at this point in a release expressed in terms of workdays, and the ratio of unplanned work to the total number of tasks completed. Looking at these side by side can help explain delays in reaching goals. |
| Sum of Task Points by Status | Sum of story points in a status, week-by-week. |
The Goal Progress by Sub-Org section surfaces related trends for each sub-org. It is composed of the following metrics:
| Metric | Description |
| Progress towards Release Plan by Sub-Org over Time | Number of tasks completed relative to the number of tasks that should have been completed at this point in a release, expressed in terms of expected workdays, broken down by sub-orgs. A negative value indicates being behind schedule whilst a positive value indicates being ahead of schedule. |
| Ratio of Unplanned to Total Work by Sub-Org over Time | Number of unplanned tasks as a percentage of the total number of tasks completed, by sub-orgs and by week. Helpful to identify which areas are often impacted by unplanned work. |
| Task Time in Stage Breakdown (Avg) by Sub-Org | Average time it takes for tasks to get done, broken down by task stage and sub-org.Note: This chart includes steps from flows that have not yet completed, so results may differ slightly from other completed-flow metrics. |
| Planned vs Unplanned Work by Sub-Org | Ratio of unplanned tasks to the total number of tasks completed, broken down by sub-orgs. Helps identify which orgs are more impacted by unplanned work on average. |
| # Tasks Completed by Sub-Org over Time | Number of tasks completed, broken down by sub-orgs, week-by-week. |
| # Completed Task Points By Sub-Org over Time | Sum of story points completed, broken down by sub-orgs, week-by-week. |
| Recent Currently In-Progress Tasks | Most recent tasks currently in progress. |
Organization Engineering Productivity Velocity Dashboard
The Organization Engineering Productivity Velocity dashboard provides a closer look at velocity metrics across and within the organization, in order to identify bottlenecks and areas for improvement.
The dashboard is structured into two primary sections: Velocity by Task Stage and Velocity by Sub-Org.
The Velocity by Task Stage section gives an in depth view at velocity by stage to help identify bottlenecks.
Note: All the charts in this section include steps from flows that have not yet completed, so results may differ slightly from other completed-flow metrics. This helps identify emerging bottlenecks and taking corrective action faster. As such, results may differ slightly from other completed-flow metrics.
It is composed of the following metrics:
| Metric | Description |
| Task Stage Cycle Time (p95) over Time | P95 time it takes for tasks to get done, broken down by stage, week-by-week. P95 means the 95th percentile, i.e. 95 percent of the data within the period is lower than this value and 5 percent of the data is higher than this value. Large variations might be worth investigating. |
| Task Time in Stage Distribution (days) by Stage | Binned distribution of the time it takes for tasks to get done, broken down by task stage. High performing teams avoid distributions skewed towards longer task times. |
| PR Stage Cycle Time (p95) over Time | P95 time it takes for changes authored by team members to go from PR creation to merge, broken down by pull request stage, week-by-week. Useful to identify bottlenecks in the review process. |
| PR Cycle Time Distribution (days) by Stage | Binned distribution of time it takes for changes authored by team members to go from PR creation to merge, broken down by stage. High performing teams avoid distributions skewed towards longer PR cycle times |
The Velocity by Sub-Org section helps identify high performing teams within an organization and replicate best practices. It is composed of the following metrics:
| Metric | Description |
| Task Cycle Time (p95) by Sub-Org | P95 overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed, broken down by sub-orgs week-over-week. |
| Task Cycle Time Distribution (days) by Sub-Org | Binned distribution of overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed, broken down by sub-orgs. |
| Pull Request Merge Rate (p95) by Sub-Org over Time | P95 number of pull requests merged per team member in a week, broken down by sub-orgs week-over-week. |
| # of Merged Pull Requests vs # of Open Pull Requests by Sub-Org | Number of merged PRs vs. currently open PRs authored by team members, broken down by sub-orgs. A large number of open pull requests compared to merged pull requests should be avoided. |
| Recent Merged Pull Requests | Most recent merged pull requests. |
Team Engineering Productivity Summary Dashboard
The Team Engineering Productivity Summary dashboard provides metrics at the team level.
The dashboard is structured into two primary sections: Progress Against Engineering Goals and Engineering.
The Progress Against Engineering Goals section shows where a team stands compared to its current goals and highlights other engineering tasks that may be causing delays. It is composed of the following metrics:
| Metric | Description |
| Progress to Goal | Number of tasks completed relative to the number of tasks that should have been completed at this point in a release, expressed in terms of expected workdays. A negative value indicates being behind schedule whilst a positive value indicates being ahead of schedule. |
| Progress towards Release Plan by Release over Time | Number of tasks completed relative to the number of tasks that should have been completed at this point in a release, expressed in terms of expected workdays, broken down by release. A negative value indicates being behind schedule whilst a positive value indicates being ahead of schedule. |
| Number of Tasks by Status | Number of tasks in each status, week-by-week. |
| Ratio of Unplanned to Total Work | Number of unplanned tasks as a percentage of the total number of tasks completed. A large amount of unplanned work is likely to prevent achieving set goals. |
| Ratio of Unplanned to Total Work by Release over Time | Number of unplanned tasks as a percentage of the total number of tasks completed, by release and by week. |
| Work Breakdown by Category over Time | Ratio of unplanned tasks to the total number of tasks completed, by week. A large percentage of unplanned work week-after-week can be an indication of poor planning. |
The Engineering Velocity section provides additional context in the form of velocity metrics. It is composed of the following metrics:
| Metric | Description |
| Avg Task Cycle Time | Average overall time it takes for tasks to be completed, starting from when they are first in progress to when they are last closed. |
| Task Stage Detail Cycle Time (p95) Over Time | P95 time it takes for tasks to get done from when they start being worked on, broken down by stage detail, week-by-week. |
| Task Time in Stage Distribution (days) by Stage Detail | Binned distribution of time it takes for tasks to get done, broken down by task stage detail. Helps identify bottlenecks in the task cycle.Note: This chart includes steps from flows that have not yet completed, so results may differ slightly from other completed-flow metrics. |
| Avg Monthly Pull Request Merge Rate | Average number of pull requests merged per team member per month. |
| PR Stage Cycle Time (p95) Over Time | P95 time it takes for changes authored by team members to go from PR creation to merge, broken down by pull request stage, week-by-week. Useful to identify bottlenecks in the review process. |
| PR Cycle Time Distribution (days) by Stage | Binned distribution of time it takes for changes authored by team members to go from PR creation to merge, broken down by stage. Useful to identify outliers in the review process.Note: This chart includes steps from flows that have not yet completed, so results may differ slightly from other completed-flow metrics. |
Setting up the Engineering Productivity Module
The Engineering Productivity Module is centered around 4 key metrics:
- Progress to Goal
- Planned and Unplanned work
- Task Cycle Time
- Pull Request Merge Rate
To compute each of these metrics, several data sources should be connected to the Faros platform, as detailed in the following sections.
Note: For most of the sources below, you will find handy validation dashboards in the Getting Started folder under Faros Reports. These dashboards are provided to all Faros customers and let you inspect the data coming from different sources such as version control systems or task management systems.
Platform Prerequisites
The following should be set up on Faros in order to get the full functionality of the Engineering Productivity module:
-
Version control, VCS: The data will be used to access deployment data to calculate PR merge rate. Team membership of the PR author is also used for team attribution.
-
Organization and Team Members: This will allow Faros to know the teams in your organization, the reporting structure, and the size of each team.
-
Team Attribution: To help determine which team each of the metrics is associated with
- Board Ownership: For bugs (unplanned work)
Progress to Goal
Schema location: tms_Task
Sources: Task Management, TMS
Validation Dashboard: TMS Source Metrics
Releases should be defined in your Task Management System that have both a start date and a release date, and the tasks required to complete the release should be linked to it as well. The “Board Ownership” association will determine which team a release on a board belongs to. As work on the release progresses, you are able to see how far you are ahead of schedule or behind based on the number of tasks completed, not yet completed, and the amount of time remaining until the release.
Planned and unplanned work
Schema location: tms_Task
Sources: Task Management, TMS
Validation Dashboard: TMS Source Metrics
Tasks of type “Bug” should be reported to the Faros platform and are considered “unplanned work”. The “Board Ownership” association will determine what team the bugs belong to and the Org/Team setup will determine the number of developers on a team.
Task Cycle Time
Schema location: tms_Task
Sources: Task Management, TMS
Validation Dashboard: TMS Source Metrics
Requires tasks status to first move into an “In Progress”-type state and then later to a “Done”-type state. Board Ownership determines which team the tasks belong to.
Pull Request Merge Rate
Schema location: vcs_PullRequest
Sources: VCS
Validation Dashboard: VCS Source Metrics
The average number of pull requests merged per month per person on a team is calculated using the “merged at” timestamp of a pull request and the number of team members on the pull request author’s team.
Updated 5 months ago