Developer Experience Module

The Developer Experience module helps you leverage employee surveys to their full potential.

It centralizes survey data from agile, team health, and developer experience surveys in one place and intersects sentiment data from employee responses with telemetry-based data from engineering operations.

This novel blended visibility into the complete developer experience allows you to take corrective measures faster and monitor their impact on engagement, retention, and operational excellence over time.

Accessing the Developer Experience Module

The Developer Experience Module is accessible under the Developer Experience folder within Faros Reports. It is a premium add-on in the Faros platform. Under Menu > Dashboards, select View All and Faros Reports. It is also accessible via the 9-dot Modules icon at the top right of the Faros application. 

Developer Experience Dashboards

The Developer Experience module is composed of five dashboards. The Developer Experience Summary dashboard provides an overview of your survey results. The other dashboards let you deep dive into four key dimensions of developer experience: Alignment and Goals, Developer Productivity, Quality and Speed and Agility. We will go into detail about each one and how to set them up below.

Developer Experience Summary Dashboard

The Developer Experience Summary dashboard summarizes survey results and provides trends by team and question categories/dimensions over time.

The dashboard is structured into three primary sections: survey score, question categories scores and surveys summary.

All metrics can be filtered by team, survey, question category and a specific time period. 

• The desired team can be set using the Teams filter, and defaults to all teams. It is also possible to click on any of the team traces to drill down to a specific team
• The desired time period is set by the Date filter at the top of the page, and defaults to the previous 12 months. 
• By default all surveys are shown. You can select a specific survey or set of surveys as desired using the Survey dropdown menu
• By default all question categories are shown. You can select a specific question category or set of categories as desired using the Question Category dropdown menu

The Survey Score section provides an average score for your surveys based on the filter selected. It all provides trends over time and a breakdown by team

The Question Categories Scores section provides a breakdown of survey scores by question category, by sub-org and over time.

The Surveys Summary section provides a summary of recent surveys as well as response count over time and by teams.

Developer Experience Alignment and Goals Dashboard

The Developer Experience Alignment and Goal dashboard summarizes survey results for the Alignment and Goals question category and intersects it with relevant metrics such as progress to goal or ratio of unplanned work to help analyze and diagnose underlying factors, compare sentiment with data, preempt issues, set goals and track progress.

The dashboard is structured into two primary sections: alignment and goals survey score, and metrics

All metrics can be filtered by team and a specific time period. 

• The desired team can be set using the Teams filter, and defaults to all teams. It is also possible to click on any of the team traces to drill down to a specific team
• The desired time period is set by the Date filter at the top of the page, and defaults to the previous 12 months. 

The Alignment and Goals Survey Score section provides an average score for Alignment and Goals questions. It all provides trends over time and a breakdown by team

The Alignment and Goals Metrics section shows relevant metrics such as progress to goal, ratio of unplanned work or backlog size vs WIP and intersects them with survey data. Like anything else in Faros, you can of course customize the metrics shown to those that make most sense for your organization and at this point in time.

Developer Experience Developer Productivity Dashboard

The Developer Experience Developer Productivity dashboard summarizes survey results for the Developer Productivity question category and intersects it with relevant metrics such as task throughput or PR Merge rate to help analyze and diagnose underlying factors, compare sentiment with data, preempt issues, set goals and track progress.

The dashboard is structured into two primary sections: developer productivity survey score, and metrics

All metrics can be filtered by team and a specific time period. 

• The desired team can be set using the Teams filter, and defaults to all teams. It is also possible to click on any of the team traces to drill down to a specific team
• The desired time period is set by the Date filter at the top of the page, and defaults to the previous 12 months. 

The Developer Productivity Survey Score section provides an average score for Developer Productivity questions. It all provides trends over time and a breakdown by team

The Developer Productivity Metrics section shows relevant metrics such as task throughput, PR throughput, task cycle time and PR merge rate and intersects them with survey data. Like anything else in Faros, you can of course customize the metrics shown to those that make most sense for your organization and at this point in time.

Developer Experience Quality Dashboard

The Developer Experience Quality dashboard summarizes survey results for the Quality question category and intersects it with relevant DORA metrics such as Change Failure Rate or Bugs to developer to help analyze and diagnose underlying factors, compare sentiment with data, preempt issues, set goals and track progress.

The dashboard is structured into two primary sections: quality survey score, and metrics

All metrics can be filtered by team and a specific time period. 

• The desired team can be set using the Teams filter, and defaults to all teams. It is also possible to click on any of the team traces to drill down to a specific team
• The desired time period is set by the Date filter at the top of the page, and defaults to the previous 12 months. 

The Quality Survey Score section provides an average score for Quality questions. It all provides trends over time and a breakdown by team

The Quality Metrics section shows relevant metrics such as Change Failure Rate, Mean Time to Resolve, Bugs Per Developer, or Incidents and intersects them with survey data. Like anything else in Faros, you can of course customize the metrics shown to those that make most sense for your organization and at this point in time.

Developer Experience Speed and Agility Dashboard

The Developer Experience Speed and Agility dashboard summarizes survey results for the Speed and Agility question category and intersects it with relevant metrics such as frequency of deployments and lead time to help analyze and diagnose underlying factors, compare sentiment with data, preempt issues, set goals and track progress.

The dashboard is structured into two primary sections: speed and agility survey score, and metrics

All metrics can be filtered by team and a specific time period. 

• The desired team can be set using the Teams filter, and defaults to all teams. It is also possible to click on any of the team traces to drill down to a specific team
• The desired time period is set by the Date filter at the top of the page, and defaults to the previous 12 months. 

The Speed and Agility Survey Score section provides an average score for Speed and Agility questions. It all provides trends over time and a breakdown by team

The Speed and Agility Metrics section shows relevant DORA metrics such as Deployments, Lead Time as well as Task to Deploy, Pull Request and PR pick-up and intersects them with survey data. Like anything else in Faros, you can of course customize the metrics shown to those that make most sense for your organization and at this point in time.

Setting up the Developer Experience Module

The Developer Experience Module focuses on analyzing employee surveys and thus requires ingesting survey data into Faros.

Optionally, it provides a way to intersect survey data with a number of other metrics around velocity, quality and task management, thus requiring a number of other data sources to be connected to Faros for the full experience.

Note: For most of the sources below, you will find handy validation dashboards in the Getting Started folder under Faros Reports. These dashboards are provided to all Faros customers and let you inspect the data coming from different sources such as version control systems or task management systems.

Ingesting survey data

There are multiple ways to import survey data into Faros, which are described here.

Please note that mapping your question categories to Faros Categories (including Alignment and Goals, Developer Productivity, Quality and Speed, and Agility) is an optional step which is not necessary to leverage the Summary Dashboard but is required to leverage the other four dashboards. If your survey questions only cover a subset of these dimensions, feel free to only map those to leverage the corresponding dashboards and you can ignore the other ones.

Once you have ingested employee surveys into Faros, you can leverage the Survey Source Metrics dashboard to get out of the box reports on survey data. This dashboard is located in the Getting Started folder under Faros Reports.

 

 The Survey Source Metrics dashboard shows recent surveys, distribution of survey types, response count by type, teams and over time and stats around time to complete and responses.

Platform Prerequisites

The following should be set up on Faros in order to get the full functionality of the Developer Experience module:

• Organization and Team Members: This will allow Faros to know the teams in your organization, the reporting structure, and the size of each team.

Alignment and Goals

To get the full functionality of this dashboard, you need to map the “Alignment and Goals” Faros category to your corresponding category. See how to do that here.

In addition, the following sources should be connected:

Progress to Goal

Schema location: tms_Task

Sources: Task Management, TMS

Validation Dashboard: TMS Source Metrics

Releases should be defined in your Task Management System that have both a start date and a release date, and the tasks required to complete the release should be linked to it as well. The “Board Ownership” association will determine which team a release on a board belongs to. As work on the release progresses, you are able to see how far you are ahead of schedule or behind based on the number of tasks completed, not yet completed, and the amount of time remaining until the release.

Ratio of unplanned work and Backlog Size/Completed Vs WIP

Schema location: tms_Task

Sources: Task Management, TMS

Validation Dashboard: TMS Source Metrics

Tasks of type “Bug” should be reported to the Faros platform and are considered “unplanned work”. The “Board Ownership” association will determine what team the bugs belong to and the Org/Team setup will determine the number of developers on a team.

Developer Productivity

To get the full functionality of this dashboard, you need to map the “Developer Productivity” Faros category to your corresponding category. See how to do that here.

Task throughput & Task Cycle Time

Schema location: tms_Task

Sources: Task Management, TMS

Validation Dashboard: TMS Source Metrics

PR throughput & Merge Rate

Schema location: vcs_PullRequest

Sources: VCS

Validation Dashboard: VCS Source Metrics

The average number of pull requests merged per month per person on a team is calculated using the “merged at” timestamp of a pull request and the number of team members on the pull request author’s team.

Quality

To get the full functionality of this dashboard, you need to map the “Quality” Faros category to your corresponding category. See how to do that here.

Change Failure Rate

Schema location: ims_Incident, cicd_Deployment

Sources: Incident Management, IMS, Events

Validation Dashboard: IMS Source Metrics, Deployment Table

Change Failure Rate is calculated as the number of incidents per deployment. Incident counts are derived from incidents reported to the Faros platform. The “Application Ownership” association will determine what team the incidents belong to. Deployments should also be reported to Faros. This metric is only computed on high-severity incidents (Sev1 or Sev2) and for deployments to the Production environment.

Mean Time to Resolve

Schema location: ims_Incident

Sources: Incident Management, IMS

Validation Dashboard: IMS Source Metrics

Mean Time to Recovery is calculated leveraging the faros-time-to-resolve flow which can be found under Settings > Flows. Incident counts, stages and times to resolve are derived from incidents reported to the Faros platform. The “Application Ownership” association will determine what team the incidents belong to.

Bugs Per Developer

Schema location: tms_Task

Sources: Task Management, TMS

Validation Dashboard: TMS Source Metrics

Tasks of type “Bug” should be reported to the Faros platform. The “Board Ownership” association will determine what team the bugs belong to and the Org/Team setup will determine the number of developers on a team.

Incidents

Schema location: ims_Incident

Sources: Incident Management, IMS

Validation Dashboard: IMS Source Metrics

Incident counts are derived from incidents reported to the Faros platform. The “Application Ownership” association will determine what team the incidents belong to.

Speed and Agility

To get the full functionality of this dashboard, you need to map the “Speed and Agility” Faros category to your corresponding category. See how to do that here.

Deployments 

Schema location: cicd_Deployment

Sources: Events

Validation Dashboard: Deployment Table

Deployment frequency measures how often an organization successfully deploys new code to production and requires deployments to be reported to Faros. 

Lead Time

Schema location: vcs_PullRequest, cicd_Build, cicd_Deployment

Sources: VCS, Events 

Validation Dashboards: VCS Source Metrics, Deployment table, Flow Duration Lead Time table

Lead Time for Changes is calculated leveraging the faros-lead-time flow which can be found under Settings > Flows. It requires VCS data to compute times from PR created to PR merged to Build and Deployment events for the rest.

Task to Deploy

Schema location: tms_Task, vcs_PullRequest, cicd_Build, cicd_Deployment

Sources: Task Management, TMS, VCS, Events 

Validation Dashboard: TMS Source Metrics, VCS Source Metrics, Deployment table, Flow Duration Lead Time table

PR & PR pick-up

Schema location: vcs_PullRequest

Sources: VCS

Validation Dashboard: VCS Source Metrics

The average number of pull requests merged per month per person on a team is calculated using the “merged at” timestamp of a pull request and the number of team members on the pull request author’s team.