Why Software Engineers Need Better Risk Assessment Tools for Pull Requests


As a software engineer, you may have shipped something that you and the reviewers knew was risky, but it was too expensive to investigate the issue further. You may have added remediation steps, but time pressure may have prevented you from doing so.

Software Engineers are inherently risk managers but often not thought of as having this responsibility. After their work is deployed however, they are expected to respond and diagnose production issues. Perhaps this is one reason why the observability tooling are so robust. It’s time to shine a light on this underemphasized yet critical job responsibility.

Having objective AI-powered tools to aid us can lead to positive behavioral change not only for engineering but for the whole organization. Shepherdly provides a predictive model that is trained on your production error and bug data. This is important because it aligns risk with customer pain, a concept the whole organization can align around and empathize with.

Here are a few ways to apply a risk assessment tool like Shepherdly to your workflow:

JIT Remediation

Know when to deploy your de-risking efforts and to what degree.

It’s common for teams to have existing playbooks for guarding against risk. The challenge is applying them with precision. Risk is a spectrum and so should the remediation tactics. Shepherdly’s risk score acknowledges this fact and provides a range of 1-100 along with the statistical probability of causing a bug. While this is team specific, a corresponding spectrum of remediation could look something like:

  • Extra reviewers with a focus on bug inspection
  • Expanded automated test coverage
  • Feature Flags
  • Increased observability (tracing, logging, product usage metrics) for changed code
  • Dedicated QA investment (if staffed)
  • Modified phased rollouts

Because Shepherdly uses the probability of a bug being introduced as its proxy for risk, paired with training on your actual production data, the risk scores are personalized to your environment and more importantly, the impact your customers feel.

Customer Driven Tech Debt

Identify & prioritize tech debt by actual business value, with objective data aligned with your customers.

One of the challenges in managing technical debt is prioritizing it effectively. Simply showing a product manager the cyclomatic complexity of your code base may not be enough to convince them. However, demonstrating that your team’s repository has the most customer-facing bugs can quickly change the conversation.

Within a repository, teams can gain precise insight into which areas of the codebase to target for refactoring by identifying the files that are the most fragile.

Sprint Planning

Understand how tech debt is going to skew your estimate.

During the scoping & estimating process, it can be tricky to convey an anticipated area of the code that will be difficult to modify. With the Hotspot view in Shepherdly, you can quickly identify if an area of the codebase you’ll be modifying is brittle and therefore will require more effort to work through in order to reduce bugs and minimize the impact of bugs occurring in production.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top