Measure Twice, Cut, and Have Fun

When I first started coding, I struggled through Computer Science classes, feeling ready to give up. But a project sparked my curiosity and passion—writing a decompression algorithm for cave diving. That experience taught me that coding can be fun, and the technical challenges less daunting when you’re engaged.

At that age, I was actively cave diving almost every weekend. For those who scuba dive and are reading this, you know that the longer you stay down the slower you have to come up to avoid “the bends”. I became curious as to why the up and down pattern in the Devils Ear Cave System sent so many divers to the decompression chamber, and started writing a (very hacky) decompression algorithm software. This project, aimed at understanding the Bühlmann ZH-L 16 and Varying Permeability Model (VPM) for dives that had a complex profile, was far more difficult than anything I encountered in class. They key thing here is I was having fun and the technical challenges didn’t seem all that technical because of it.

After college I worked at a start-up that did a lot of fun things. We were the first FAA approved iPad app for pilots to access their manuals while flying. Later we extended that to other forms and even Dispatch Releases. This period allowed me to travel to various states/countries for sales and implementation support, as well as “developer weeks” where the entire development team flew into Wisconsin and had hackathon style collaboration. It was one of the most challenging but fun periods in my career.

At this stage in life I’m now a new parent as are a significant portion of my co-workers. Actually, one cool thing about working at Empower is the great support Empower gives for new parents with up to 16 weeks of paid leave for the primary parent, a benefit that almost 7% of our employees have taken advantage of year to date!

Family commitments mean that the brute force of staying up all night to build something you’re really proud of isn’t an option anymore. When you combine this with having had so many awesome experiences over 10+ years of professionally coding, sometimes it’s hard to celebrate the wins and make sure it’s still fun.

So now let’s talk about the title of my blog— a play on measure twice, cut once. We’re not carpenters, we’re software developers. It’s okay to cut over and over and test in production. At Empower, measurements are central to everything we do, they’re how we move fast, and they’re how we drive meaningful business impact.

Let’s break down measurements into the key areas we want to talk about

• Building the right features
• Avoiding premature optimizations
• Building a culture where development is fun

Building the Right Features

Our primary focus is avoiding the creation of features with no business value—this approach is crucial for the company’s health and prevents developer burnout. At Empower, we heavily rely on Amplitude to guide our users towards positive experiences. My team uses a single dashboard displaying all critical metrics, which we review after each release and at the beginning of each week to track our progress. We prioritize anomalies in event data equally with end-user bugs, as this data drives our decision-making process.

Consider the chart below (with sensitive data removed). To illustrate our approach, here’s a chart comparing user behavior on our starting screen versus the desired outcome. This analysis allows us to pinpoint areas needing attention. We’ve grouped the data by a property representing various ongoing experiments, allowing us to see the conversion success rate for each. This insight helps us identify which areas of our funnel need attention and which are performing well. We aim to focus on features with lower conversion rates—below 50%—rather than those already achieving over 83% conversion.0%.

Avoiding Premature Optimizations

To abstract away specific business concerns, let’s come up with the fictional Entity Framework scenario of a User object along with a couple of rows of type GroupMembership with virtual property Group. As we start this, we can assume we only have 2-3 rows of GroupMembership when we first introduce the feature, and that we only need the GroupId and StartDate, not necessarily any information off the Group property.

For now, we have an existing method GetUser() that looks something like this

dbContext
	.Users
	.Where(x=> x.UserId == userId && x.IsActive == true && !x.IsFraud)
	.SingleOrDefault();

The first thing we can do is measure our query performance and setup alerting around it. For now, let’s grab the user’s group memberships.

var startTime = Stopwatch.GetTimestamp();
var user = await userRepository.GetUser(userKey);
var userGroupMembership = user.GroupMemberships.ToList();
Activity.Current?.AddTag(TelemetryEventMeasurementKeys.GroupMemberships.GetUsersGroupMembershipsDuration, new EventMetric(Stopwatch.GetElapsedTime(startTime).TotalMilliseconds));

As anyone with experience with Entity Framework knows, doing user.GroupMemberships results in another round trip to the database. That probably isn’t worth adding another UserGroupMembershipsRepository and risking missing the Fraud or Active checks above (or others that get introduced), nor is it worth always using .Include(x⇒ x.GroupMemberships) as the user might be queried in dozens of places. Here we have traded minimal code complexity for a slight performance loss— this is almost always the right decision at a start-up.

We can now utilize Application Insights Metrics to determine the p99, p95 and p90 of this query and setup an alert when it goes beyond a threshold we’re comfortable with.

customMetrics
| where name has "{metricNameGoesHere}"
| project valueSum, valueCount, timestamp
| summarize
    PercentileDuration99 = percentile((valueSum / valueCount), 99),
    PercentileDuration95 = percentile((valueSum / valueCount), 95),
    PercentileDuration90 = percentile((valueSum / valueCount), 90)
| summarize AverageP99 = avg(PercentileDuration99 / 1000), AverageP95 = avg(PercentileDuration95 / 1000), AverageP90 = avg(PercentileDuration90 / 1000)

So what we’ve done here is we’ve sped up development by reusing code, we’ve lowered the effort it takes to test in future releases by lowering the number of code paths, and we’ve protected ourselves against being blindsided by a code path getting slow to the point it times out.

Building a Culture Where Development is Fun

At the top of Diamondback, AZ

To bring it all back together, most software developers who are good at it, enjoy it. Empower focuses on having a fantastic culture even though we are a remote first company. Twice a year we have a company onsite where we bring the global team to a centralized location and split our time between business and pleasure.

At our last Phoenix onsite those activities included a choose your own adventure pack of

• Gold Mine Tour
• ATV’ing in the desert
• Guided rock climbing
• A choice between easy, medium or difficult hikes
• …and more!

On the Cash Advance Pod, we spend 30 minutes a day with “office hours” near the end of the day. Think of these as what you would do going out after work or going on a long lunch if we were all in the office.

This time has helped us with initiatives such as

• Driving conversion rates by sharing knowledge between our iOS/Android/Backend teams to explore key areas where we can improve the experience.
• Discussing technologies like SignalR, and finding ways they can improve the user experience in production
• Building a sense of team so that your co-workers aren’t just a Slack bubble, they’re someone you enjoy conversing with and want to support

Last, we take the time to do “roundups” where every pod (small team) gets the opportunity to present recent wins to everyone including the most senior of the leadership team. Of particular note compared to other places I’ve worked, leadership teams remain very engaged in these presentations— you don’t feel like you’re presenting to someone who has something better to do with their time. These celebrations of small wins ensure teammates feel recognized for their accomplishments and remain engaged.

---

About James Garrett

👋 I’m James, a seasoned backend developer with interest in crafting solutions on the C#/dotnet stack, scalability and performance challenges are my favorite. My career in software development has been a combination of startups and Fortune 500’s with startup like cultures. Mid week you’ll find me coding away, but weekends are time for fishing, hiking, boating— anything outdoors!

In my toolbox, JetBrains Rider, macOS, The Azure Cloud platform and of course the best coding language (C#) are my preferred environment. Their intuitive interfaces and powerful features enhance my developer experience.

_While I love software development, I don’t find it particularly fun in a vacuum. The projects I’ve most enjoyed in my career are those where I get interact with a team to solve new challenges— that’s why I love it here at Empower. _

What keeps me excited to come to work everyday is the fact we have a strong team based culture. Almost daily I’ll hop on a call with fellow Engineers or Product Managers to pair up on something. Culture isn’t an accident here, it’s an investment. Twice a year we bring the team together in person to get to know one another, learn what different pods are up to and form relationships.

Connect with James