Feature Flags: Get your code out now - Part 2

Implementing Feature Flags Client-side

Sample code

In the previous post, Feature Flags - Part 1, we learned about what feature flags are, their benefits and dangers. This post will explain how you can implement feature flags in to the client-side of your own applications.

Implementing feature flags requires you to create a divergence in your logic where the new functionality will be called, rather than modifying the existing functionality to meet the requirements of the new feature.

Feature Flag Scenarios

Feature Flag Scenarios

At this divergence we create a decision point, which redirects the users flow through the application based on the value of the feature flag. In the image above, I show 3 code flows. The first is the current path. The second is a simple decision point where we have an entirely new flow which does not reuse any previous code. The third shows a more complex flow where we reuse some code, business logic perhaps, and have more than one decision point, as the flow progresses.

The sample code below uses a flag to determine whether or not to call a web API to retrieve a list of characters, or return a static list of predefined characters. Although the code is written in TypeScript, the concepts can be carried on to other client-side languages.

getAllCharacters(): Observable<string[]> {
    if (this.useWebApi) {
        return this.http.get(this.ninjasUrl)
                .map((res: Response) => res.json())
                .catch(this.handleError);
    } else {
        return Observable.of(this.characters);
    }
}

To decouple the client-side code from a specific feature flag service, I like to wrap it in an application level service, as seen below. This provides the ability to swap out the feature flag provider at any point. Below is the complete feature flag service.

export class FeatureFlagService {
  ldClient: any;
  flags: any;
  flagChange: Subject<Object> = new Subject<Object>();
  constructor() {

    this.flags = {'ln-search': false, 'ln-api': false};

    this.ldClient = LDClient.initialize('<LaunchDarkly_api_key>',
      {key: 'generic_user_id', anonymous: true });

      this.ldClient.on('change', (flags => {

        if (flags['ln-search'] !== undefined) {
          this.flags['ln-search'] = flags['ln-search'].current;
        }
        if (flags['ln-api'] !== undefined) {
          this.flags['ln-api'] = flags['ln-api'].current;
        }

        this.flagChange.next(this.flags);
      }));

      this.ldClient.on('ready', () => {
        this.setFlags();
      });
    };

    setFlags() {
      this.flags = this.ldClient.allFlags();
      this.flagChange.next(this.flags);
    };

    changeUser(user) {
      if (user !== 'Anonymous') {
        this.ldClient.identify({key: user, name: user, anonymous: false});
      } else {
        this.ldClient.identify({key: 'anon', anonymous: true});
      }
    };
  }

Let’s break down the above service and look at the individual pieces. With feature flags, we want to be as explicit as possible when implementing them in order to ensure clear understanding of what the code is doing. The code below defines the explicit flags we care about and defaults them to false. This prevents any accidental release of features before the application can get the actual values from the service provider.

this.flags = {'ln-search': false, 'ln-api': false};

We then initialize our connection to the feature flag service provider, as seen below. Here we are utilizing LaunchDarkly as our service provider and configure it for an anonymouse user. If at the point of initialization, the user is know, then we would go ahead and pass it in to the service initialization.

this.ldClient = LDClient.initialize('<LaunchDarkly_api_key>',
      {key: 'generic_user_id', anonymous: true });

If we intend to serve both unauthenticated and authenticated users, we can utilize the code below to change the user configured during initialization.

changeUser(user) {
    if (user !== 'Anonymous') {
    this.ldClient.identify({key: user, name: user, anonymous: false});
    } else {
    this.ldClient.identify({key: 'anon', anonymous: true});
    }
};

Once our client is initialized, we want to ensure we call the service to obtain the correct value settings for our user, and notify any subscribers.

We define our event property to allow subscribers to be notified when flag values change.

flagChange: Subject<Object> = new Subject<Object>();

Subscribe to the LaunchDarkly client’s change notification and call the setFlags() method.

this.ldClient.on('ready', () => {
    this.setFlags();
});

Inside setFlags() we want to get all flags for our client from the feature flag service and notify any subscribers there was a change in the flags value.

setFlags() {
    this.flags = this.ldClient.allFlags();
    this.flagChange.next(this.flags);
};

We also don’t want to require our users to refresh or restart the application in order to gain access to new features. We can subscribe to an event provided by our service provider, or if they don’t provide this feature, implement polling logic to notify us when the value of a feature flag we care about changes, and notify any subscribers to our event.

this.ldClient.on('change', (flags => {
    if (flags['ln-search'] !== undefined) {
        this.flags['ln-search'] = flags['ln-search'].current;
    }
    if (flags['ln-api'] !== undefined) {
        this.flags['ln-api'] = flags['ln-api'].current;
    }

    this.flagChange.next(this.flags);
}));
Feature Flags: Get your code out now - Part 1

As many teams and organizations begin to transform the way they plan, build and deliver software, as well as, obtain feedback and iterate, they begin to adopt new practices to aide in this transformation. Automating build and release pipelines has increased the efficiency and stability of releasing our software, but how can we ensure we can continually release and deliver new features and fixes without destabalizing the entire application. This is where feature flags come in to play.

Most of us have most likely used feature flags previously, we just never called it a feature flag. A Feature Flag is simply a way to turn functionality on or off, most likely through a configuration setting, without the need to deploy new code. To utilize the feature flag, we create a divergence in our code, directing users down the new path or the old path based on the flag setting, rather than replacing the existing logic. We’ll explore this more in the implementation section in part 2 of this post.

Benefits of feature flags

Decoupling the feature release from code deployment through feature flagging, reduces the risk of breaking the existing production application and allows us to continually provide bug/hotfixes to our customers at a regular interval.

Another benefit to using feature flags is the ability to turn on a new feature with a simple flip of a flag, allowing the feature to be launched whenever the prodcut owner or primary stakeholder deem it complete, or complete enough, or on a specfic product launch timeline.

The type of feature flag we’ve been talking about up to now has been a simple on/off flag. Flags also have the ability to target specific individuals, or geographical locations. With this added flexibility, we can now begin to release our features to users for beta testing, customer/stakeholder feedback, production load testing, percentage based roll-out or pay-per-feature functionality.

In the past, if we wanted to give our customers/users early access to a feature, we’d spin up new infrastucture for the new beta site and although new tooling has made it easy to replicate an environment, we’re now incurring the cost of a separate set of infrastucture. With the feature flag approach, we would add our users to a whitelist activating the feature for them inside our existing infrastructure, reducing the number of environments we manage. This also has the added benefit of being able to perform A/B, or blue/green, to test a new workflow and measure the result without affectingyour entire customer base and then turn off, or on, the new flow based on the results.

And when you’re ready to roll your new feature to production, but are worried your new feature and current infrastrcture might not support a full production rollout, you can utilize the same functionality used during A/B testing and slowly roll the new feature out to all your customers.

One of the great success stories I’ve heard of using feature flags is Facebook’s messaging feature. Long before it was exposed to the public, Facebook was testing its messaging functionality evertyime a user visited it’s page. It would take a small subset of your friends, send them a message, and then expect to get a response from a percentage of the messages sent. This allowed them to gain feedback on how the messaging system was performing, as well as, identify weak points in the design. Then when they decided to launch the messaging functionality, they flipped a flag and users were immediately able to use the new feature.

Dangers

With all the benefits we gain from utilizing feature flags, comes great responsibility. There are a couple of notable dangers you should be aware of when implementing feature flags. As teams begin to utilize feature flags, the amount of technical debt in the application starts to grow. This technical debt is in the form of unused code, or code bloat. To combat this issue, when your new feature gains 100% usage and is deemed stable, the next iteration should remove the unused code.

One way I’ve seen suggested is to create a parallel branch with the code being replaced removed. The idea is the developers have intimate knowledge of the code being replaced and is the ideal time to remove it. When the code reaches 100% usage then you merge the “clean” code into the main branch. This does require you to maintain 2 code bases until the feature has reached entropy.

Another danger is feature flag reuse. You should never reuse a previously used feature flag because you could accidentally turn on a previously deprecated feature, or poor performing feature, which may have not been properly removed. Knight Capital, a trading firm managing billions of trades worth billions of dollars daily. They were in the process of replacing the existing engine, deploying the new code manually to their servers (8 in all) over a 5 day period. Unfortunately, one of the servers was missed and still contained the old code. When the flag was triggered, 7 of the servers functioned as expected and the 8th server began running the deprecated code. In about 45 minutes, the company incurred a $465 million loss and with only $365 million in cash and equivalents, became bankrupt. Read Knightmare: A DevOps Cautionary Tale to learn more about the incident and some lessons learned.

The reason I mention the Knight Capital incident is not to scare you away from feature flags, but to bring to light the need and importance of managing our technical debt in a timely manner. It is important to also understand any code change could experience this type of incident if we don’t manage our technical debt.

Some of the feature flagging services, like LaunchDarkly, provide the ability to show when a flag is ready for removal by showing when all users are receiving the same flag value.

Build vs. Buy

While building a simple feature flag system isn’t overly complex, you need to ask yourself what business do you want to be in and estimate the cost accordingly. Maintaing your own system means as business changes, upgrades may be necessary and accounted for with new initiatives and development language support.

I would recommend starting out with a SaaS provider and down the road reevaluate this decision if the need arises for additional functionality. You may be able to partner with the SasS provider to get this new functioanlity, saving the costs of an internal system.

Check out Feature Flags - Part 2 to learn how to implement feature flags.