Rings of Validation

Spotnana’s Travel-as-a-Service Platform is used by a wide range of personas including travelers, travel arrangers, company administrators, travel agents, and travel management company (TMC) administrators. 

When new features are ready, we have to ensure that:

1. We can test features internally to catch bugs and improve our user experience before release.
2. We have a mechanism to gather feedback across all functions inside Spotnana before deploying features to our customers.
3. Our teams are empowered to demo features that may not be deployed to all our customers yet.

The solution

Rings of Validation is a method of releasing new features to staggered user groups before the feature is made available to everyone. This process ensures features are tested by small, focused groups before a larger, more diverse group tries them out. Here is how it works at Spotnana: 

Spotnana users are grouped under these categories:

• Ring 0: The Spotnana development team (R & D team)
• Ring 1: All Ring 0 users + Spotnana’s Travel Operations team
• Ring 2: All Ring 1 users + Remaining Spotnana employees
• Ring 3: All Ring 2 users + All Spotnana users

When a new platform feature is developed, it is made available to the whole development team at Spotnana after testing. The R & D team is the first set of people to use the feature and provide feedback.

Once all the feedback of Ring 0 users has been incorporated, the feature  is then released to Ring 1, which includes travel agents in our Travel Operations team. Spotnana Travel Operations team members are true travel domain experts who have experienced the rarest travel scenarios, workflows, and information. Their industry experience helps us spot any gaps  and gives our developers an opportunity to improve features before a broader release.

Once our Travel Ops team is satisfied,  a feature is made available to Ring 2, which includes all Spotnana employees across the globe. The first-hand experience of employees using the feature gives us the opportunity to do a true  beta test. As the number of users accessing the feature increases, we get a stronger signal on the  usability and versatility of our feature implementation. 

Once the feature is performing well with internal users, it’s ready for deployment to all Spotnana customers. At this point, we have a high degree of confidence that the feature will delight our customers, and we release the feature to Ring 3, which  consists of all authenticated Spotnana platform users.

The implementation

Every new Spotnana feature is associated with a feature flag which is enabled or disabled for users in different Rings. Features with disabled feature flags won’t function on Spotnana’s Online Booking Tool (OBT) or be available via our Spotnana Platform API. 

We use Optimizely for managing feature flags across our frontend and backend applications. Optimizely provides us with:

• The React SDK for feature flags on the frontend.
• The Java SDK for feature flags on the backend.
• Multiple environments to manage flags differently per environment.
• An easy-to-use, rich interface for creating flags, building Rings with multiple conditions, and associating features to Rings for staggered release.

One of the complications of using Optimizely is that it is blocked by many web browser ad blockers. We discovered that our frontend application can’t download feature flag data from Optimizely if an ad blocker is active.

We introduced  a proxy server between our frontend and Optimizely to solve for ad blocker use by our customers. The proxy server downloads feature flag data from Optimizely periodically and stores it locally. Our frontend then communicates with the proxy server to download this data, providing users with a consistent user experience.

A closer look

A new feature that reaches production deployment on the app is initially enabled for Ring 0. Optimizely configuration for this new feature looks like this:

When the feature is ready to go live for everyone including all Spotnana customers, the Optimizely configuration looks like this:

To further illustrate  how Rings are configured on Optimizely, here is an example of Ring 2, which includes all Spotters. Notice that Rings employ user email ids or email-based patterns to define membership.

As we implemented Rings of Validation, we quickly realized it was important to let Spotnana employees move across Rings at will. Let’s say one of our sales people is demoing Spotnana to a prospective customer, who asks about a feature that is coming soon. The ability to move across rings allows our sales executive to change Rings to show the requested feature while also demoing our currently released set of features. 

Here is the workflow when a user jumps from one Ring to another:

When a user jumps from a larger Ring (more users, fewer features) to a smaller Ring (fewer users, more features) we add the user to all Rings bigger than the smaller Ring because the mature features are already present there. When a user jumps from a smaller Ring to a larger Ring we need to ensure that the user is not part of any intermediate Ring, or the user will still be able to access features which haven’t been released to the user’s active Ring. 

Here is how a user jumps from one Ring to another from within the application:

User is currently in Ring 3 (Everyone)

User clicks the “change” link to jump to Ring 1 where a feature is currently accessible.

The user clicks on “Travel Operations” Ring (Ring 1) and waits for a couple of minutes for the switch to take effect.

All features undergoing validation by our Travel Operations team (Ring 1) are now available to this user.

With our Rings of Validation strategy, we solved all the three challenges we discussed at the start of this blog. All Spotnana users are able to access the right features, we get feedback from across the company as a feature matures, and team members are empowered to switch between Rings as needed.

What’s next

New use cases have emerged with the potential to increase the productivity of our Rings of Validation feature. Here are some of the ideas our team is currently working on:

1. Providing our customers with the ability to preview features early in the development process and prepare for when the features are ready.
2. Allowing our channel partners to roll out features to their clients in phases and manage their onboarding effectively.
3. Implementing a framework for A/B testing multiple  versions of a feature to determine which design is most beneficial.
4. Using Rings of Validation data for insights into feature progression from development to rollout, allowing us to improve our development process, analyze our implementation process, and more effectively iterate on new features.