ConfigCat Stack ep.1 - Overview, POC

In the next few articles I would like to introduce the core infrastructure of ConfigCat, how it has evolved over time and what challenges we faced through its validation phases.

ConfigCat is a startup by definition, founded and completely bootstrapped by 6 engineers. As a startup usually "starts up" with limited or zero revenue with lots of initial expenses, we decided to pay close attention to how much we spend on our infrastructure.

This decision and vision accompanied us through the years and helped us maintain really fair prices for our customers and a real "forever free" plan too.

The POC

Our proof of concept application was a real swiss army knife, a single ASP.NET Core MVC web application. It included our homepage, the documentation, the management dashboard for feature flags, and it also served the so called "config.json" files which became the corner stone of ConfigCat.

Our decision was a MySQL database backed by an ASP.NET Core MVC web application running on cheap but reliable virtual private servers.

homepage

Virtual private servers

When the time came to choose between hosting solutions, we analysed some big cloud providers like Azure, AWS, GCE and also some smaller competitors like Linode and Digitalocean price and feature-wise. In the end, we chose the smallest VPS at Linode.

5 USD/month for running our full infrastructure... sounds pretty cost-efficient (and crazy at the same time) for a startup, huh?

MySQL

We wanted a cheap, stable and fast relational database which can handle big loads easily.
MSSQL and Oracle were too expensive for our budget, but fortunately we all had some experience with using MySQL, so we ended up with a single node running a MySql database along with our web application together. That was our one and only server. Oh sweet memories... I mean real memories, not RAMs :). As we had a really lightweight database structure powered up by Entity Framework Core through the Pomelo.EntityFrameworkCore.MySql connector, which handles a MySql database easily, we didn't even had to look into our database for a long time.

ASP.NET Core MVC

As we all had a strong .NET background and as .NET Core can easily run on a simple Linux VPS, the fastest solution for a POC was an ASP.NET Core MVC web application.
When the time came and we needed feature flags in our web application, there were no questions asked, we all knew that we'll use ConfigCat within itself. This was also a great playground for testing our .NET SDK. As we were the first real customers, we got a really good impression about what it's like to integrate it into an application.
Using ConfigCat as a customer and not just as a developer helps us a lot to locate the pain-points in terms of usage and UX.

dashboard

Nginx

ConfigCat's corner stone is the feature flag evaluation for our Customers. We tried to come up with a cost-efficient feature flag evaluation solution so we decided to put our logic into the SDKs.
This decision allowed us to only serve a simple "config.json" file - containing all information to evaluate a feature flag on client side - from our servers instead of executing a bunch of evaluation logic that would result in much more load on the server side. Of course this decision had some downsides too, we had to make sure that all of our SDKs are evaluating the feature flags in the exact same way. This is why we have 1500+ test cases in all of our SDKs.

While searching for a lightweight and easily configurable web server for Linux, Nginx proved to be the best for our use case. It could easily serve the "config.json" files and our ASP.NET Core web application efficiently through HTTPS.

Deployment, CI/CD

Especially during a POC phase the iteration cycles are really short, we used to deploy at least 2-3 times a day to production. There was no question in using at least 2 environments (test and production) and a CI/CD pipeline so we don't have to manually release our application.
We decided to use Azure Devops for our web application build and release pipelines and Travis CI for our Open source SDKs to provide as much transparency as possible.

Error logging

We had great experiences with Sentry which is an error logging and application monitoring platform. It provides great integration with most of the programming languages and has a really great dashboard to track exceptions. We use Sentry in all of our components since the very beginning.
It has a free tier which suits a startup very well. The limitations in this free tier also works as a pressure on us to keep the exception and error count as low as possible.

Sentry integration works the same way as ConfigCat works, you have to include an SDK in your application. Sentry showed us a great example to provide pre-built code snippets for an easy first integration with a sense of achievement.

Monitoring

As we decided to go with an "everything is on one virtual server" infrastructure, application monitoring was a must have for the POC phase. Our applications were already reporting exceptions/errors to Sentry so our main goal for the POC phase was to also monitor that the services are up and running.
We used Pingdom for simple health checks on our core endpoints. It gave us insights about the availabaility and the performance of the most critical endpoints.

Final thoughts

For a POC phase the hardest part is to maintain a fine line between the must haves and the fine to haves in terms of components/infrastructural elements/features.
We had to consider every aspect of our application and infrustructural part whether it was worth the price and time or not in this phase.
Error/application monitoring was a must have next to an easily configurable and cheap infrastructure.
Using ConfigCat's feature flags in the ConfigCat web application itself helped us a lot to get a sense about the use-and-feel of ConfigCat and to detect the pain points during this early phase.

This one-server approach proved to be a good, reliable and really cost-efficient solution for a POC phase, and served ConfigCat for more than a year, before we upgraded its components and infrastructural elements.