Onfido Tech Blog: Building An Architecture That Scales – Part II, The How, Why and When

In our first post introducing our Building an Architecture that Scales series we talked about the how to scale a rails application from a different perspective – working scalably. By scaling, we meant allowing a large team to work in parallel, in the same modules in the code base, while easily detecting failures, working flexibly, and re-using code as needed. Most importantly, we meant increasing the overall productivity and efficiency of the team.

Today we are going to start the substantive part of the series and dig deeper into some ideas and methods we use at Onfido to make it easy to scale your rails application not horizontally nor vertically but diagonally! We will call this scaling “Diagonal Scaling” to emphasise the difference between the other two well known types of scaling (Horizontal & Vertical).

Diagonal Scaling:

Diagonal scaling is a concept we have introduced which help achieve 4 basic benefits:

1- Productivity.

2- Code Reusability.

3- Efficiency.

4- Code Self-Explanation.

We will refer to these outcomes with the acronym PRES.

We see the implementation of diagonal scaling and PRES as the responsibility of the whole team, not just developers. Diagonal scaling is not just about coding, it’s about the development environment, deployment strategy, planning and much more.

Is ‘diagonal scaling’ really ‘scaling’?:

Some people might think that the diagonal scaling as we describe it may not fit the traditional description of scaling; there are certain discrepancies between our definition and the definition of scaling used most often. Nonetheless, we will prove with specific examples in this series how organizing your code and making a better architecture can sometimes even help improve the performance of the overall application.

If our thinking is correct, our methodology improve performance far more cheaply and efficiently than by increasing resources or adding more nodes to the system. In fact, diagonal scaling can make you get rid of some of unnecessary nodes you previously added, lowering the amount of memory your application is consuming and making you use fewer cores – costing you less in the long run.

The ‘How’

So let’s start and talk about the specifics – how to diagonally scale a rails application.

Rails is well-known for its MVC model. Almost all people know what MVC stands for but fewer actually know the exact responsibilities of each letter. While we can actually go into a debate what every component of the Model, View and Controller should do, let’s mention the most agreed upon roles:

Model: The model represents the information and the data from the datastore (not necessarily a database). It should be as independent from the datastore as possible (although lots of people are blaming rails for coupling the model layer with ActiveRecord)

View: The view is the presentation layer for your application responsible for rendering your data representations.

Controller: The controller merely connects the model with the view (although some people prefer to allow some minor logic in the controller too)

As you see, we can’t say anything in this vast world of ideas, discussions and thoughts without ‘but’ or ‘although’ so if there is anything certain in our world, then it is the uncertainty of everything and that the only static fact is that we have to change and cope with our requirements.

To make a long story short, you have to adapt and evolve as needed, using data and knowledge of the system to drive decisions.

Now, let’s mention some of the anti-diagonal scaling mistakes that can decrease the diagonal scalability of your application in any of the MVC parts and hence make you lose one or multiple PRES factors. We’ll present a series of problems and ask you all to post your solutions.

Note: These are just quick basic examples and we will go into deeper look inside each of these components in a later blog posts.

Problem #1

problem_1

Problem #2

problem_2 (1)

Problem #3

problem_3

Problem #4

Have you had 100+ models in your app with tangled business logic and data objects making it very hard to reuse, maintain or test?

Had this problem before and managed to find a working solution for your case? If so, please share it with us.

Why

Well, don’t you need your code to be organized, reusable and self-explanatory, and your team to be productive and efficient? Your overall architecture to be scalable and extendable? I believe these are the dream goals of every team – never achieved or will be achieved 100%. The goal of the diagonal scaling is to put these things into consideration from the very beginning and to design some sort of framework to let you work on all these aspects throughout the development process.

When

From the very beginning to the very end!

Diagonal scaling with its aspects should be a metric discussed and evaluated in every meeting and, as we mentioned in our previous post, even if you don’t have to build fully scalable code from the beginning, you have to think about how the code will have to scale tomorrow when building the code blocks of today.

The earliest your start, the less you suffer from accrued technical debt later on. To cut it short: think about scaling from day one, and apply when possible.

In the next 2 posts we will discuss how to diagonally scale the models layer and what we have explored about the best way to use them and where to put the business logic. We will also explore different patterns including policies, services, CQRS and much more!

Stay tuned!

Author image
Sharkasy "Shakes" is a senior software engineer at Onfido and his ambition is to drive technological innovation in his field. Shakes loves running, writing, watching cartoons and playing with kids.
top