Onfido Tech Blog: Building An Architecture That Scales - Part I

To build a large and complicated structure, you have to work together and think ahead

Hello, and welcome to the first in our new series of Onfido blog posts brought to you by members of the Onfido technology team. We’re always looking for ways to make ourselves more open and available to the public and our customers, and you can expect more technology blogs like this one regularly from now on.

The first of our tech blog posts is by our developer Ahmed on building a technical architecture that scales – a vital consideration when building successful web applications.

This is the introduction to a series of in-depth articles about scaling Rails architecture for medium and large scale applications. It is important to emphasise at the outset that the term ‘scale’ does not always refer to scaling only for high performance – for example, allowing processing of 1M+ requests/day or a high concurrency of users per second (e.g. 200,000 or more). Rather, this series is meant to give advice on how to scale the design and architecture of your application to allow greater number of team members to simultaneously and collaboratively work on the code base, enabling a significant capability for adding features and improving code re-usability, as well as improving the detection and correction of bugs and decreasing the time spent learning the code base by newcomers. 

As a team, we have found that previous discussions of scalability from this perspective are rare, often scattered in approach and frequently summarised in just a few patterns or strategies. As a result, it is not at all easy to reach a conclusion on how best to construct a scalable architecture of this nature.

Our main focus and examples will all be based in Rails, so we recommend a basic level of knowledge about the framework in order to get the most out of this series. However, a lot of practices that we will outline and follow across this series can be generalized and used in many other frameworks apart from Rails.

Rails is a very powerful full stack framework, widely used by many developers across the globe, and is well known for its reasonable learning curve and the high productivity of its developers, backed by the fastness and easiness of writing Ruby code. Just a few lines in Ruby can easily substitute for far more complex and lengthy code in many other frameworks. However, this productive capacity is a double edged weapon, as sometimes developers conflate productivity with coding as fast as possible. Productivity does not simply mean developing fast, it also means clearing boundaries and obstacles out of your way, thus freeing yourself from drag in your development process that prevents you from focusing on the piece of business logic you are developing.

It may be suggested that writing scalable code can impact upon short-term productivity because you will have to think deeply and carefully before you make any changes– that and you may want to start by writing complex code just because it follows a good design pattern you read in “Head First”. We will show you in this series that this impression is totally mistaken and that one of the best reasons for developing code that scales is the increased productivity it yields.

This mistake is particularly common in growing startups: the fear that talking about scaling in the way I have outlined might be premature and cost time and money. There is an element of truth in that objection – but developers should be aware of that, even if they don’t have to build fully scalable code from the beginning, they have to think about how the code will have to scale tomorrow when building the code blocks of today. It’s a balance – of course we don’t encourage over-engineering!

I hope this blog post has outlined what my series will attempt to accomplish. Some of the topics that will be discussed in this series are: how to scale your controllers, models, views, helpers, assets and database; when to favour a design pattern over the other; what not to do, when to scale and much more.

Stay tuned and and expect the first of our in depth articles in the New Year.

Author image
Sharkasy "Shakes" is a senior software engineer at Onfido and his ambition is to drive technological innovation in his field. Shakes loves running, writing, watching cartoons and playing with kids.