Welcome, Guest: Join Nairaland / LOGIN! / Trending / Recent / New
Stats: 2,778,940 members, 6,622,499 topics. Date: Wednesday, 08 December 2021 at 05:29 PM

Backend Software Engineers, Let's Talk About System Design - Programming - Nairaland

Nairaland Forum / Science/Technology / Programming / Backend Software Engineers, Let's Talk About System Design (9404 Views)

Why Are Backend Developers Paid More Than Frontend Developers? / Enyata Wants To Build A Sustainable Community For African Software Engineers / 19 Tips For Programmers/software Engineers In 2019 (2) (3) (4)

(1) (2) (3) (4) (Reply) (Go Down)

Backend Software Engineers, Let's Talk About System Design by SilverG33k(m): 9:26pm On Dec 05, 2020
Honestly, let's just catch cruise here.

Now here's a quick question, you're to build nairaland backend, how would you do that?

What technologies would you use, how would you handle scalability issues. Note than, the expected traffic for this system is 1k req/s just for fun tho

No yabbing

7 Likes 3 Shares

Re: Backend Software Engineers, Let's Talk About System Design by talk2hb1(m): 12:50am On Dec 06, 2020
Tooh,
Waiting for Experience Bosses to Come Tutor us on this.

3 Likes 1 Share

Re: Backend Software Engineers, Let's Talk About System Design by ULOPVQ: 4:51am On Dec 06, 2020
Re: Backend Software Engineers, Let's Talk About System Design by DC49: 6:40am On Dec 06, 2020
SilverG33k:
Honestly, let's just catch cruise here.

Now here's a quick question, you're to build nairaland backend, how would you do that?

What technologies would you use, how would you handle scalability issues. Note than, the expected traffic for this system is 1k req/s just for fun tho

No yabbing

In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately.
Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling support to handle traffic surges.
I'd utilize a caching system like Memcached to reduce load on the back-end server, store the user generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare.
This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it.

86 Likes 12 Shares

Re: Backend Software Engineers, Let's Talk About System Design by DC49: 6:41am On Dec 06, 2020
DC49:


In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately.
Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling support to handle traffic surges.
I'd utilize a caching system like Memcached to reduce load on the back-end server, store the user generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare.
This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it.
Re: Backend Software Engineers, Let's Talk About System Design by Karleb(m): 7:20am On Dec 06, 2020
Decoupled architecture. Noted!

@Mukina2 This deserves front page

1 Like

Re: Backend Software Engineers, Let's Talk About System Design by SilverG33k(m): 5:16pm On Dec 06, 2020
DC49:


In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately.
Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling support to handle traffic surges.
I'd utilize a caching system like Memcached to reduce load on the back-end server, store the user generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare.
This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it.

I love this, interesting, you would definitely be needing devops. Great thinking really

1 Like

Re: Backend Software Engineers, Let's Talk About System Design by SilverG33k(m): 5:35pm On Dec 06, 2020
What if you we try to reduce cost by

Using just AWS, or a good old server, right, host the database on a separate server, and the other contents(images, videos, profile pictures), serve the media files with cloudflare or AWS cloudfront, then in cases where the homepage gets accessed by thousands of users, a Memcached would work, I prefer to use Redis however, both does the same thing, so instead of reading the popular topics and posts off the database, it reads from the cache, which reduces load on the database server.

For analytics, i always prefer and suggest to use a nosql db like mongo db, cassandra or faunadb, anything flexible actually, obviously our core database is going to be a relational database e.g mysql, oracle so we don't want to put too much load on it, we just need to put in the critical data there, for scalability, so this means that all of those number of views, daily active users, page views, e.t.c would go to a nosql database, users logs as well, requests and responses(if you're keeping track) all of these analytical data should make use of firebase database or any nosql db you want

Am i missing anything, yes, anytime i remember, I'd share, this is how i am going to build mine, its not a general rule that everyone must follow sha, let us understand that

18 Likes 1 Share

Re: Backend Software Engineers, Let's Talk About System Design by SilverG33k(m): 5:38pm On Dec 06, 2020
talk2hb1:
Tooh,
Waiting for Experience Bosses to Come Tutor us on this.

lmao look at this boss oh, chai

Okay, Sir... How would you handle 10 backend devs working with 4 different languages
Re: Backend Software Engineers, Let's Talk About System Design by talk2hb1(m): 5:42pm On Dec 06, 2020
SilverG33k:


lmao look at this boss oh, chai

Okay, Sir... How would you handle 10 backend devs working with 4 different languages
This sha want to intimidate me 10backend Dev, 4 different languages.
I go dey form big boss ni ooo grin

2 Likes

Re: Backend Software Engineers, Let's Talk About System Design by tensazangetsu20(m): 5:43pm On Dec 06, 2020
DC49:


In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately.
Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling support to handle traffic surges.
I'd utilize a caching system like Memcached to reduce load on the back-end server, store the user generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare.
This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it.
I am building something very similar. The frontend is React. The backend is WordPress.
Re: Backend Software Engineers, Let's Talk About System Design by Fash20: 3:23am On Dec 07, 2020
Good... I love to learn from booses... I haven't gotten the chance to work with or learn from one directly but I love what I have read here so far
Re: Backend Software Engineers, Let's Talk About System Design by cixak95211: 6:58am On Dec 07, 2020
SilverG33k:
Honestly, let's just catch cruise here.

Now here's a quick question, you're to build nairaland backend, how would you do that?

What technologies would you use, how would you handle scalability issues. Note than, the expected traffic for this system is 1k req/s just for fun tho

No yabbing

NL say she has 2.5M members, if that is anything to go by then indeed it has to be a microservice. There must be very bad guys amongst these members, so an API Gateway is a must for the backend. I'll likely go with Kubernetes so it handles upscaling and downscaling for me without having me loose sleep. NGINX is very much ok as an LB for both frontend and backend, i prefer a round-robin as against weighted since the K8 engine will spin up equally weighted infra. For the DB, i'll likely go with Mongodb since it will allow me run a search engine without the need for Algolia . . oh well Algolia can come later , but at least I can do database sharding right out of the box. You dont expect me to dump 2.5M parent records which will span into hundreds of millions, imagine one user has 100 posts, which is way likely to be more each week. Fact is i cannot dump those in one single DB, no maatter hwo powerful it is, It has to be redundant. Redis is needed for caching frontpage topics, so i dont have to serve it all the time, for each request. I could cache with NGinx too but i prefer redis. LRU is powerful.
Choice of server infra . . a simple linux image will do, ssd based and most importantly, all servers, FE, BE and DB must be encrypted at rest. I'll use a KMS for that. I'll leave one bastion host available for maintenance work and make sure all servers only communicate via a VPC.

16 Likes 1 Share

Re: Backend Software Engineers, Let's Talk About System Design by SilverG33k(m): 8:02am On Dec 07, 2020
cixak95211:


NL say she has 2.5M members, if that is anything to go by then indeed it has to be a microservice. There must be very bad guys amongst these members, so an API Gateway is a must for the backend. I'll likely go with Kubernetes so it handles upscaling and downscaling for me without having me loose sleep. NGINX is very much ok as an LB for both frontend and backend, i prefer a round-robin as against weighted since the K8 engine will spin up equally weighted infra. For the DB, i'll likely go with Mongodb since it will allow me run a search engine without the need for Algolia . . oh well Algolia can come later , but at least I can do database sharding right out of the box. You dont expect me to dump 2.5M parent records which will span into hundreds of millions, imagine one user has 100 posts, which is way likely to be more each week. Fact is i cannot dump those in one single DB, no maatter hwo powerful it is, It has to be redundant. Redis is needed for caching frontpage topics, so i dont have to serve it all the time, for each request. I could cache with NGinx too but i prefer redis. LRU is powerful.
Choice of server infra . . a simple linux image will do, ssd based and most importantly, all servers, FE, BE and DB must be encrypted at rest. I'll use a KMS for that. I'll leave one bastion host available for maintenance work and make sure all servers only communicate via a VPC.

Database sharding how did I miss that... Excellent point there, very necessary, why would the DB be encrypted tho? Or what part of the DB needs to be encrypted, please share, because I am just considering the speed and complexity

1 Like

Re: Backend Software Engineers, Let's Talk About System Design by Karleb(m): 8:10am On Dec 07, 2020
FE, BE, VPC, KMS
What do they mean?
Re: Backend Software Engineers, Let's Talk About System Design by Amoto05(m): 12:34pm On Dec 07, 2020
k
Re: Backend Software Engineers, Let's Talk About System Design by Dottore: 12:35pm On Dec 07, 2020
Ok
Re: Backend Software Engineers, Let's Talk About System Design by Jahzrockballer(m): 12:36pm On Dec 07, 2020
Just curious, why is no one talking about ruby on rails Just saying though

2 Likes

Re: Backend Software Engineers, Let's Talk About System Design by Lovelies1: 12:36pm On Dec 07, 2020
Re: Backend Software Engineers, Let's Talk About System Design by cinoedhunter: 12:37pm On Dec 07, 2020
Please what language are you guys speaking here sad undecided cryayam not understanding

18 Likes 1 Share

Re: Backend Software Engineers, Let's Talk About System Design by Olalekank(m): 12:38pm On Dec 07, 2020
.
Re: Backend Software Engineers, Let's Talk About System Design by JoeEeL(m): 12:40pm On Dec 07, 2020
Good. These are the kind of threads that, once in a while, should be popping up on front page and not ridiculously silly-ass posts like Regina daniels prevents Ned's 7th wife, Toke Makinwa stuns with plastic-ass ...

Backend, as strictly relates to Web architecture, NodeJS is my go-to. Of course with Express as the de-facto server framework. Request / sec, as far as node is concerned is nearly 2x, response time is as much as 35% faster. Also it abstracts quirks, inconsistencies and is the perfect syntactic sugar.

Looking to annex its powers to FE, one has to be thinking in terms of React -- which leads you to the promised land -- THE MERN stack

React helps you define the UI once and for all. Then, when the state of the app changes, the UI is rebuilt to react to the change -- acting on its name.
Small manageable components enable to build fairly complex apps but the sweetness of this is, you're still running on vanilla JS anyways. DOM node-hunting is kept at barest minimum; an avoidance of pain you might argue but darn! I was good at nifty DOM selection in my jQuery heydays. Anyways, all you do is maintain the state of your app (with a regular old JavaScript object) and the rest just naturally follows.

17 Likes

Re: Backend Software Engineers, Let's Talk About System Design by cixak95212: 12:41pm On Dec 07, 2020
SilverG33k:


Database sharding how did I miss that... Excellent point there, very necessary, why would the DB be encrypted tho? Or what part of the DB needs to be encrypted, please share, because I am just considering the speed and complexity

2 Likes

Re: Backend Software Engineers, Let's Talk About System Design by ipressplay: 12:44pm On Dec 07, 2020
DC49:


In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately.
Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling support to handle traffic surges.
I'd utilize a caching system like Memcached to reduce load on the back-end server, store the user generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare.
This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it.

This like you said is very expensive.

Why not build with Laravel, its scalable and enjoys massive support.

Hosting with VPS or dedicated server. Better still use AWS for hosting and different buckets for different files.

My thoughts.

7 Likes

Re: Backend Software Engineers, Let's Talk About System Design by SenorFax(m): 12:46pm On Dec 07, 2020
Please guys, start a new one for learners like us. I love the languages you are speaking, but I'm a newbie to software techs

4 Likes

Re: Backend Software Engineers, Let's Talk About System Design by ElRapido: 12:47pm On Dec 07, 2020
I dey laugh here
Re: Backend Software Engineers, Let's Talk About System Design by kennyflames(m): 12:49pm On Dec 07, 2020
k
Re: Backend Software Engineers, Let's Talk About System Design by Opexzy: 12:49pm On Dec 07, 2020
I will come back to give my opinion...
Re: Backend Software Engineers, Let's Talk About System Design by benedictuyi(m): 12:52pm On Dec 07, 2020
Following
Re: Backend Software Engineers, Let's Talk About System Design by WiszyFraud: 12:52pm On Dec 07, 2020
cool
*Tightens seatbelt*

Time to learn

1 Like 1 Share

(1) (2) (3) (4) (Reply)

Developing An Operating System - What Are The Prerequisites / Gov Ambode Has Plans For One Million Lucky Lagosians / Python Programmers Lets Meet Here!!!

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2021 Oluwaseun Osewa. All rights reserved. See How To Advertise. 167
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.