The Backend Engineering Show with Hussein Nasser: Recent Episodes
Hussein Nasser
Welcome to the Backend Engineering Show podcast with your host Hussein Nasser. If you like software engineering you’ve come to the right place. I discuss all sorts of software engineering technologies and news with specific focus on the backend. All opinions are my own.
Most of my content in the podcast is an audio version of videos I post on my youtube channel here http://www.youtube.com/c/HusseinNasser-software-engineering
Support my work on PayPal https://bit.ly/33ENps4
🧑🏫 Courses I Teach https://husseinnasser.com/courses Support this podcast: https://anchor.fm/hnasr/support
Fundamentals of Database Engineering udemy course https://databases.winMySQL has been having bumpy journey since 2018 with the release of the version 8.0. Critical crashes that made to the final product, significant performance regressions, and tons of stability and bugs issues. In this video I explore what happened to MySql, are these issues getting fixed? And what is the current state of MySQL at the end of 2024. 0:00 Intro 2:00 MySQL 8.0 vs 5.7 Performance11:00 Critical Crash in 8.0.38, 8.4.1 and 9.0.0 15:40 Is 8.4 better than 8.0.36?16:30 More Features = More Bugs22:30 Summary and my thoughts resources https://x.com/MarkCallaghanDB/status/1786428909376164263https://www.percona.com/blog/do-not-upgrade-to-any-version-of-mysql-after-8-0-37/http://smalldatum.blogspot.com/2024/09/mysql-innodb-vs-sysbench-on-large-server.htmlhttps://www.percona.com/blog/mysql-8-0-vs-5-7-are-the-newer-versions-more-problematic/
Fundamentals of Operating Systems Course https://oscourse.winIn this video I use strace a performance tool that measures how many system calls does a process makes. We compare a simple task of reading from a file, and we run the program in different runtimes, namely nodejs, buns , python and native C. We discuss the cost of kernel mode switches, system calls and pe0:00 Intro5:00 Code Explanation6:30 Python9:30 NodeJS12:30 BunJS13:12 C16:00 Summary
Fundamentals of Operating Systems Course https://os.husseinnasser.comWhen do you use threads?I would say in scenarios where the task is either 1) IO blocking task2) CPU heavy3) Large volume of small tasksIn any of the cases above, it is favorable to offload the task to a thread.1) IO blocking taskWhen you read from or write to disk, depending on how you do it and the kernel interface you used, the write might be blocking. This means the process that executes the IO will not be allowed to execute any more code until the write/read completes.That is why you see most logging operations are done on a secondary thread (like libuv that Node uses) this way the thread is blocked but the main process/thread can resume its work.If you can do file reads/writes asynchronously with say io_uring then you technically don't need threading. Now notice how I said file IO because it is different than socket IO which is always done asynchronously with epoll/select etc.2) CPU heavyThe second use case is when the task requires lots of CPU time, which then starves/blocks the rest of the process from doing its normal job. So offloading that task to a thread so that it runs on a different core can allow the main process to continue running on its the original core.3) Large volume of small tasksThe third use case is when you have large amount of small tasks and single process can't deliver as much throughput. An example would be accepting connections, a single process can only accept connections so fast, to increase the throughput in case where you have massive amount of clients connecting, you would spin multiple threads to accept those connections and of course read and process requests. Perhaps you would also enable port reuse so that you avoid accept mutex locking.Keep in mind threads come with challenges and problems so when it is not required.0:00 Intro1:40 What are threads?7:10 IO blocking Tasks17:30 CPU Intensive Tasks22:00 Large volume of small tasks
I am fascinated by how timeouts affect backend and frontend programming.
When a party is waiting on something you can place a timeout to break the wait. This is useful for freeing resources to more critical processes, detecting slow operations and even avoiding DOS attacks.
Contrary to common beliefs, timeouts are not exclusive to request processing, they can be applied to other parts of the frontend-backend communications. Let us explore this briefly.
0:00 Intro
2:30 Connection Timeout
5:00 Request Read timeout
10:00 Wait Timeout
12:00 Usage Timeout
14:00 Response Timeout
16:00 Canceling a request
19:50 Proxies and timeouts
Learn more about database and OS internals, check out my courses
Fundamentals of database engineering https://databases.win
Fundamentals of operating systems https://oscourse.win
This new PostgreSQL 17 feature is game changer.
You see, postgres like most databases work with fixed size pages. Pretty much everything is in this format, indexes, table data, etc. Those pages are 8K in size, each page will have the rows, or index tuples and a fixed header. The pages are just bytes in files and they are read and cached in the buffer pool.
To read page 0, for example, you would call read on offset 0 for 8192 bytes, To read page 1 that is another read system call from offset 8193 for 8192, page 7 is offset 57,345 for 8192 and so on.
If table is 100 pages stored a file, to do a full table scan, we would be making 100 system calls, each system call had an overhead (I talk about all of that in my OS course).
The enhancement in Postgres 17 is to combine I/Os you can specify how much IO to combine, so technically while possible you can scan that entire table in one system call doesn’t mean its always a good idea of course and Ill talk about that.
This also seems to included a vectorized I/O, with preadv system call which takes an array of offsets and lengths for random reads.
The challenge will become how to not read too much, say I’m doing a seq scan to find something, I read page 0 and found it and quit I don’t need to read any more pages. With this feature I might read 10 pages in one I/O and pull all its content, put in shared buffers only to find my result in the first page (essentially wasting disk bandwidth, memory etc)
It is going to be interesting to balance this out.
Fundamentals of Operating Systems Course https://os.husseinnasser.comWhy Windows Kernel connects slower than Linux I explore the behavior of TCP/IP stack in Windows kernel when it receives a RST from the backend server especially when the host is available but the port we are trying to connect to is not. This behavior is exacerbated by having both IPv6 and IPv4 and if the happy eye ball protocol is in place where IPv6 is favorable. 0:00 Intro0:30 Fundamentals TCP/IP3:00 Unreachable Port Behavior6:00 Client Kernel Behavior (Linux vs Windows)11:40 Slow TCP Connect on Windows15:00 localhost, IPv6 and IPv420:00 Happy Eyeballs28:00 Registry keys to change the behavior31:00 Port Unreachable vs Host Unreachablehttps://daniel.haxx.se/blog/2024/08/14/slow-tcp-connect-on-windows/
In this episode of the backend engineering show I describe an interesting bug I ran into where the web server ran out of ephemeral ports causing the system to halt.
0:00 Intro
0:30 System architecture
2:20 The behavior of the bug
4:00 Backend Troubleshooting
7:00 The cause
15:30 Ephemeral ports on loopback
Fundamentals of Operating Systems Course https://os.husseinnasser.comLinux I/O expert and subsystem maintainer Jens Axboe has submitted all of the IO_uring feature updates ahead of the imminent Linux 6.10 merge window.In this video I explore this with a focus on what zerocopy. 0:00 Intro0:30 IO_uring gets faster 2:00 What is io_uring7:00 How Normal Copying Work12:00 How Zero Copy Works13:50 ZeroCopy and TLShttps://www.phoronix.com/news/Linux-6.10-IO_uringhttps://lore.kernel.org/io-uring/fef75ea0-11b4-4815-8c66-7b19555b279d@kernel.dk/?s=09
Fundamentals of Operating Systems Course https://oscourse.winLooks like fedora is compiling cpython with the -o3 flag, which does aggressive function inlining among other optimizations.This seems to improve python benchmarks performance by at most 1.16x at a cost of an extra 3MB in binary size (text segment). Although it does seem to slow down some benchmarks as well though not significantly.O1 - local register allocation, subexpression elimination O2 - Function inlining only small functionsO3 - Agressive inlining, SMID 0:00 Intro1:00 Fedora Linux gets Fast Python5:40 What is Compiling?9:00 Compiling with No Optimization12:10 Compiling with -O115:30 Compiling with -O220:00 Compiling with -O323:20 Showing NumbersBackend Troubleshooting Coursehttps://performance.husseinnasser.com
https://oscourse.win
Allegro improved their Kafka produce tail latency by over 80% when they switched from ext4 to xfs. What I enjoyed most about this article is the detailed analysis and tweaking the team made to ext4 before considering switching to xfs. This is a classic case of how a good tech blog looks like in my opinion.
0:00 Intro
0:30 Summary
2:35 How Kafka Works?
5:00 Producers Writes are Slow
7:10 Tracing Kafka Protocol
12:00 Tracing Kernel System Calls
16:00 Journaled File Systems
21:00 Improving ext4
26:00 Switching to XFS
Blog
https://blog.allegro.tech/2024/03/kafka-performance-analysis.html
Get my backend course https://backend.win
Google submitted a patch to Linux Kernel 6.8 to improve TCP performance by 40%, this is done via rearranging the tcp structures for better cpu cache lines, I explore this here.0:00 Intro0:30 Google improves Linux Kernel TCP by 40%1:40 How CPU Cache Line Works6:45 Reviewing the Google Patchhttps://www.phoronix.com/news/Linux-6.8-Networkinghttps://lore.kernel.org/netdev/20231129072756.3684495-1-lixiaoyan@google.com/Discovering Backend Bottlenecks: Unlocking Peak Performancehttps://performance.husseinnasser.com
0:00 Intro
2:00 File System Block vs Database Pages
4:00 Torn pages or partial page
7:40 How Oracle Solves torn pages
8:40 MySQL InnoDB Doublewrite buffer
10:45 Postgres Full page writes
Get my backend course https://backend.win
Cloudflare has announced they are opening sources Pingora as a networking framework! Big news, let us discuss
0:00 Intro
0:30 Reasons why Cloudflare built Pingora?
3:00 It is a framework!
7:30 What in Pingora?
11:50 Security in Pingora
13:45 Multi-threading in Pingora
21:00 Customization vs Configuration
25:00 Summary
https://blog.cloudflare.com/pingora-open-source/?utm_campaign=cf_blog&utm_content=20240228&utm_medium=organic_social&utm_source=twitter
https://backend.win
https://databases.win
I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows.
Everything goes to pages of fixed size and those pages are flushed to disk.
Each database define page size differently based on their workload, for example MongoDB default page size is 32KB, MySQL InnoDB is 16KB and PostgreSQL is 8KB.
The trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API.
In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills.
Let us get started.
In this video I explore the type of languages, compiled, garbage collected, interpreted, JIT and more.
I talk about default values and how PostgreSQL 14 got slower when a default parameter has changed. Mike's bloghttps://smalldatum.blogspot.com/2024/02/it-wasnt-performance-regression-in.html
Background writing is a process that writes dirty pages in shared buffer to the disk (well goes to the OS file cache then get flushed to disk by the OS) I go into this process in this video
Fragmentation is a very interesting topic to me, especially when it comes to memory. While virtually memory does solve external fragmentation (you can still allocate logically contiguous memory in non-contiguous physical memory) it does however introduce performance delays as we jump all over the physical memory to read what appears to us for example as contiguous array in virtual memory. You see, DDR RAM consists of banks, rows and columns. Each row has around 1024 columns and each column has 64 bits which makes a row around 8kib. The cost of accessing the RAM is the cost of “opening” a row and all its columns (around 50-100 ns) once the row is opened all the columns are opened and the 8 kib is cached in the row buffer in the RAM.The CPU can ask for an address and transfer 64 bytes at a time (called bursts) so if the CPU (or the MMU to be exact) asks for the next 64 bytes next to it, it comes at no cost because the entire row is cached in the RAM. However if the CPU sends a different address in a different row the old row must be closed and a new row should be opened taking an additional 50 ns hit. So spatial access of bytes ensures efficiency, So fragmentation does hurt performance if the data you are accessing are not contiguous in physical memory (of course it doesn’t matter if it is contiguous in virtual memory). This kind of remind me of the old days of HDD and how the disk needle physically travels across the disk to read one file which prompted the need of “defragmentation” , although RAM access (and SSD NAND for that matter) isn’t as bad. Moreover, virtual memory introduces internal fragmentation because of the use of fixed-size blocks (called pages and often 4kib in size), and those are mapped to frames in physical memory. So if you want to allocate a 32bit integer (4 bytes) you get a 4 kib worth of memory, leaving a whopping 4092 allocated for the process but unused, which cannot be used by the OS. These little pockets of memory can add up as many processes. Another reason developers should take care when allocating memory for efficiency.
In this video I explore the hidden costs of sending a request from the frontend to the backend
Heard
https://medium.com/@hnasr/the-journey-of-a-request-to-the-backend-c3de704de223
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)https://database.husseinnasser.com Why create Index blocks writesIn this video I explore how create index, why does it block writes and how create index concurrently work and allow writes.0:00 Intro1:28 How Create Index works4:45 Create Index blocking Writes5:00 Create Index Concurrently
HTTP/3 is getting popular in the cloud scene but before you migrate to HTTP/3 consider its cost. I explore it here.0:00 Intro HTTP/3 is getting popular3:40 HTTP/1.1 Cost5:18 HTTP/2 Cost6:30 HTTP/3 Costhttps://blog.apnic.net/2023/09/25/why-http-3-is-eating-the-world/
The Encrypted Client Hello or ECH is a new RFC that encrypts the TLS client hello to hide sensitive information like the SNI. In this video I go through pros and cons of this new rfc.0:00 Intro2:00 SNI4:00 Client Hello8:40 Encrypted Client Hello11:30 Inner Client Hello Encryption18:00 Client-Facing Outer SNI21:20 Decrypting Inner Client Hello23:30 Disadvantages 26:00 Censorship vs Privacy ECH https://blog.cloudflare.com/announcing-encrypted-client-hello/https://chromestatus.com/feature/6196703843581952
From the frontend through the kernel to the backend process
When we send a request to a backend most of us focus on the processing aspect of the request which is really just the last step.
There is so much more happening before a request is ready to be processed, most of this step happens in the Kernel. I break this into 6 steps, each step can theoretically be executed by a dedicated thread or process. Pretty much all backends, web servers, proxies, frameworks and even databases have to do all these steps and they all do choose to do it differently.
Grab my backend performance course https://performance.husseinnasser.com
0:00 Intro
3:50 What is a Request?
10:14 Step 1 - Accept
21:30 Step 2 - Read
29:30 Step 3 - Decrypt
34:00 Step 4 - Parse
40:36 Step 5 - Decode
43:14 Step 6 - Process
Medium article
https://medium.com/@hnasr/the-journey-of-a-request-to-the-backend-c3de704de223
In a wonderful blog, Kyle explores the pains he faced managing a Postgres instance for a startup he works for and how enabling partitioning sigintfically created wait events causing the backend and subsequently NGINX to through 500 errors.
We discuss this in this video/podcast
https://www.kylehailey.com/post/postgres-partition-pains-lockmanager-waits
WebTransport is a cutting-edge protocol framework designed to support multiplexed and secure transport over HTTP/2 and HTTP/3. It brings together the best of web and transport technologies, providing an all-in-one solution for real-time, bidirectional communication on the web.
fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells. However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells. Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync 0:00 Intro1:00 A Write doesn’t write2:00 File System Page Cache6:00 Fsync7:30 SSD Cache9:20 SSD ignores the flush9:30 15 Year old Firefox fsync bug12:30 What happens if SSD loses power15:00 What options does Postgres exposes?15:30 open_sync (O_SYNC)16:15 open_datasync (O_DSYNC)17:10 O_DIRECT19:00 fsync20:50 fdatasync 21:13 fsync = off23:30 Don’t make your API simple26:00 Database on metal?
ego is the main problem to a defective software product. the ego of the engineer or the tech lead seeps into the quality of the product. Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)https://backend.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)https://database.husseinnasser.com
In version 5.3, MongoDB introduced a feature called clustered collection which stores documents in the _id index as oppose to the hidden wiredTiger hidden index. This eliminates an entire b+tree seek for reads using the _id index and also removes the additional write to the hidden index speeding both reads and writes.
However like we know in software engineering, everything has a cost. This feature does come with a few that one must be aware of before using it. In this video I discuss the following
- How Original MongoDB Collections Work
- How Clustered Collections Work
- Benefits of Clustered Collections
- Limitations of Clustered Collections
Prime video engineering team has posted a blog detailing how they moved their live stream monitoring service from microservices to a monolith reducing their cost by 90%, let us discuss this0:00 Intro2:00 Overview10:35 Distributed System Overhead21:30 From Microservices to Monolith 29:00 Scaling the Monolith32:30 Takeawayshttps://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)https://backend.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)https://database.husseinnasser.comIn a row-store database engine, rows are stored in units called pages. Each page has a fixed header and contains multiple rows, with each row having a record header followed by its respective columns. When the database fetches a page and places it in the shared buffer pool, we gain access to all rows and columns within that page. So, the question arises: if we have all the columns readily available in memory, why would SELECT * be slow and costly? Is it really as slow as people claim it to be? And if so why is it so? In this post, we will explore these questions and more.0:00 Intro1:49 Database Page Layout5:00 How SELECT Works10:49 No Index-Only Scans18:00 Deserialization Cost21:00 Not All Columns are Inline28:00 Network Cost36:00 Client Deserializationhttps://medium.com/@hnasr/how-slow-is-select-8d4308ca1f0c
Lambda now supports Response payload streaming, now you can flush changes to the network socket as soon as it is available and it will be written to the client socket. I think this is a game changing feature
0:00 Intro
1:00 Traditional Lambda
3:00 Server Sent Events & Chunk-Encoding
5:00 What happens to clients?
6:00 Supported Regions
7:00 My thoughts
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
Cloudflare released a blog detailing a vulnerability that has been in their system for nearly two years. it is related to mTLS or mutual TLS and specifically client certificate revocation. I explore this in details 0:00 Intro3:00 The Vulnerability7:00 What happened?8:50 Certificate Revocation12:30 Rejecting certain endpoints 17:00 Certificate Authentication20:30 Certificate serial number24:00 Session Resumption (PSK)35:00 The bug37:00 How they addressed the problemFundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)https://backend.husseinnasser.com
BGP (Border gateway protocol) withdrawals caused the Virgin media ISP customers to lose their Internet connection. I go into details on this video. 0:00 Intro2:00 What happened?4:11 How BGP works?11:50 Version media withdrawals15:00 Deep dive Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)https://backend.husseinnasser.com
GitHub Accidentally Exposed their SSH RSA Private key, this is the message you will get . @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!Someone could be eavesdropping on you right now (man-in-the-middle attack)!It is also possible that a host key has just been changed.The fingerprint for the RSA key sent by the remote host isSHA256:uNiVztksCsDhcc0u9e8BujQXVUpKZIDTMczCvj3tD2s.Please contact your system administrator.Add correct host key in ~/.ssh/known_hosts to get rid of this message.Host key for github.com has changed and you have requested strict checking.Host key verification failed.In this video I discuss how bad is this,.0:00 Intro1:10 What happened?3:00 SSH vs TLS Authentication 6:00 SSH Connect7:45 How bad is the github leak? 15:00 What should you do?18:50 Is ECDSA immune?https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
How Linus Tech Tips channel got Hacked
In this short video we explain how was it possible for Linux to get hacked with cookies hijacking. 0:00 Intro0:47 TLDR what happened 5:10 Cookies in Chrome7:30 Cookies Hijacking8:46 Session Tokens (Access/Refresh)10:00 Remedies
Get my database engineering course https://database.husseinnasser.com
In this video I do a deep dive in all locks obtained by postgres, I learned a lot while making this video and hope you enjoy it.
0:00 Intro
2:30 What are Locks?
5:30 Overview of Postgres Locks
9:10 Table-Level Locks
11:40 ACCESS EXCLUSIVE
17:40 ACCESS SHARE
19:00 ROW SHARE
20:15 ROW EXCLUSIVE
21:15 SHARE UPDATE EXCLUSIVE
23:30 SHARE
24:50 SHARE ROW EXCLUSIVE
25:18 EXCLUSIVE
25:30 Table Lock Conflict Matrix
28:30 Row-Level Locks
30:00 FOR UPDATE
33:00 FOR NO KEY UPDATE
34:00 FOR SHARE
34:40 FOR KEY SHARE
35:10 Row Lock Conflict Matrix
39:25 Page-Level Locks
42:00 Deadlocks
46:00 Advistory Locks
47:20 Summary
https://www.postgresql.org/docs/current/explicit-locking.html
Pinterest moves to HTTP/3 on all their clients and edge CDNs this year. They witnessed interesting gains but not without good lesson learned. The main one was the mismatch of alt-svc vs DNS ttls.I cover this on the next episode of the backend engineering course.0:00 Intro2:00 Moving h2 to h3 through alt-svc5:00 Why HTTP/36:00 HTTP/1 vs HTTP/29:00 TCP Head of Line blocking in HTTP/211:00 How HTTP/3 addresses HOL12:15 Connection Migration13:30 Stream level congestion control14:10 1-RTT - 0-RTT15:41 Pinterest challenges moving HTTP/319:00 Migration21:15 Future work22:30 Summaryarticle https://medium.com/pinterest-engineering/pinterest-is-now-on-http-3-608fb5581094Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
On March 7 2023, Loom users started seeing each others data as a result of cookies getting leaked from the CDN. This loom security breach is really critical. Let us discuss 0:00 Intro 1:00 Why Cookies 2:00 How this happens 5:50 What caused it? 7:30 How Loom solved it? 8:20 Reading the RCA 10:30 Remedies
Discord engineering goes into details of how they migrated from Cassandra to ScyllaDB, improved the performance of their reads and writes and rearchitected their backend to support the new load. It is an interesting episode lets get into it
0:00 Intro
1:50 Relational vs Distributed
7:00 The Cassandra Troubles
11:00 SnowFlake vs UUID
14:30 B+Tree
19:20 B+Tree and SSDs
25:30 LSM Trees
31:00 Hot partitions
36:00 Cassandra Garbage Collector Pauses
40:00 Changing the Architecture
45:00 The Data Services
55:00 The Migration
1:02:00 Zoned Named Spaces
1:04:00 Summary
Article here How Discord Stores Trillions of Messages
https://discord.com/blog/how-discord-stores-trillions-of-messages
Creating a listener on the backend application that accepts connections is simple. You listen on an address-port pair, connection attempts to that address and port will get added to an accept queue; The application accepts connections from the queue and start reading the data stream sent on the connection.
However, what part of your application does the accepting and what part does the reading and what part does the execution? You can architect your application in many ways based on your use cases. I have a medium post just exploring the different options.
In this video I explore the PostgreSQL process architecture in details. Please note that the information here is derived from both the Postgres doc and code. Discussions about scalability and performance are solely based on my opinions.
0:00 Intro
1:30 Overview
3:30 Postgres MVCC
5:30 Processes vs Threads
7:40 Postmaster Process
8:00 Backend Processes
13:30 Shared Buffers
14:52 Background Workers
17:18 Auxiliary Processes
17:45 Background Writer
22:30 Checkpointer
23:40 Logger
24:06 Autovacuum Launcher and Workers
25:30 WAL Processes
28:53 Startup Process
Read full article
https://medium.com/@hnasr/postgresql-process-architecture-f21e16459907
The Alt-Svc header/frame is a capability that allows the server to adverse alternative services to the connected application available in protocols, ports or domains. It is available as a response header alt-svc and also as an HTTP/2 frame. Let us discuss this capability.
0:00 Intro
1:38 what is alt-svc?
5:30 uses of h3 in alt-svc
8:00 alt-svc header
10:00 Alt-svc header with 103 early hints
14:48 h2 altsvc frame
18:30 SVCB DNS record
21:20 Summary
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
Correction: Google is implementing the proposal originally submitted by researchers from Georgia institute of tech. I incorrectly said in the video that google is proposing this .
Google is finally implementing a proposal from 2008 by researchers from Georgia institute of technology to make DNS cache poisoning .
https://astrolavos.gatech.edu/articles/increased_dns_resistance.pdf
https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00
0:00 Intro
2:00 How DNS Work
5:00 DNS Cache Poisoning
14:00 gOoGLe dot CoM
16:20 ASCII 0x20 casing
18:30 Randomizing the casing with encryption
22:30 limitations of this proposal
24:00 Credits
https://dropbox.tech/infrastructure/increasing-magic-pocket-write-throughput-by-removing-our-ssd-cache-disks
In this episode of the backend engineering show I’ll discuss how Dropbox improved their write through put by 20% by removing all their SSDs (yes I was surprised too). DropBox uses an SSD layer as a write-back cache with SMR drives as their backend persistent storage. They changed their model to write directly to the hard drives.
0:00 Intro
2:00 Article Summary
3:00 SMR Drives
6:00 SSD Cache & WriteBack
8:00 Replacing Cache
9:30 Storage Engine Background
14:30 Why did they do it
15:00 The limitation of SSDs & Zoned Namespaces
19:30 Updating the Storage Engine
22:30 Tradeoffs
26:00 Rollout
28:00 Summary
The communication between backend applications and database systems always fascinated me. The protocols keep evolving and we are in constant search for an efficient protocol that best fit the workload of Backend-DB communication.
In this episode of the backend engineering show I go through a blog written by @PlanetScale doing an experimentation of using HTTP/3 and HTTP/2 comparing it with MySQL Binary protocol.
https://planetscale.com/blog/faster-mysql-with-http3
0:00 Intro
7:45 MySQL Binary vs HTTP
10:20 The Tests
15:00 Connection Cost + Select 1
22:00 Parallel Select
26:00 The cost of H2 and H3
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Shopify posted a blog on tips to for scalable payment system, one tip peeked my interest related to switching from UUID to ULID. I explore the reasoning behind this in this video.
https://shopify.engineering/building-resilient-payment-systems
0:00 Intro
1:30 idempotency
6:30 UUID vs ULID
9:50 Clustered Index
13:30 Why UUID4 Inserts are slow
17:15 How ULID helps Shopify
22:00 Problem with tail pages
25:00 Does ULID help in all cases?
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows. Everything goes to disk, the trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API. In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills. Let us get started.
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com
The User Interface/User Experience has great impact on the backend architecture and scalability. In this podcast I discuss three UI/UX that affected backend design and scalability.
0:00 Intro
1:40 UI vs UX
4:30 Google Chrome OmniBox
12:30 1 out of X Page
20:00 YouTube Notification
Resources
https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
In this video I explain how DHCP work and how it updates DNS entries for new hosts joining the network. I'll also mention Zero Config
0:00 Intro
1:00 the Network configuration
6:00 Showing DHCP in Wireshark
6:30 DHCP Discover
14:40 DHCP Offer
19:00 DHCP Request
21:30 DHCP ACK
22:00 How DHCP Updates DNS
26:15 Zero Configuration (mDNS, Link-local)
Resources
Dhcp https://datatracker.ietf.org/doc/html/rfc1541
Dynamic updates , dhcp RFC2136
https://datatracker.ietf.org/doc/html/rfc2136
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_dhcp/configuration/xe-3se/3850/dhcp-xe-3se-3850-book/dhcp-client-option-12.pdf
RFC 1497
https://www.rfc-editor.org/rfc/rfc1497.html
https://www.rfc-editor.org/rfc/rfc6762#ref-Zeroconf
Link-local
https://www.rfc-editor.org/rfc/rfc3927
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
Certificates provide a way to authenticate both the server and the client and are included as part of the TLS handshake. However, the certificates can be large because the full certificate chain is included in the handshake. The large certificates can go up to 10KB in size and take multiple segments to deliver and assemble. RFC 8879 discusses how TLS compression can be achieved, I discuss that in this podcast. Enjoy.
0:00 Intro 4:15 Certificate Chain 6:00 Faking the chain 8:50 Certificate Stores 10:30 Including ROOT cert in the chain 12:00 The performance penalty of large certificate chain 20:15 RFC 8879 TLS Certificate Compression 23:00 How Compression Works in TLS 1.2 vs TLS 1.3 30:30 What could go wrong? Resources https://datatracker.ietf.org/doc/rfc8879/ https://www.rfc-editor.org/rfc/rfc5246 https://www.rfc-editor.org/rfc/rfc6928.html
Two new vulnerabilities in openssl were discovered, we discuss them in this video
https://www.openssl.org/news/secadv/20221101.txt
Google recent paper on protective load balancing in TCP attempts to improve packet drops and latency by making the host change the flow path using the IPv6 Flow label. The Linux kernels gets the PLB support in Linux 6.2 this December, let us discuss with this is.
In this episode of the backend engineering show I go through an article I wrote discussing NodeJS Asynchronous I/O
https://medium.com/@hnasr/when-nodejs-i-o-blocks-327f8a36fbd4
Learn the fundamentals of network engineering, get my udemy course
https://network.husseinnasser.com
Buy me a coffee if you liked this
https://www.buymeacoffee.com/hnasr
0:00
3:00 Part 1 Socket/IO
9:48 Part 2 File I/O
12:42 Part 3 DNS
16:22 Part 4 NodeJS Single Threaded
19:10 Part 5 NodeJS Thread Pool
21:23 Part 6 DNS lookup bottleneck in Node
Buy me a coffee if you liked this https://www.buymeacoffee.com/hnasr
In this podcast I explain the NGINX internal process architecture, how NGINX spins up multiple processes per core, how each process is pinned to a CPU core for minimum context switching, how NGINX accepts connections , parses requests and talks to the backend. Get my introduction to NGINX udemy course https://nginx.husseinnasser.com
Cloudflare identified several limitations in NGINX architecture and decided to write their own reverse proxy.
0:00 Intro 1:53 What NGINX is used for 3:37 NGINX Architecture 7:52 NGINX Limitations 17:12 Cloudflare Pingora
Buy me a coffee if you liked this https://www.buymeacoffee.com/hnasr
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Introduction to NGINX (link redirects to udemy with coupon) https://nginx.husseinnasser.com Python on the Backend (link redirects to udemy with coupon) https://python.husseinnasser.com Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Resources Resources:
https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/
https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/amp/
In this episode of the backend engineering show I discuss the evolution of multi-threading apps, their pros and cons and then I go through 5 threading model and how they interleave with backend connection management between the threads and requests handlings. Enjoy
To learn more about networking fundamentals check out my udemy course Fundamentals of Networking for Effective Backends Head to https://network.husseinnasser.com for a discount coupon
0:00 Intro 2:00 Single Threading 6:30 Multi-Threading 14:15 Connection Listener 20:15 How Connections are Established 29:00 Single Listener/Worker thread 33:30 Single Listener, Multiple Worker threads 39:00 Single Listener, Multiple Workers with load balancing 42:10 Multiple Listeners on the same port (SO_REUSEPORT) 45:20 Multiple Single Threaded Backend
Memcached is an in memory cache with one major feature be a transient cache. Memcached has a very simple design. It was originally designed to help with database load by storing the query result in memory to avoid further querying the database. By default it has no authentication, a simple text protocols, servers don’t talk to each other. This video discuss the architecture of the cache, design choices and have some critics of the design choices. I go through a demo at the end using docker, telnet and nodes. Enjoy
0:00 Intro
4:40 What is Memcached?
7:45 Memory management
16:00 LRU
25:17 Threading and Connections
30:40 Read Example
34:30 Write Example
36:17 Write and Read collisions
39:40 Locking
40:30 Distributed Cache
43:30 Memcached with Docker/Telnet/NodeJS
45:00 Spin up a Memcached Docker container and telnet
52:17 Memcached and NodeJS
56:15 Four Memached Servers with NodeJS
01:01:00 Summary
Resources
https://www.cloudflare.com/learning/ddos/memcached-ddos-attack/
https://holmeshe.me/understanding-memcached-source-code-IV/
https://github.com/memcached/memcached/blob/master/doc/protocol.txt
https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-threads.html
https://holmeshe.me/understanding-memcached-source-code-I/
https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-memory.html
https://support-acquia.force.com/s/article/360005256114-Memcached-in-detail
https://www.alibabacloud.com/blog/redis-vs-memcached-in-memory-data-storage-systems_592091
https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf
https://memcached.org/blog/persistent-memory-2/
https://memcached.org/blog/modern-lru/
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I go through the main job of the network interface controller (NIC for short) and how the datacenter is pushing it to the limit by allowing it to do more TCP/IP processing, creating what is being popularized as smartNIC.
0:00 Intro
1:20 What is a NIC?
3:40 NIC job
8:00 When does the OS get involved
12:40 Promiscuous mode
14:00 SmartNIC
18:30 Disadvantages
Resources
https://developer.nvidia.com/networking/ethernet-adapters
https://www.theregister.com/2022/08/11/smartnics_network_market/
https://arxiv.org/abs/1803.09615
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I discuss consistent hashing a very important algorithm in distributed computing specially in database systems such as Apache Cassandra and DynamoDB.
0:00 Intro
2:00 Problem of Distributed Systems
5:00 When to Distribute
7:00 Simple Hashing
9:30 Where Simple Hashing Breaks
11:40 Consistent Hashing
18:00 Adding a Server
21:15 Removing a Server
22:30 Limitations
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I go through and discuss the Homa Protocol paper which attempts to replace TCP as a protocol in the data centers. I learned a lot from this paper, I have my criticisms of certain aspects, timestamps for topics discussed below.
It appears there is a path to replace TCP in the datacenter and professor John tries to explain this path.
Referenced materials mentioned in the episode
Overview paper
https://web.stanford.edu/~ouster/cgi-bin/papers/replaceTcp.pdf
Homa 2018 paper (Details)
https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf
NIC Offloading in Linux
https://en.wikipedia.org/wiki/TCP_offload_engine#Support_in_Linux
Curl disabling Nigel Algo
https://github.com/curl/curl/commit/4732ca5724072f132876f520c8f02c7c5b654d9
0:00 Intro
3:00 The nature of networking data center
5:30 TCP Segments
7:30 There is no “Request” in TCP
12:00 What so unique about Data centers?
14:00 Message Throughput vs Data throughput
18:25 Congestion Control
22:38 Homa’s Congestion Control
25:00 Server Core Load Balancing
28:30 NIC offloading
30:00 Everything Wrong about TCP
37:00 Why not QUIC?
40:00 Limitation of Streaming
44:10 Load Balancing Stream Reading
47:15 Can we treat Segments as Messages?
51:00 Dispatching Messages is Easier
53:00 Connection Orientation
1:00:00 Sender Driven Congestion Control
1:03:00 In Order Packet Delivery
1:07:00 DCTCP
1:08:30 Homa is Message Based
1:11:00 Home is Connection Less
1:12:00 Receiver Driven Congestion Control
1:15:19 Out of Order Packets
1:16:20 Homa API is not Compatible with TCP
1:17:40 Will Homa come to HTTP?
1:18:45 Conclusion
Support this podcast: https://anchor.fm/hnasr/support
ByteDance, the company behind TikTok is proposing few patches to the linux kernel to make kernel reboots via kexec go from 500ms down to 15 ms saving huge time in updating kernel on thousands of machines. Let us discuss this
0:00 Intro
1:30 Linux Kernel Reboot Options
2:30 how kexec works
4:00 The optimization
5:00 Going through the patch
6:00 Updating Servers at TikTok scale
9:00 Summary
https://www.phoronix.com/news/Bytedance-Faster-Kexec-Reboot
https://lore.kernel.org/lkml/20220725083904.56552-1-huangjie.albert@bytedance.com/
Support this podcast: https://anchor.fm/hnasr/support
We discuss the CVE-2022-2274 OpenSSL Vulnerability.
The OpenSSL 3.0.4 release introduced a serious bug in the RSA
implementation for X86_64 CPUs supporting the AVX512IFMA instructions.
This issue makes the RSA implementation with 2048 bit private keys
incorrect on such machines and memory corruption will happen during
the computation. As a consequence of the memory corruption an attacker
may be able to trigger a remote code execution on the machine performing
the computation.
0:00 Intro
1:00 CVE-2022-2274
3:00 AVX512IFMA CISC
5:00 How the bug works
7:10 How can it be triggered
Resources
https://www.openssl.org/news/secadv/20220705.txt
https://github.com/openssl/openssl/issues/18625
https://guidovranken.com/2022/06/27/notes-on-openssl-remote-memory-corruption/
https://eprint.iacr.org/2018/335
https://github.com/openssl/openssl/commit/4d8a88c134df634ba610ff8db1eb8478ac5fd345
https://linux.die.net/man/3/bn_internal
https://www.microfocus.com/documentation/enterprise-developer/ed60/ES-WIN/GUID-E3960B1E-C42E-4748-A5EB-6E12507C9CD7.html
https://www.microcontrollertips.com/risc-vs-cisc-architectures-one-better/
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Postgres 15 introduces a new syntax to treat nulls as not distinct. This will allow developers to create a unique constraint on nullable fields and have inserts with nulls fail instead of having it allowed based on today's standard.
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
https://blog.rustprooflabs.com/2022/07/postgres-15-unique-improvement-with-null
https://www.postgresql.org/message-id/flat/84e5ee1b-387e-9a54-c326-9082674bde78%40enterprisedb.com
Support this podcast: https://anchor.fm/hnasr/support
memchr() is a function that searches in a memory block for a character, this has been recently improved in the latest linux patch and we can learn something from this improvement I think, lets discuss.
https://www.phoronix.com/scan.php?page=news_item&px=Linux-Kernel-Faster-memchr
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Amazon Aurora PostgreSQL-Compatible Edition now supports PostgreSQL major version 14 (14.3). Let us discuss this news.
https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-aurora-supports-postgresql-14/
0:00 Intro
1:00 Database on the Cloud options
3:45 Amazon Aurora supports Postgres 14
6:00 Postgres 14 vs Postgres 13
Support this podcast: https://anchor.fm/hnasr/support
Rogers Communications, the Canadian ISP is down. One of its ASNs (they have many) is AS812 with over 5 million IP addresses. Tried a few and none of them can be pinged from the US (not sure if ICMP is disabled or not)
This is a huge deal for all Canadians and businesses affected. So sorry for every one who is affected by this, and kudus to all the engineers at Rogers working to fix this for the past what? 11 hours now?
If an ASN goes dark like a Facebook or a Cloudflare that isn't a big deal you can go without using Facebook for a day. But if this is your ISP's ASN that connects you to the rest of the World goes down, actual users won't be able to connect.
We still don't know the cause but my guess it might be a bad BGP entry? that's what happened to Cloudflare or FB, could be something different. again so sorry for all my Canadian followers affected. Will make a video when I get a chance
Resources
https://www.bigdatacloud.com/asn-lookup/AS812
https://www.bigdatacloud.com/asn-lookup/AS7018
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I’ll discuss Fill Factor index property, an important key parameter for a good performing database. I will also discuss how it is a double edge sword and can break or Make your db. Chapters 0:00 Intro 1:48 Start of the Show 4:30 Database Storage Representation 6:30 IO Cost 10:00 Index Page 13:30 Page Splits Cost 18:00 Indexing UUIDs 19:45 FillFactor 23:15 Bad FillFactor values 26:40 Fragmentation In Indexes 30:30 Summary
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss early hints HTTP 103 status code, the problem it solves and the challenges it has.
Support this podcast: https://anchor.fm/hnasr/support
YugabyteDB is a postgres compatible and cloud native database. Read committed isolation level is a critical feature and adding it might lure more postgres customer’s to move to the cloud native database. But will they compete in front of Google’s new AlloyDB ? 0:00 Yogabyte implements Read committed 4:20 Isolation Levels 8:40 Can YugabyteDB compete with Google AlloyDB? https://www.theregister.com/2022/06/28/yugabytedb_215/ https://docs.yugabyte.com/preview/architecture/transactions/explicit-locking/
Support this podcast: https://anchor.fm/hnasr/support
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Transport Layer security or TLS is a protocol that encrypted the communication between client and server. TLS can also be used to authenticate the server, when the client also requires authentication this is called Mutual TLS and this is the topic of today’s show.
0:00 Introduction
2:00 What is TLS?
7:00 Server Authentication TLS
14:00 Advantages of one way TLS
19:44 Disadvantages of one way TLS
29:00 mTLS
31:00 Advantages of MTLS
37:00 The Problems of mTLS
43:00 Summary and my Thoughts
Support this podcast: https://anchor.fm/hnasr/support
In this episode we go through the cloud flare outage blog. https://blog.cloudflare.com/cloudflare-outage-on-june-21-2022/ Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I go through the Extensible Prioritization Scheme for HTTP. This RFC addresses the problem in HTTP where the client prefers certain requests to be served before others. This RFC was designed by Lucas Pardue from Cloudflare and Kazoo Oku from Fastly. 0:00 The Request Priority Problem 13:00 How RFC 9218 addresses the problem 25:00 HTTP/1.x Backends 32:00 Summary RFC 9218 https://datatracker.ietf.org/doc/rfc9218/
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain what is a zero day vulnerability or attack. Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
IPv4 UniCast Extensions Project attempts to unreserve old IPv4 subnets so they are publicly addressable. Is it worth it? Let us discuss https://blog.apnic.net/2022/05/31/cutting-down-on-ip-address-waste/Unicast Use of the Formerly Reserved 127/8 https://datatracker.ietf.org/doc/draft-schoen-intarea-unicast-127/ Unicast Use of the Lowest Address in an IPv4 Subnet https://datatracker.ietf.org/doc/draft-schoen-intarea-unicast-lowest-address/ Unicast Use of the Formerly Reserved 240/4 https://datatracker.ietf.org/doc/draft-schoen-intarea-unicast-240/ Unicast Use of the Formerly Reserved 0/8 https://datatracker.ietf.org/doc/draft-schoen-intarea-unicast-0/ https://twitter.com/TheRealRevK/status/1532033651650830339 Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Fantastic gains in Postgres 15! improved sorting by 371%. Normally you can avoid sorting by using an index access path. However, you can’t always have an index on every group by, order by, join or distinct predict. This will come handy. Let us discuss
Resources
https://www.phoronix.com/scan.php?page=news_item&px=PostgreSQL-15-Faster-Sort
https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/speeding-up-sort-performance-in-postgres-15/ba-p/3396953#change3
https://www.postgresql.org/docs/15/release-15.html#id-1.11.6.5.4
https://techcommunity.microsoft.com/t5/sql-server-blog/understanding-sql-server-fast-forward-server-cursors/ba-p/383556
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I discuss UDP hole punching in details. Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
As a new parent, you get exposed to a whole new world. There is this new toddler song “Five little monkeys jumping on the bed” where it starts with five monkeys jumping on the bed and one falls off the bed and the mom calls the doctor and then another monkey falls and the mom calls the doctor again. This keeps going until all monkeys are no longer on the bed. This clearly looks like a software engineering problem. In the face of it is extreme inefficiency in IO with the mom calling the doctor 5 times in span of 3 minutes. The mom could decide to wait for all monkeys to fall and call the doctor once. A Method that is known as batching. Two problems with this approach; the first is latency the mom can’t wait, she is worried, she needs immediate answer from the doctor for the first monkey that fell. The second problem is we don’t know if more monkeys are going to fall off the bed, so while the mom waiting, the first monkey will be in pain and will “starve”. We makes decisions like this constantly in software engineering. In some database systems for example commits from transactions are grouped and batched for few microseconds/milliseconds so more commits can arrive and the database can flush/fsync the WAL once. This is specially important in highly concurrent system because disk fsync is costly and bypasses the operating system cache. However, this comes at a cost of slight delay in transaction commits but less IO in general. To batch or not to batch. The five little monkey problem doesn’t have one right solution it all depends.
Support this podcast: https://anchor.fm/hnasr/support
Eric Dumazet, a Linux kernel and a Google Principal software engineer pushed a new change to Linux to support a new feature in Linux called BIg TCP. How about we discuss this resources https://www.phoronix.com/forums/forum/software/general-linux-open-source/1325637-linux-5-19-networking-brings-big-improvements-with-big-tcp-purelifi-more-hardware https://patchwork.kernel.org/project/netdevbpf/patch/20220524203159.1189780-1-kuba@kernel.org/https://datatracker.ietf.org/doc/html/rfc2675
Support this podcast: https://anchor.fm/hnasr/support
Database logging is a critical feature to maintain durability, in this show I discuss them in details 0:00 intro 1:00 ad 3:30 Start of the Show 6:00 What is commit 9:30 What if a commit fails half way 11:00 WAL 23:00 Checkpointing 27:00 fsync 33:00 Undo logs Get $20 off of your first order of cometeer quality delicious coffee https://cometeer.com/husseinnasser use coupon HUSSEIN20 ☕️
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss the developer mental health, stress, pressure, expectation, how to understand and shed light on the issues, and what might cause it all. Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain how we can use two-phase commit protocol works to achieve atomic distributed transaction. 0:00 What is atomicity 3:00 Distributed Transaction 6:00 Two phase commit protocol 13:00 Limitations of Two phase commit protocol Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Welcome to another database question. In this question I created a community poll question and provided some answers. All answers can be correct of course but the question is what is the most efficient? this is what I try to explore in this video and compare how different database platforms such as mysql or Postgres differ in those implementations;
Table T with three integer fields A, B and C.
A has a primary key clustered index.
B has a secondary index.
Which query is more likely to be the most efficient?
0:00 Intro
1:00 The Question
3:20 WHERE C BETWEEN 50,50000
5:30 WHERE B BETWEEN 50,50000
13:20 WHERE C BETWEEN 50,50000
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
WhatsApp Went down again, this is an early report with brief analysis, enjoy. https://engineering.fb.com/2020/10/21/networking-traffic/how-facebook-is-bringing-quic-to-billions/
Support this podcast: https://anchor.fm/hnasr/support
DNS or Domain Name System, despite its drawbacks, is brilliantly designed for scale. We can learn few lessons from this protocol especially when designing our own apps. This episode of the backend engineering show I go through how DNS works, the pros and the cons and attacks that happened on this system.
0:00 Intro
2:00 Overview DNS
7:40 How DNS works (Details)
15:44 DNS uses UDP
19:30 DNS Poisoning
24:10 is DNS really distributed?
26:30 How Attackers Abuse DNS
30:30 How Chrome overloaded the ROOT servers for 12 years
Resources
https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/
https://www.cloudflare.com/learning/dns/what-is-dns/
https://www.cloudflare.com/learning/dns/dns-cache-poisoning/
https://blog.cloudflare.com/sad-dns-explained/
https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
https://nginx.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show we discuss the Internet Protocol. A beautiful, elegant protocol that made the Internet possible. We discuss why we need an IP address, how routing is done and how VPNs take advantage of the agnostic nature to encrypt traffic
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show we discuss an article written by the Forem team illustarting a bug they recently fixed in their caching layers. They go into good level of details explaining the 3 levels of caching that they have. I go through this article in this episode and give my opinion at the end as to why we shouldn’t really cache unless we need to. Enjoy
0:00 Intro
1:00 Three level of caching summary
7:39 Edge Caching
8:30 Backend Rails Caching
10:30 Database Caching
15:00 Understand why you Cache
Article
https://dev.to/devteam/the-three-caches-of-forem-492p
Support this podcast: https://anchor.fm/hnasr/support
Backend database applications relay on good storage systems for performance, durability and low latency. SSDs have been a great invention that changed the storage game compared to mechanical drives. However, SSDs came with their own sets of problems which can cause database reads and writes to plummet after a while. This episode discusses the current limitations of SSDs include garbage collection, wear leveling, over provisioning and how zoned name spaces “may” fix this.
0:00 Intro
1:40 Block
4:00 Logical Block Addressing
6:00 Flash Translation Layer
6:50 DRAM
8:50 Erase unit
11:11 Namespace
12:00 Summary
13:40 Garbage collection
16:49 Over provisioning
18:00 Write amplification
21:28 Wear leveling
Resources
https://www.guru3d.com/news-story/new-malware-bypasses-security-measures-by-using-ssd-over-provisioning.html
https://www.snia.org/educational-library/zoned-namespaces-zns-ssds-disrupting-storage-industry-2020
https://www.youtube.com/watch?v=cbX3P56Jp0o&feature=emb_title
Support this podcast: https://anchor.fm/hnasr/support
Google linux boxes have over 16 NVMe SSD PCIe Express drives.When a shutdown signal is sent to linux, the OS iterate through each NVMe and send synchronous request to shutdown that takes 4.5 seconds. This adds up to over a minute to achieve a reboot. Google patches linux with asynchronous Shutdown APIs
Resources https://www.phoronix.com/forums/forum/phoronix/latest-phoronix-articles/1316262-google-has-a-problem-with-linux-server-reboots-too-slow-due-to-too-many-nvme-drives
patch https://lore.kernel.org/lkml/20220328230008.3587975-1-tansuresh@google.com/
Sync vs async https://www.youtube.com/results?search_query=asynchronous+vs+synchronous+hussein
Support this podcast: https://anchor.fm/hnasr/support
Hash tables are effective for caching, database joins, sets to check if something is in a list and even load balancing, partitioning and sharding and many other applications. Most programming languages support hash tables. However they don’t come without their cost and limitations lets discuss this. 0:00 Intro 1:50 Arrays 3:50 CPU Cost (NUMA/M1 Ultra) 6:50 Hash Tables 10:00 Hash Join 16:00 Cost of Hash Tables 20:00 Remapping Cost Hash Tables 22:30 Consistent hashing
Support this podcast: https://anchor.fm/hnasr/support
Aggregate functions like Count, max, min, avg performance really depends on how did you tune your database for that kind of workload. Let us discuss this.
0:00 Intro
1:22 SELECT COUNT(*)
4:30 SELECT AVG(A)
5:15 SELECT MAX(A)
8:00 Best case scenario
11:30 Clustering
14:00 Clustering Sequential Writes
17:19 Clustering Random Writes
20:30 Summary
Support this podcast: https://anchor.fm/hnasr/support
99% of phishing attacks can be avoided by looking at the URL. However this secuirty researcher proves that not enough. https://mrd0x.com/browser-in-the-browser-phishing-attack/
Support this podcast: https://anchor.fm/hnasr/support
On March 8 2022 Spotify and Discord experienced an outage latest 2-3 hours. The reason was a configuration on the xDS formats on Google Traffic Director. Let us discuss how this change caused the outage and what Spotify did to mitigate that outage without relying on Google restoring the service back up. Resources Spotify outage https://engineering.atspotify.com/2022/03/incident-report-spotify-outage-on-march-8/ Google Cloud outage https://status.cloud.google.com/incidents/LuGcJVjNTeC5Sb9pSJ9o Envoy xDS https://blog.envoyproxy.io/the-universal-data-plane-api-d15cec7a Microservices scaling with common sense https://www.youtube.com/watch?v=NsIeAV5aFLE CARDS 4:36 Miicros https://www.youtube.com/watch?v=NsIeAV5aFLE 9:30 Spotify Hermes https://www.youtube.com/watch?v=fMq3IpPE3TU&t=21s 0:00 Intro 2:00 Spotify Outage 3:30 Microservices 6:10 Service Discovery 10:00 Spotify Quick Workaround 12:15 Google Traffic Director Outage
Support this podcast: https://anchor.fm/hnasr/support
The Russia-Ukraine Cyberwar has reached a peak this past week. In this video I explain the multiple ways a Denial of Service attacks can happen and how to protect against them Intro 0:00 What is DOS? 1:00 Long running requests 1:42 Crash backend processes 3:30 Exhaust Max connections 4:25 Large response (web traffic) 5:42 Lots of requests 8:27 Complex request (Regex bug) 9:57 Prevention 12:27
Support this podcast: https://anchor.fm/hnasr/support
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com
Chrome, Edge and Firefox are approaching version 100. that version is 3 digit version where most websites only checked two digits. You can imagine what can go wrong. Lets discuss
Support this podcast: https://anchor.fm/hnasr/support
On 2/22/2022 Slack experienced a 3 hours outage from 6AM to 9AM PST because of a configuration change. That change lead to increase load to their database infrastructure which causes queries to starve. Lets discuss
https://status.slack.com/2022-02-22
Support this podcast: https://anchor.fm/hnasr/support
It looks like there is demand for bringing QUIC and HTTP/3 to NodeJS, someone donated $1000 dollar to make it happen. However, what are the use cases for supporting HTTP/3 in Nodejs? I talk about whether HTTP/3 should come to NodeJS in this video.
https://rysolv.com/issues/detail/863986ce-c8d5-466d-abf4-476d3177452d
Support this podcast: https://anchor.fm/hnasr/support
Five major Canadian banks went offline for hours blocking access to online and mobile banking as well as e-transfers for customers. There are no reports of what might have caused the outage so In this video we explain what might have caused it. https://www.bleepingcomputer.com/news/security/canadas-major-banks-go-offline-in-mysterious-hours-long-outage/
Support this podcast: https://anchor.fm/hnasr/support
NVMe changed the game of fast SSD storage. The spec is being extended to support networked storage, I have some doubts that would like to discuss in this video. Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Introduction to NGINX (link redirects to udemy with coupon) https://nginx.husseinnasser.com Python on the Backend (link redirects to udemy with coupon) https://nginx.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
I stumbled upon a research calculating how big of quantum computer required to break bitcoin public key encryption. It will take them 300 million quantum bits to find the corresponding private key given a public key. That wasn’t really the most instructive thing really from that research. The interesting thing is how bitcoin design carefully hides information about the original owner which makes breaking it really challenging.
In this episode I discuss what does it take to break bitcoin encryption.
0:00 Intro
3:00 Public Key Cryptography
6:30 Bitcoin Keys ( Private key/public key and bitcoin address)
8:18 Why do we need a Bitcoin address
11:13 How minors verify transactions
17:30 When is the bitcoin public key visible?
19:12 The Wallet
20:47 What does it take to break bitcoin?
-find wallet.dat file which contain all the keys, some upload that online
-obtain public key from digital sign during a transaction posting. If someone used that public key , obtained the private key, then used it to generate a new ransaction to a different address before this transaction get posted.
-have a bitcoin address in mine satoshi, find out the public key not possible, then find out the private key from the public key, you can take all the time you need here. Because once you find those puppiest you are off to the races
25:20 Bits of security
28:00 My thoughts
Resources
https://avs.scitation.org/doi/10.1116/5.0073075
Support this podcast: https://anchor.fm/hnasr/support
NodeJS introduces importing network based modules in their latest release. Importing network based modules using https: and http: is supported under the --experimental-network-imports flag. This allows web browser-like imports to work in Node.js with a few differences due to application stability and security concerns that are different when running in a privileged environment instead of a browser sandbox.
Import HTTPS
https://github.com/nodejs/node/pull/36328/files
Support this podcast: https://anchor.fm/hnasr/support
I asked a question on my community post What port does your computer connect to when you visit google.com on your browser for the very first time? The answer really depends on the network configuration and that's what I discuss in this video. The answer could be HTTP port 80, HTTPS port 443, DNS port 53 or other answers that aren't even listed here including DNS over HTTPS and DNS over TLS. Lets discuss
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I discuss the ramification of index page splits which results in fragmented index yielding slow query performance when using indexes. I go through what a page is, how a page is read and written in the database and finally discuss the current solutions to address this problem such as reindexing, fill factor and choosing a good index key type. Get my Fundamentals of Database Engineering Udemy course https://database.husseinnasser.com 0:00 Intro 1:50 What is a Page? 5:00 How to Read and Write a page 8:45 Batching Writes and WAL 11:45 The Order of Content in the Page 16:00 Page Splits 21:30 Solving Index Fragmentation 28:30 Summary
Support this podcast: https://anchor.fm/hnasr/support
Multitenancy refers to placing two or more clients, customers or tenants so they share a single resource. The idea behind Multitenancy is to save on resources instead of giving each client its own resource (storage, app, memory, network etc..). You slice each part and give it to dedicated client. The number one rule is those tenants shouldn’t be aware of the existence of other tenants, they need to be completed isolated, no one tenant can read or influence the other’s tenant share. If this is confusing don’t worry I’ll go through examples in this episode of the backend engineering show.
0:00 Intro
2:50 Ssd Multitenancy
6:45 Database Multitenancy
10:14 Networking Multitenancy (SDN)
16:55 VMs and Containers Multitenancy
19:00 Application Multitenancy
Support this podcast: https://anchor.fm/hnasr/support
Scientists at the Physics and Engineering Department of the UK’s Lancaster University published a new paper describing ULTRARAM. A Low-Energy, High-Endurance, Compound-Semiconductor Memory on Silicon. I think this tech is a game changer but I still have my doubts. Let us discuss 0:00 Intro 0:30 RAM 4:00 SSD 11:00 UltraRAM 17:30 Doubts
- RAM (Dynamic RAM)
- Fast access with capacitors
- random access
- requires power to be refreshed
- Flash NAND (SSD)
- Pros fast random access (FTL)
- Block storage
- Requires high voltage (20V to erase/ 7.5 to program)
- Low durability endurance which then (write amplification + garbage collection )
- UltraRAM uses a new novel approach to the memory cell design that only need 2.5 V to program/erase, low power which saves the memory cell lifetime https://www.sciencedirect.com/topics/engineering/fowler-nordheim-tunnelling
- High endurance program/erase last longer
- High retention
- GC/WA not required
- Limitations: still we know very little
https://onlinelibrary.wiley.com/doi/10.1002/aelm.202101103
Support this podcast: https://anchor.fm/hnasr/support
enom my DNS registrar had an outage which took down my site husseinnasser.com lets discuss 0:00 Intro 5:00 How DNS Works 12:00 Outage report 16:00 Will I migrate my DNS? 21:40 Does decentralize mean anything anymore?
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I’ll discuss the low code movements and their pros and cons.
0:00 Intro
2:00 Evolution of Languages
8:40 Low Code Movement
10:00 Side Effect of Low Code
13:00 Leaky Abstractions
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
https://nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
https://nginx.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
These are some fundamental technologies that I want to explore in 2022. 0:00 The Way I learn 4:20 Database Engineering 7:00 Network Engineering 9:30 Hardware and Operating Systems 15:00 The problem with software engineering Collateral Knowledge Video https://www.youtube.com/watch?v=6YKbVpWmeLM&t=3s Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🔥 Members Only Content https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Introduction to NGINX (link redirects to udemy with coupon) https://nginx.husseinnasser.com Python on the Backend (link redirects to udemy with coupon) https://nginx.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Welcome to another dev tools video, in this video I use dev tools to break down how Tiktok website works to pull short-form content.
We will use therock as a subject. It is an interesting episode, enjoy
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I explain the journey of an HTTP request that gets initiated from a click on a link. I discuss DNS, TCP, API Gateways, reverse proxies, load balancers, backend web servers, and much more.
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I explain the journey of an HTTP request that gets initiated from a click on a link. I discuss DNS, TCP, API Gateways, reverse proxies, load balancers, backend web servers, and much more.
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Right after the latest patch log 4j 2.16, a new denial of service vulnerability surfaced on log4j resulting in a new 2.17 patch. Let us discuss.
0:00 log4dos
4:50 History of log4j fixes
15:20 All love to the open source maintainers
Previous Backend Engineering show episode coverage of log4shell
https://youtu.be/77XnEaWNups
https://logging.apache.org/log4j/2.x/security.html#
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, we go through a deep dive to uncover the reason behind the outage on December 7th, 2021 Amazon outage.
RCA
https://aws.amazon.com/message/12721
All outage analysis videos
https://www.youtube.com/watch?v=dhZ5--R42AM&list=PLQnljOFTspQXdkZLiYCCh_5RBP1-T-Rnx
Get the Fundamentals of Database Engineering udemy course
https://database.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, we go through a deep dive to uncover the reason behind the outage on December 7th, 2021 Amazon outage.
RCA
https://aws.amazon.com/message/12721
All outage analysis videos
https://www.youtube.com/watch?v=dhZ5--R42AM&list=PLQnljOFTspQXdkZLiYCCh_5RBP1-T-Rnx
Get the Fundamentals of Database Engineering udemy course
https://database.husseinnasser.com
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I discuss the log4j vulnerability (CVE-2021-44228 also known as log4shell) that took the Internet by storm.
0:00 Intro
1:00 log4j
5:30 How the attack started
11:00 Attack with DNS
17:00 Remote Code Execution
23:00 Remedy
31:00 Scanning
References
https://nvd.nist.gov/vuln/detail/CVE-2021-44228
https://youtu.be/oC2PZB5D3Ys
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show taken from my udemy Q&A I discuss the reasons behind Postgres HOT Optimization or heap only tuple.
Support this podcast: https://anchor.fm/hnasr/support
Postgresql database implements MVCC by creating a new row version for any update/delete/or insert. While this is a sound implementation to support concurrent transactions reading old version of the rows, it does have some side effects and this is what I want to discuss in this episode of the backend engineering show.
0:00 Intro
1:00 Postgres MVCC design
6:42 MVCC on other databases
11:15 Side-effects of Row-versioning
17:00 Postgres HOT optimization
19:50 How Index bloat affects Performance
24:20 How Postgres 14 Addresses index bloat
Cards:
14:12 b-tree https://www.youtube.com/watch?v=a1Z40OC553Y
Resources
https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION
Support this podcast: https://anchor.fm/hnasr/support
Head to https://database.husseinnasser.com to get a discount coupon for my introduction to database engineering.
In this video, I answer some of your questions on the Introduction to Database Engineering Course.
Support this podcast: https://anchor.fm/hnasr/support
head to https://database.husseinnasser.com to get a discount coupon for my Introduction to Database Engineering course
In this episode of the backend engineering show, I’ll discuss three instances where indexes can be useless and might add overhead to your operations. Let us discuss.
0:00 Intro
1:34 What is an Index?
4:00 The Cost of Indexes
6:40 Most values are the similar
13:00 WHERE UPPER(NAME)=‘RICK’
17:10 Composite Index
23:00 How do I know if I’m using an Index
Support this podcast: https://anchor.fm/hnasr/support
The cost of a long-running update transaction that eventually failed in Postgres (or any other database for that matter. In Postgres, any DML transaction touching a row creates a new version of that row. if the row is referenced in indexes, those need to be updated with the new tuple id as well. There are exceptions with optimization such as heap only tuples (HOT) where the index doesn’t need to be updated but that doesn’t always happens. If the transaction rolls back, then the new row versions created by this transaction (millions in my case) are now invalid and should NOT be read by any new transaction. You have two solutions to address this, do you clean all dead rows eagerly on transaction rollback? Or do you do it lazily as a post process? Postgres does the lazy approach, a command called vacuum which is called periodically Postgres attempts to remove those dead rows and free up space in the page. Whats the harm of leaving those dead rows in? Its not really correctness issues at all, in fact transactions know not to read those dead rows by checking the state of the transaction that created them. This is however expensive, the check to see of the transaction that created this row is committed or rolled-back. Also the fact that those dead rows live in disk pages with alive rows makes an IO not efficient as the database has to filter out dead rows. For example, a page may have contained 1000 rows, but only 1 live row and 999 dead rows, the database will make that IO but only will get a single row of it. Repeat that and you end up making more IOs. More IOs = slower performance. Other databases do the eager approach and won’t let you even start the database before rolling back is successfully complete, using undo logs. Which one is right and which one is wrong? Here is the fun part! Nothing is wrong or right, its all decisions that we engineers make. Its all fundamentals. Its up to you to understand and pick. Anything can work. You can make anything work if you know what you are dealing with. If you want to learn about the fundamentals of databases and demystify it check out my udemy course
https://database.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I’ll discuss all HTTPS/TLS binding options in Microsoft IIS and also explain why every web server and reverse proxy should have some of these fine level control. Chapters 0:00 Intro 1:00 Require Server Name Indication (SNI) 5:00 Disable TLS 1.3 Over TCP 8:30 Disable Legacy TLS 10:00 Disable OCSP Stapling 12:00 Disable QUIC 14:30 Disable HTTP/2 17:30 Certificate Get my database course https://database.husseinnasser.com Get my NGINX course https://nginx.husseinnasser.com Get my Python on the Backend course https://python.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
I get a lot of emails asking me to talk about graph databases, so I want to start researching them, but I wanted to give you guys the framework of how I think about any databases to defuse any “magic” that might be there.
In this video, I discuss what constrains a database and how the use cases try to work around them.
0:00 Intro
1:50 What constrains a database?
4:00 Indexing Techniques
5:30 Storage Engines - Row-Store
9:00 Columnar Databases
12:00 Graph use cases
16:00 Graph Storage Engines
Learn the fundamentals of databases, grab my my Introduction to Database Engineering udemy course here for a discount coupon https://database.husseinnasser.com 🧑🏫
Support this podcast: https://anchor.fm/hnasr/support
Certificates contain useful metadata including the public key, domain name, signature, etc. However, the private key can be leaked which causes the certificate to be invalid/dangerous to keep around. In that particular situation, we need a mechanism to revoke certificates and that is what I’m going to discuss in this show.
0:00 Intro
0:30 Why Certificates
12:00 Certificates can go bad
14:50 Certificate Revocation Lists (CRLs)
18:30 OCSP (Online Certificate Status Protocol)
20:40 OCSP Stapling
24:30 Best certificates are short
26:30 Summary
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
In this episode, I go through the Facebook detailed article regarding their October 4th, 2021 outage and discuss it in length. enjoy
Facebook blog: https://engineering.fb.com/2021/10/05/networking-traffic/outage-details/
0:00 Introduction on Facebook Networking Architecture
12:00 The Cause of the Outage
17:00 What’s DNS
23:00 DNS Servers disabled BGP ads
27:00 Could the outage have been prevented?
32:00 Why did it take so long?
38:00 Why you can’t just flip everything on
41:30 Summary
🧑🏫 Courses I Teach
https://database.husseinnasser.com
https://nginx.husseinnasser.com
https://python.husseinnasser.com
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🏭 Backend Engineering Videos in Order
https://backend.husseinnasser.com
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
Support this podcast: https://anchor.fm/hnasr/support
Quick summary of the Facebook, Instagram and WhatsApp outage on Oct 4th 2021
Support this podcast: https://anchor.fm/hnasr/support
In this episode I discuss my recent flight through Alaskan airlines and how they block certain services but allow only texting.
Support this podcast: https://anchor.fm/hnasr/support
Spook.js is a new transient execution side channel attack which targets the Chrome web browser. We show that despite Google's attempts to mitigate Spectre by deploying Strict Site Isolation, information extraction via malicious JavaScript code is still possible in some cases.
Resources
https://www.spookjs.com/
https://www.chromium.org/developers/design-documents/site-isolation
Paper: https://www.spookjs.com/files/spook-js.pdf
Chapters
0:00 Process Isolation in Chrome
8:00 Spook.js subdomain Attack
12:00 Spook.js Extension Attack
13:00 Summary
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I’ll go through the SSL Stripping attack, what caused it, what were the mitigations enforced to solve it, and why it is still a problem. 0:00 Intro 2:10 The Web Security Model 14:30 SSL Stripping Example 22:00 How to Solve SSL Stripping? 27:00 Limitations of HSTS 31:00 Summary Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🔥 Members Only Content https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg Support my work on PayPal https://bit.ly/33ENps4 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Let us take a moment to appreciate the TCP/IP design by discussing it.
Support this podcast: https://anchor.fm/hnasr/support
The gRPC team just submitted a proposal to support HTTP/3. This is big news and we have been waiting for a long time for this. In this show, I’ll discuss why would you want gRPC, why gRPC picked HTTP/2, what is actually wrong with HTTP/2 and what HTTP/3 solves. And Finally, I’ll discuss what is wrong with HTTP/3 and pitfalls that you as gRPC user might run into. Let us discuss!
0:00 Intro
1:30 why gRPC
5:20 gRPC & HTTP/2
7:30 gRPC & HTTP/3
8:50 What is wrong with HTTP/2
29:30 What is good about HTTP/3
37:00 What’s wrong with HTTP/3
Resources
https://github.com/grpc/proposal/pull/256/files
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
I discuss keepalive in TCP and HTTP and its pros and cons
0:00 Intro
3:28 What is KeepAlive
8:30 TCP KeepAlive
10:30 Middleboxes and Single-Path TCP
16:30 Middle Boxes and Keepalive
19:30 The FTP KeepAlive trap
25:00 HTTP KeepAlive
30:00 What's a good keepalive timeout?
https://daniel.haxx.se/blog/2020/02/10/curl-ootw-keepalive-time/
https://datatracker.ietf.org/doc/html/rfc1122#section-4.2.3.6
https://datatracker.ietf.org/doc/html/rfc793#section-3.5
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
In this video I’ll illustrate how a proxy server works under the hood. I Will go through how connection establishment works without a proxy, with an HTTP proxy and finally with HTTPS proxy in tunnel mode and TLS termination mode.
0:00 Intro
1:00 Connection Establishment without a Proxy
5:00 Connection Establishment with an HTTP Proxy
15:22 Connection Establishment with an HTTPS Proxy (Tunnel mode)
19:50 Connection Establishment with an HTTPS Proxy (TLS Termination)
cards
16:25 https://youtu.be/PAJ5kK50qp8 connect method
Support this podcast: https://anchor.fm/hnasr/support
I discuss the concept of process forking, copy on write (COW) aka shadowing, and how Redis the in-memory database take advantage of that for asynchronous snapshotting,
https://redis.io/topics/persistence
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I discuss database clustering. This is also known as table clustering, clustered index or Index organized table all names represents the same thing. I will talk about the benefits of clustering and also the disadvantages of implementing clustering. This feature is also implicitly implemented in certain databases.
More readings https://www.postgresql.org/docs/14/sql-cluster.html https://oracle-base.com/articles/8i/index-organized-tables https://docs.microsoft.com/en-us/sql/relational-databases/indexes/clustered-and-nonclustered-indexes-described?view=sql-server-ver15 https://dev.mysql.com/doc/refman/5.7/en/innodb-index-types.html
Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🔥 Members Only Content https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg Support my work on PayPal https://bit.ly/33ENps4 🧑🏫 Courses I Teach https://husseinnasser.com/courses 🏭 Backend Engineering Videos in Order https://backend.husseinnasser.com 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🎙️Listen to the Backend Engineering Podcast https://husseinnasser.com/podcast
Support this podcast: https://anchor.fm/hnasr/support
In this video, I’ll explain synchronous vs asynchronous operations and then discuss examples where this shows up. In programming, real-time messaging, database systems, and operating systems.
0:00 Definition Synchronous vs Asynchronous
4:15 sync vs async In Programming
7:50 Sync vs Async In Real-time messaging
17:00 Sync vs Async In Database Replication
23:50 Sync vs Async in Database Commits
29:30 Sync vs Async in fsync Operating System cache
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the Backend Engineering show, I'll discuss the advanced section settings in the Application Pool in Internet Information Services in IIS. Chapters 0:00 Intro 0:30 What is an Application Pool 3:00 IIS Multi-processing 5:18 .NET CLR Version 6:00 32-bit apps 6:21 Pipeline Mode (CGI vs ISAPI vs Native) 8:45 Max Queue 10:18 CPU Limiting 16:00 Processor Affinity 20:00 Summary
Support this podcast: https://anchor.fm/hnasr/support
While the benefits of partial indexes can have a great impact on your database system performance, the implications are also great if misused. Let us discuss partial indexing (Also known as filtered indexes in SQL Server) Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🔥 Members Only Content https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg Support my work on PayPal https://bit.ly/33ENps4 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Replication is the process of pushing changes from the master node to worker replica nodes in a database system to allow for horizontal scalability. One of the methods of replication is statement-based which is popular in MySQL. In this episode of the Backend Engineering Show, I explain why statement-based replication is actually a bad idea. Resources https://www.youtube.com/watch?v=jsWwFL_iqVM https://dev.mysql.com/doc/refman/8.0/en/replication-sbr-rbr.html https://engineering.fb.com/2021/07/22/data-infrastructure/mysql/ https://eng.uber.com/postgres-to-mysql-migration/
Support this podcast: https://anchor.fm/hnasr/support
This episode of the backend engineering show is sponsored by my friends at RedisLabs. I’m going to break this video into three sections, we will first define features that qualify a primary database? Then we will see if Redis actually check the boxes of a primary database and finally we explore the features of Redis that take it beyond a primary database. I was personally surprised by most of those.
Chapters
0:00 Intro
1:00 What Qualifies a primary database
3:00 Does Redis Check the boxes?
7:40 beyond Redis
Resources
Try Free: https://bit.ly/3hWr1Uj
Redis Advantages: https://bit.ly/3ztx2xw
Martin Fowler talking about Impedance Mismatch: https://bit.ly/36ZEOD8
Transactions: https://bit.ly/3wTMKAw
ACID 0.5 MM Ops/Seconds on AWS: https://bit.ly/3ruMB5s
Consistency and Durability: https://bit.ly/3wYNLr8
Watch and Rollbacks: https://redislabs.com/blog/you-dont-need-transaction-rollbacks-in-redis/#:~:text=Redis%20has%20a%20main%2C%20single,is%20required%20to%20implement%20WATCH
Redis Enterprise https://redislabs.com/redis-enterprise/advantages/
https://redis.io/topics/transactions#cas
https://redislabs.com/blog/you-dont-need-transaction-rollbacks-in-redis/#:~:text=Redis%20has%20a%20main%2C%20single,is%20required%20to%20implement%20WATCH.
https://docs.redislabs.com/latest/rs/concepts/data-access/consistency-durability/
Support this podcast: https://anchor.fm/hnasr/support
There was a two hours DNS outage on a company called Akamai that broke several services today July 22, 2021, https://appleinsider.com/articles/21/07/22/akamai-dns-problem-causing-wide-internet-issues
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
IIS (Internet Information Services) is Microsoft's Windows web server. It is feature-rich and very easy to enable. I have been using it for a long time but I noticed I never actually make a video about it.
In this video, I'll explore the IIS binding and explain all the options in that form.
I might make more videos in the future to explore different aspects of this web server.
Videos mentioned
Leaky abstractions https://youtu.be/4a3bI7AYsy4
HTTP/2 Limitations https://youtu.be/CUiBVTcgvBU
OCSP Stapling https://youtu.be/g08Omc1wi0s
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In today's show I go through the NodeJS Security Releases for the month of July 2021, lots of interesting vulnerabilities to discuss.
0:00 Intro
1:00 CVE-2021-22918 - libuv DNS Out of bounds Crash
3:40 CVE-2021-22921 - Node Windows installer Local Privilege Escalation
7:30 CVE-2021-27290 - ssri Regular Expression Denial of Service (ReDoS)
Resources
https://nodejs.org/en/blog/vulnerability/july-2021-security-releases/
https://hackerone.com/reports/1211160
https://snyk.io/vuln/SNYK-JS-SSRI-1085630
Support this podcast: https://anchor.fm/hnasr/support
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
In this episode, I’d like to discuss the methods of scaling CPU-bound or intensive workloads on the backend. This show will discuss the scaling of the workload on a single machine taking full advantage effectively of all its resources, then we will discuss horizontal scalability to multiple machines.
0:00 Intro
1:00 What do I mean by Scaling
3:20 CPU-Intensive/Bound Workload
6:00 Effective Scaling CPU-Bound Backends in Single Machine
12:00 How Hyperthreading can be useful
15:00 Horizontally Scale to multiple Machines
SO_REUSEPORT https://lwn.net/Articles/542629/
@Gary Explains Hyperthreading https://www.youtube.com/watch?v=mSZpDF-zUoI
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
MongoDB, Postgres, Microsoft SQL Server, or MySQL, or any other database manages concurrency control differently. There are two methods, pessimistic and optimistic, both have their pros and cons. Let explore how different databases implement this and what is the effect on performance/scalability.
This is often known as Optimistic vs pessimistic locking. Although I don't really like to use locking with this because it confuses the story.
0:00 Intro
2:20 What is Concurrency Control
6:00 Pessimistic Concurrency Control
14:50 Optimistic Concurrency Control
Resources
https://www.postgresql.org/docs/13/mvcc.html
http://source.wiredtiger.com/develop/architecture.html
https://docs.microsoft.com/en-us/troubleshoot/sql/performance/resolve-blocking-problems-caused-lock-escalation
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🔥 Members Only Content
https://www.youtube.com/playlist?list=UUMO_ML5xP23TOWKUcc-oAE_Eg
Support my work on PayPal
https://bit.ly/33ENps4
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
@MrRajputHacker @Th3Pr0xyB0y found critical universal XSS (an XSS that affects the entire browser, not just one page) on Microsoft Edge. They responsibly reported the bug and detailed it in their article. Let us discuss
Resources
https://cyberxplore.medium.com/how-we-are-able-to-hack-any-company-by-sending-message-including-facebook-google-microsoft-b7773626e447
https://docs.microsoft.com/en-us/DeployEdge/microsoft-edge-relnotes-security
https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-34506
https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-34475
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show I'll discuss the difference between b-tree and b+tree why they were invented, what problems do they solve, and the advantages and disadvantages of both. I'll also discuss the limitation of implementing b-tree over b+tree and how Discord ran into a memory limitation using b-tree Mongo.
Check out my udemy Introduction to Database Engineering course https://husseinnasser.com/courses Learn the fundamentals of database systems to understand and build performant backend apps
0:00 Data structure and algorithms
1:30 Working with large datasets
6:00 Binary Tree
8:30 B-tree
19:30 B+ tree
22:00 B-tree vs B+ tree benefits
25:00 MongoDB Btree Indexes Trouble
30:00 Summary
working with a billion row table (Members only)
https://youtu.be/wj7KEMEkMUE
indexing video
https://youtu.be/-qNSXK7s7_w
Discord moving from MongoDB to Cassandra
https://www.youtube.com/watch?v=86olupkuLlU
https://blog.discord.com/how-discord-stores-billions-of-messages-7fa6ec7ee4c7
MongoDB Indexes
https://docs.mongodb.com/manual/indexes/
Postgres Indexes
https://www.postgresql.org/docs/13/btree-implementation.html
btree code
https://www.cs.usfca.edu/~galles/visualization/BPlusTree.html
https://www.cs.usfca.edu/~galles/visualization/BTree.html
Support my work on PayPal https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑🏫
Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
SSH Wormable, Written in Bash and VERY hard to detect. Let’s discuss the DarkRadiation ☢️ Ransomware. This new ransomware is cut from a different cloth. Let us discuss
-
SSH Wormable
-
Encrypts with AES (OpenSSL)
-
It mutates so anti-viruses can’t catch it
-
Bash
-
Still under development
https://www.trendmicro.com/en_us/research/21/f/bash-ransomware-darkradiation-targets-red-hat--and-debian-based-linux-distributions.html
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
The ALPACA attack stands for application layer protocol confusion attack and discovered by a group of German computer scientists. Let us spend some time analyzing how this attack really works and how dangerous this is.
Resources
https://alpaca-attack.com/ALPACA.pdf
https://var.thejh.net/http_ftp_cross_protocol_mitm_attacks.pdf
https://github.com/RUB-NDS/alpaca-code
https://github.com/RUB-NDS/alpaca-code/blob/master/testlab/servers/files/nginx-attacker/html/upload/ftps.html
https://twitter.com/lambdafu/status/1404567396443164683
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
This Indian computer scientist uncovered a severe bug that allows anyone to view private content. Let’s see how he did it.
https://link.medium.com/goNhkJgv9gb
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
It is inevitable that a backend service will need to get restarted to pick up a new code change, configuration change, or get out of an invalid state. In this show, I'll discuss why do we need restart services and what alternative ways are there to get around it. And then I'll talk about how to achieve a zero-downtown restart and it is not straightforward as one might think.
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
CAP stands for Consistency, Availability, and Partition tolerance. Understanding the CAP theorem can help engineers make better design choices when building distributed systems. In this show, I will explain the CAP theorem and how you can use it to make tradeoffs in your backend design. You probably already are using the CAP theorem without even knowing.
Resources https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/#:~:text=The%20CAP%20theorem%20states%20that,to%20network%20partitions%20(P).
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Fastly, a very popular CDN went down and took down many services, let’s talk about what could have caused this.
Resources
https://status.fastly.com/incidents/vpk0ssybt3bj
https://twitter.com/fastly/status/1402221348659814411?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1402221348659814411%7Ctwgr%5E%7Ctwcon%5Es1_c10&ref_url=https%3A%2F%2Ftwitter.com%2F
https://apple.news/ASVV6TIepT8GPIEDjFbyNRg
Support my work on PayPal https://bit.ly/33ENps4 Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
On May 27, 2021, Klarna, a popular fintech company has suffered a serious exposure of personal data which caused a planned outage.
Resources
https://twitter.com/KezStew/status/1397845638956605440
https://www.klarna.com/us/blog/detailed-incident-report-incorrect-cache-configuration-leading-to-klarna-app-exposing-personal-information/
https://en.wikipedia.org/wiki/Klarna#cite_note-22
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Dominos Pizza India hacked and 13TB of customers' data is now on the dark web. https://www.indiatoday.in/technology/news/story/leaked-data-of-dominos-india-users-now-available-on-search-engine-created-by-hacker-1805595-2021-05-22 Support my work on PayPal https://bit.ly/33ENps4 Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support this podcast: https://anchor.fm/hnasr/support
QUIC is officially an IETF standard after a very long time. Is this going to replace the TCP protocol?
https://www.theregister.com/2021/05/31/quic_becomes_standard/
https://datatracker.ietf.org/doc/html/rfc9000
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I go through the lifetime of an HTTP request and why it is extremely difficult to cancel an HTTP request in a real production environment.
Support my work on PayPal https://bit.ly/33ENps4 Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode, I will discuss the different types of database replication and the pros and cons of each, streaming, binary, logical, synchronous, asynchronous, one-way and two-way replication. Stay tuned if you like databases and check out my database engineering course head to husseinnasser.com/courses for a discount code
Support my work on PayPal https://bit.ly/33ENps4 Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode, I will discuss Tor’s circuit Establishment which is the core of the Tor protocol.
https://svn-archive.torproject.org/svn/projects/design-paper/tor-design.pdf
https://youtu.be/gIkzx7-s2RU
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
This is an honest video about burnout and what a content creator can do to avoid it creating content on YouTube.
Support my work on PayPal https://bit.ly/33ENps4
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the backend engineering show, I'll discuss long polling technique of backend communication. I will also touch upon Polling and Pushing too and the pros and cons of each.
- Intro 0:00
- Polling 2:45
- Pushing 6:30
- Long Polling 18:00
- SSE 23:00
Support my work on PayPal https://bit.ly/33ENps4 Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the Backend Engineering show, we will go through the new features in Postgres 14. Here is a rundown of improvements made to the database platform with timestamps.
0:00 Intro
2:20 Performance
18:50 Data Types and SQL
23:00 Administration
32:30 Replication and Recovery
35:47 Security
Postgres 14 Beta 1
https://www.postgresql.org/about/news/postgresql-14-beta-1-released-2213/
Support my work on PayPal https://bit.ly/33ENps4
Become a Member on YouTube
🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode of the Backend Engineering Show, I’ll explain the OSI Model with an example. I start with the physical layer which is often ignored moved up to the application layer, presentation layer, session layer, transport layer, IP layer, and data link layer. I believe every software engineer should understand the OSI Model as it helps cement the fundamental understanding of networking applications.
Intro 0:00
Layer 1 Physical 4:00
Layer 7 Application 9:45
Layer 6 Presentation 11:30
Layer 5 Session 14:20
Layer 4 Transport 15:00
Layer 3 IP 18:00
Layer 2 Data Link 20:00
Summary 28:00
Support my work on PayPal https://bit.ly/33ENps4
Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In today's show, I discuss the nature of communications in database systems and how the pattern completely changed with 3-tier web architecture. I also discuss whether multiplexing protocols such as HTTP/2 and QUIC can help elevate some of the inefficiencies introduced.
-
Intro 0:00
-
Communication Protocols 2:00
-
3 Web Tier Architecture 8:00
-
Connection Pooling 14:50
-
Database Connection Multiplexing 23:40
-
Will Databases handle high concurrency 32:00
Support my work on PayPal https://bit.ly/33ENps4
Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
DST Root CA X3 Expires on September 2021, a ROOT certificate that signs Let's Encrypt Certificate authority, a very popular CA. In this video, I will discuss the ramification of this change.
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
🏭 Backend Engineering Videos in Order
https://backend.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Salesforce services went down as a result of a DNS update, let us discuss how can tiny DNS unavailability cause a severe outage of 5 hours.
From salesforce "On May 11, 2021, at approximately 21:08 Universal Coordinated Time (UTC), the Salesforce Technology team became aware of a service disruption across Salesforce production instances. The disruption impacted the ability for users to log into their Salesforce environments within the core Salesforce services, Marketing Cloud, Commerce Cloud, Government Cloud, Experience Cloud, Heroku, Pardot, and Vlocity. In addition, the status.salesforce.com Trust site was also unavailable, and customers were unable to log support cases. Some customers may have also experienced issues with Multi-Factor Authentication (MFA) during the incident. "
Resources
https://help.salesforce.com/articleView?id=000358392&type=1&mode=1
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫 Courses I Teach
https://husseinnasser.com/courses
🏭 Backend Engineering Videos in Order
https://backend.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this show, I go into detail on how HAProxy achieved 2 million HTTP requests per second. This is a very well-written article that discusses how the HAProxy team benchmarked the product on a 64 core ARM machine leading to over 2 million requests per second. There are many components and low-level points that I try to elaborate on, timestamps below.
0:00 Intro
2:40 Summary of the Article
11:55 Latency and Throughput in HAProxy 2.3 vs 2.4
21:00 How TCP Connections Affects Performance
28:00 Maximum Packets we can get in 100Gbps Network?
35:00 How 64 Cores are divided between workloads
40:00 Tail latencies HAProxy 2.3 vs 2.4
42:50 How TLS Affects Performance?
HAProxy Blog https://www.haproxy.com/blog/haproxy-forwards-over-2-million-http-requests-per-second-on-a-single-aws-arm-instance/
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this show, I discuss why we have 3 data models in database systems, OLTP (Online Transactional Processing) OLAP (Online Analytical Processing), and HTAP (Hybrid Transactional Analytical Processing). I’ll also explain the difference between them, the use of ETL tools (extract transform load) to load data from transactional to analytical databases, and what is the future of HTAP.
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Watch this if you are using IP Address validation in both NodeJS and Python, these two libraries strip leading zeros which can lead to server side request forgery. Let us discuss
Resources
https://www.bleepingcomputer.com/news/security/critical-netmask-networking-bug-impacts-thousands-of-applications/
https://www.bleepingcomputer.com/news/security/python-also-impacted-by-critical-ip-address-validation-vulnerability/
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this video, I’ll discuss the Pingback attack, a new clever attack that uses both DLL files through Oracle Component Interface (OCI.dll) and ICMP protocol to deliver commands between the victim machines and the command center.
Resources
https://thehackernews.com/2021/05/new-pingback-malware-using-icmp.html
https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol
https://en.wikipedia.org/wiki/Oracle_Call_Interface
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this podcast I’ll explain the message queues, the request response pattern and the publish subscribe pattern. I will also illustrate the main differences between them and when to use over another.
0:00 Intro
0:30 Message Queues in 60 Seconds
1:24 When to Use Message Queues?
14:33 Request Response Pattern
20:00 Request Response Pros & Cons
24:11 Publish Subscribe Pattern in 60 Seconds
25:13 Publish Subscribe Pattern
31:49 Publish Subscribe Pattern Pros and Cons
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
502 Bad Gateway is one of the most infamous errors on the backend, it usually means “hey something wrong with your backend server” but it doesn’t really give enough information.
In this video,
I’ll go through details on why proxies and gateways like NGINX and HAProxy should consider throwing more fine detailed HTTP error codes. 502 Bad Gateway The server was acting as a gateway or proxy and received an invalid response from the upstream server.
0:00 intro
3:45 What Causes a 502 Bad Gateway?
8:00 Cloudflare HTTP error codes
13:00 Security Implications
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode I’ll talk about how VPN works, networking, IPSec and will also discuss the benefits of VPN and what happens when a VPN is hacked?
* Intro 0:00
* How Networking Works? 2:20
* How VPN Works? 10:00
* VPN Benefits 17:50
* What happens when VPN is hacked 20:20
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
There is an ongoing situation with the Linux kernel community and the University of Minnesota Department of Computer Science & Engineering. We discuss this in this episode and I give my opinion
Support this podcast: https://anchor.fm/hnasr/support
Auth0 went down on April/20/2021 and this is the early report. Let us discuss. This incident affects: Auth0 US (PROD) (User Authentication, Machine to Machine Authentication, Multi-factor Authentication, Management API), Auth0 US (PREVIEW) (User Authentication, Machine to Machine Authentication, Multi-factor Authentication, Management API), and Management Dashboard (manage.auth0.com). 0:00 Update on Auth0 outage 6:00 Speculation of the outage https://auth0.com/blog/how-we-store-data-in-the-cloud-at-auth0/#Redis https://status.auth0.com/incidents/zvjzyc7912g5?u=v0zzz6jxvbv7
Support this podcast: https://anchor.fm/hnasr/support
Let us discuss the complexity behind this trojan hack, the multi-layer approach of hiding the RAT (remote access trojan) is absolutely genius.
https://en.wikipedia.org/wiki/HTML_Application https://en.wikipedia.org/wiki/Portable_Network_Graphics https://blog.malwarebytes.com/malwarebytes-news/2021/04/lazarus-apt-conceals-malicious-code-within-bmp-file-to-drop-its-rat/
Support this podcast: https://anchor.fm/hnasr/support
Few vulnerabilities in WhatsApp for Andriod discovered that allow an attacker to send an HTML file attachment full access to the user's media, voice notes, pictures, and eventually chat messages (through TLS session resumption keys). In this video, we will discuss the scope of this attack. The vulnerabilities have been patched by facebook.
Full article from CENSUS labs discussing in detail how to carry POC attack. https://census-labs.com/news/2021/04/14/whatsapp-mitd-remote-exploitation-CVE-2021-24027/
Support this podcast: https://anchor.fm/hnasr/support
Leaky abstractions occur when the consumer of the abstraction started asking questions about certain behavior which ends up with the need to understand the details behind the abstraction. Joel Spolsky coined this term and in this video I’d like to discuss this concept and provide few examples of my own experience towards leaky abstractions. Let us get on with the show.
6:00 Postgres Dead Tuples
7:25 MySQL Clustering
9:23 Axios HTTP Library
11:30 ORMs (N+1)
13:30 Beyond Abstractions
15:30 TCP
19:30 HTTP/2
27:00 Microservices
28:40 Index Only Scans Postgres
33:35 git
34:50 Summary
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Two weeks ago the PHP source code git server got hacked and two malicious commits were made to the source code. Since then the PHP maintainers identified the source of the hack, let us discuss
Support this podcast: https://anchor.fm/hnasr/support
Light episode today let's have some fun with Q&A, I collected some questions on Twitter and YouTube community and I'm going to attempt to answer them here.
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this episode, we will discuss NULLs in database systems. I’ll go through the following:
What is Null?
NULLs persistence
- Whether you store a 0 or 2 billion value in the field 32bit integer field it costs 32 bit
- when you store a NULL in 32 bit integer field we save 32 bit but add overheads
When NULLs are naughty
- Semantics and inconsistent result
- Select count(*). Includes nulls
- count(column) ignores nulls
- T is NULL returns the null rows
- T is NOT NULL returns not null rows
- T In (NULL) returns nothing
- T not in NULL returns nothing
- Some database don’t index nulls
When NULLs are useful
- I don’t have value , I don’t wish to provide a birthday
- not applicable field for certain use cases but not others fat tables (denormlization)
- Fat tables with many columns makes your rows longer which means fewer rows fit in your page (show pic).. NULLs help here .. that are NULL, it yields shorter rows, instead of storing a default 0 value
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
The open web application security project is a recognized entity that helps developers identify critical security vulnerabilities to build secure web applications. In this video I will go through the 10 vulnerabilities and explain each one and give examples and anecdotes from real life examples.
0:00 Building Secure Backends
2:30 Injection
4:50 Broken Authentication
6:43 Sensitive Data Exposure
11:00 XML External Entities (XXE)
13:45 Broken Access Control
17:00 Security Misconfiguration
19:00 XSS
22:45 Insecure Deserialization.
24:48 Using Components with Known Vulnerabilities.
26:00 Insufficient Logging & Monitoring.
Resources
https://owasp.org/www-project-top-ten/
Cards
2:50 SQL Injection https://www.youtube.com/watch?v=Azo9tDUtC9s
4:20 Best practices building REST https://www.youtube.com/watch?v=6zHWU7zBep0&list=PLQnljOFTspQUybacGRk1b_p13dgI-SmcZ&index=4
8:30 TLS playlist youtube.com/playlist?list=PLQnljOFTspQW4yHuqp_Opv853-G_wAiH-
15:00 HTTP Smuggling https://www.youtube.com/watch?v=PFllH0QccCs
19:22 XSS https://www.youtube.com/watch?v=pD6C1-zSxIM
25:10 OpenSSL Crash https://youtu.be/aDPQ0_MyRnc
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Caching is the hardest problem in building software, and having the browser cache is not any different. In this video, I'll discuss Jake Archibald's article
https://jakearchibald.com/2016/caching-best-practices/
0:00 Intro
2:00 Pattern 1: Immutable content + long max-age
5:40 Pattern 2: Mutable content, always server-revalidated
8:00 max-age on mutable content is often the wrong choice
12:20 CDN and Caching
Article
https://jakearchibald.com/2016/caching-best-practices/
https://twitter.com/jaffathecake
Support this podcast: https://anchor.fm/hnasr/support
Write Amplification Is a phenomenon where the actual writes that physically happen are multiples of the actual writes desired. In this episode, I'll discuss 3 types of write amplifications and their effects on performance and lifetime of storage mediums.
0:00 intro
2:00 Application write amplification
4:30 Database write amplification
9:30 SSD Disk write amplification
16:00 SSD hates BTrees
20:00 summary
Resources
https://en.wikipedia.org/wiki/Write_amplification
https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/
https://youtu.be/5Mh3o886qpg
Support this podcast: https://anchor.fm/hnasr/support
Microsoft Had an Outage on April 1st that is caused by DNS surge, let us discuss this. Bonus I’ll also discuss the outage that happened on March 18th cpu 100% utilization
RCA - DNS issue impacting multiple Microsoft services (Tracking ID GVY5-TZZ)
Summary of Impact: Between 21:21 UTC and 22:00 UTC on 1 Apr 2021, Azure DNS experienced a service availability issue. This resulted in customers being unable to resolve domain names for services they use, which resulted in intermittent failures accessing or managing Azure and Microsoft services. Due to the nature of DNS, the impact of the issue was observed across multiple regions. Recovery time varied by service, but the majority of services recovered by 22:30 UTC.
0:00 April/1st Outage - DNS Issue
13:30 March/18th Outage - CPU 100%
RCA
https://status.azure.com/en-us/status/history/
Support this podcast: https://anchor.fm/hnasr/support
Hey Hussein I have a 2 million row table used in my CRUD python app, I’m worried that as the table grow my inserts will slow down, should I consider sharding my database or partition the table? thank you I’m avid of simplicity in design if I can do it in one machine I’ll do it. Sharding/Partitioning are all great
inserts are fast, queries are slow 0:00 inserts can be slow 3:00
indexes/stored procedures
selects, updates, and deletes can be slow 12:00
add proper indexes.
simplicity wins, premature optimization is bad 15:20
crazy things that people say like microservices day 1 scares me
Support this podcast: https://anchor.fm/hnasr/support
Enabled by default, libcurl supports the use of TLS 1.3 session tickets to resume previous TLS sessions to speed up subsequent TLS handshakes.
When using a HTTPS proxy and TLS 1.3, libcurl can confuse session tickets arriving from the HTTPS proxy but work as if they arrived from the remote server and then wrongly "short-cut" the host handshake. The reason for this confusion is the modified sequence from TLS 1.2 when the session ids would provided only during the TLS handshake, while in TLS 1.3 it happens post hand-shake and the code was not updated to take that changed behavior into account.
4:00 http connect
https://curl.se/docs/CVE-2021-22890.html
Support this podcast: https://anchor.fm/hnasr/support
Two malicious commits were pushed to the php-srcGit repository maintained by the PHP team on their git.php.net server. The commits were found and reverted two hours after it was committed. PHP is moving to github as a result.
Article
https://www.bleepingcomputer.com/news/security/phps-git-server-hacked-to-add-backdoors-to-php-source-code/
Support this podcast: https://anchor.fm/hnasr/support
We have been told to take care of our private key that we use on backend servers without clear instructions as to what could happen when that key is leaked. In today’s backend engineering show I discuss exactly what could go wrong when your backend server private key is leaked. Let us discuss
Intro 0:00 What is a Certificate? 1:10 Where is the Private Key used? 4:10
TLS 1.2 with RSA 4:20 Why RSA no longer used 9:00 TLS 1.3 & TLS 1.2 Digital Signature 12:00
How often should you recycle Private Keys 19:00
Resources https://blog.cloudflare.com/advanced-certificate-manager/ https://heartbleed.com/ https://cabforum.org/ https://en.wikipedia.org/wiki/DigiNotar https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.sec.doc/q009960_.html
Support this podcast: https://anchor.fm/hnasr/support
6 months ago, Jake Miller released a blog article and python tool describing H2C smuggling, or http2 over cleartext smuggling. By using an obscure feature of http2, an attacker could bypass authorization controls on reverse proxies.
Sean managed to leverage Jack’s original research to bypass reverse proxy rules, lets discuss My original Video on Jack’s h2c smuggling https://youtu.be/B2VEQ3jFq6Q This article https://blog.assetnote.io/2021/03/18/h2c-smuggling/
Support this podcast: https://anchor.fm/hnasr/support
On Thursday, OpenSSL maintainers released a fix for two high severity vulnerabilities, let us discuss the impact.
- OpenSSL two major vulnerabilities 0:00
- why OpenSSL 1:00
- Bug 1 - Renegotiating TLS 1.2 (CVE-2021-3449) 3:50
- Bug 2 - Cert verification bypass (CVE-2021-3450) 8:42
- Update to OpenSSL 1.1.1k 12:30
Resources
https://www.openssl.org/news/vulnerabilities.html
https://arstechnica.com/gadgets/2021/03/openssl-fixes-high-severity-flaw-that-allows-hackers-to-crash-servers/
Support this podcast: https://anchor.fm/hnasr/support
Node JS Is single-threaded asynchronous non-blocking javascript runtime, but it's not always single-threaded there are occasions where nodejs uses multi-threading, so the questions we will try to answer in this video, when is nodejs single-threaded and when does it use multi-threading and how will that affect my app?
- Event Loop single thread, that really just loops for callbacks 0:00
- Threading in Node jS (libuv) 4:00
- used for
- IO/intensive
- DNS queries
- file system reads
- CPU intensive
- crypto
- compression
- process.env.UV_THREADPOOL_SIZE=1
Examples 8:00
Cluster Nodejs 16:00
Example 1
HTTP server return 1
HTTP server while 1
HTTP server with file system read async
HTTP server with file system read sync
HTTP server with fetch call to server (dns)
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
Slack started migrating from HAProxy to Envoy for their backend architecture, in this video, I’ll discuss their recent article when they moved the WebSockets portions, why they moved from HAProxy to Envoy and their production plans.
Resources
Article https://slack.engineering/migrating-millions-of-concurrent-websockets-to-envoy/
RFC8441 https://tools.ietf.org/html/rfc8441
3:15 Websockets Crash Course https://youtu.be/XgFzHXOk8IQ
9:50 HAProxy Runtime API https://youtu.be/JjXUH0VORnE
20:00 Slack Jan 4th outage https://www.youtube.com/watch?v=dhZ5--R42AM
23:00 RFC8441 Bootstrapping Websockets HTTP/2 https://youtu.be/wLdxC9gesBs
Support this podcast: https://anchor.fm/hnasr/support
In this video, I'll discuss RFC8441 bootstrapping WebSockets with HTTP/2 which I believe a critical protocol to allow WebSockets tunneling to scale on the backend. We will also discuss the current state of the art of Proxy and Backend Supports for this tech. Let us have a discussion.
0:00 Intro
3:00 WebSockets over HTTP/2
7:40 Proxy Supports
13:15 Browsers Supports
14:00 Summary
RFC 8441
Resources
RFC8441
https://tools.ietf.org/html/rfc8441#section-4
nginx support
https://trac.nginx.org/nginx/ticket/1992
haproxy support
https://github.com/haproxy/haproxy/issues/162
Chrome support
https://www.chromestatus.com/feature/6251293127475200
Firefox support
https://bugzilla.mozilla.org/show_bug.cgi?id=1434137
envoy support
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/upgrades
Support my work on PayPal
https://bit.ly/33ENps4
Become a Member on YouTube
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
🧑🏫Courses I Teach
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
In this video we will explore one of the most popular side attacks CRIME Compression Ratio Info-leak Made Easy) and the different ways to mitigate this. Intro 0:00 * HTTP/1.1 SPDY header compression 4:00* TLS compression * Response body attackers can’t inject 13:00 * Mitigations 14:10 * HPACK/QPACK * TLS Padding
Support this podcast: https://anchor.fm/hnasr/support
On March 15, 2021, users couldn’t sign in to Microsoft services the majority of the impact was with teams but other services were affected. A similar outage happened back in Sep 2020 (I covered it here https://www.youtube.com/watch?v=0ozri9APCv0&t=68s)
Microsoft 365 Service health status
https://twitter.com/MSFT365Status/status/1371546946263916545
Support this podcast: https://anchor.fm/hnasr/support
In today's show, I'll answer the question do backend connections max out? There are many aspects to this question and I want to try to tackle all of them. I'll also mention the efforts that the @Cloudflare and team are doing to improve the CONNECT with MASQUE protocol Tune in to the Backend engineering Show with Hussein Nasser on your fav podcast player.
Support this podcast: https://anchor.fm/hnasr/support
OVHcloud is Europe's largest cloud provider, with facilities across the region. They were hit with a big fire that completely destroyed an entire datacenter.
What happened? 0:00
What is the effect? 3:00
What OVH is going to do? 6:00
Resources
https://www.ovh.ie/news/press/cpl1786.fire-our-strasbourg-site
http://travaux.ovh.net/?do=details&id=49484
https://twitter.com/olesovhcom/status/1369504527544705025
Support this podcast: https://anchor.fm/hnasr/support
Firefox is implementing a feature that might end website tracking, let's get into how it works.
https://blog.mozilla.org/security/2021/02/23/total-cookie-protection/
https://hacks.mozilla.org/2021/02/introducing-state-partitioning/
Support this podcast: https://anchor.fm/hnasr/support
On the evening of March 8, GitHub invalidated all authenticated sessions on GitHub.com created prior to 12:03 UTC on March 8 out of an abundance of caution to protect users from an extremely rare, but potentially serious, security vulnerability affecting a very small number of GitHub.com sessions.
Let us discuss
https://github.blog/2021-03-08-github-security-update-a-bug-related-to-handling-of-authenticated-sessions/
Support this podcast: https://anchor.fm/hnasr/support
For the longest time, all browsers will always use HTTP in schemeless URLs (when HTTP or HTTPS is not specified). Chrome is flipping this with version 90
Chapters
-
HTTPS by Default 0:00
-
What happens Today 1:00
-
What will happen in Chrome 90 4:00
-
HSTS? 6:20
-
is HTTPS everywhere dead? 7:10
-
How to Enable 8:20
Video https://youtu.be/XrlfX0duLKQ
https://latesthackingnews.com/2021/03/01/google-will-launch-https-first-approach-with-urls-from-chrome-90
Support this podcast: https://anchor.fm/hnasr/support
MinIO, an S3 Compliant object-store suffered from a Server Side Request Forgery Vulnerability in early Feb 2021 which has been fixed quickly and addressed. In this video we go through the bug and what can we learn from it
Support this podcast: https://anchor.fm/hnasr/support
In this video, I discuss why QUIC will make a great communication protocol for databases and how it solves a critical problem with stateless web applications. Web applications use database connection pooling to establish database connections on the backend. But that creates other sorts of problems.
Support this podcast: https://anchor.fm/hnasr/support
Nodejs Updates are now available for v10.x, v12.x, v14.x and v15.x Node.js release lines for the following issues.
0:00 Intro
1:50 HTTP/2 Unknown Protocol
4:24 Localhost6 DNS Rebinding
6:55 Integer overflow OpenSSL
Resources
https://nodejs.org/en/blog/vulnerability/february-2021-security-releases/
Support this podcast: https://anchor.fm/hnasr/support
This is unacceptable and the entitlement towards open-source maintains needs to STOP!
Danial’s blog https://daniel.haxx.se/blog/2021/02/19/i-will-slaughter-you/
Support curl by becoming a backer https://opencollective.com/curl#backer
Support this podcast: https://anchor.fm/hnasr/support
Let us go through an absolutely fantastic article and journey of how a single change in HAProxy config drove this SRE into a frenzy to find out what went wrong. A fantastic read. https://about.gitlab.com/blog/2021/01/14/this-sre-attempted-to-roll-out-an-haproxy-change/?utm_medium=social&utm_source=linkedin&utm_campaign=blog
Support this podcast: https://anchor.fm/hnasr/support
From time to time I like to loiter on people’s GitHub Repos look through issues submitted and see if there are interesting hidden gems and bugs that would make a good lesson or learning experience and boy did I find one for you. This bug is caused in stripe-node code in AWS Lambda serverless environment where requests are failing intermittently. We discuss how AWS serverless container re-use can cause this and how stripe solved it. Resources https://github.com/stripe/stripe-node/issues/1040 Intermittent Error: write EPIPE when running stripe client in AWS Lambda · Issue #1040 · stripe/stripe-node · GitHub https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/
Support this podcast: https://anchor.fm/hnasr/support
XMPP or the Extensible Messaging and Presence Protocol originally named Jabber[1]) is an open communication protocol designed for instant messaging (IM), presence information, and contact list maintenance. it is used by almost all large messaging systems such as whatsapp, facebook, google talk and others. In this video we will go through XMPP architecture, explain how it works and then finallly show how to spin up an XMPP chat server and connect to it from node js.
Support this podcast: https://anchor.fm/hnasr/support
In this video I go over the critical timeouts on a Proxy system such as reverse proxy or load balancer and how can you configure each one to protect against attacks or outages. Nginx and HAProxy just a few proxies that you can configure to be load balancers.
Support this podcast: https://anchor.fm/hnasr/support
Guys this is absolutely genius and nuts! I have never seen anything like this before. This guy got access to paypal json and saw some private packages.. created public ones with a similar name and then made them do bad things, then thing because firewalls will shut those down.. he used DNS
DNS requests are practically safe so firewalls allow them
11:05 chrome root https://youtu.be/qpC1YH0FhuY
https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
Support this podcast: https://anchor.fm/hnasr/support
Reads and Writes don’t have to live in the same database, data model or even the same service. Let us discuss CQRS
no separation
one service that does read/write
partial separation
You can keep one service but backend have multiple connections with different users same database
full separation
Read services / write services
two databases OLAP / OLTP
Pros
scalability
security
Cons
complex and very hard to follow, what we see with microservices..
resources
https://martinfowler.com/bliki/CQRS.html
http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/
Support this podcast: https://anchor.fm/hnasr/support
In this video, I will discuss the new TLS extension Encrypted Client Hello which is a new mechanism to encrypt the entire client hello, very interesting and elegantly design but I have my few reservations and criticisms. Let us discuss.
Intro 0:00
Classic TLS with SNI 7:00
ESNI 9:30
ECH 12:30
Limitations and Problems 21:00
Let's say the backend server hosts example.com with the cert of example.com and let us call this the “real” SNI. To support ECH, the same server should also host a client facing cert, lets call it server-ech.com with corresponding server-ech.com cert. So your server IP address is hosting two domains. example.com and ( server-ech.com just to support ech)
So when you perform an oDoH/DoH query looking for example.com you will get back the IP address of example.com, (which is the same ip address as server-ech.com), you will also get the ephemeral public key of example.com from the HTTPSSVC DNS record. This will be used to encrypt the inner client hello, and finally, you will get a record of the front-facing server domain name which is server-ech.com.
The client prepares the TLS ECH, it builds the outer client hello with the SNI value as server-ech.com, and the inner client hello with the real SNI (example.com) along side ALPN and other stuff..
It then uses the public key of example.com retrieved from the HTTPS DNS record to encrypt the inner client hello. The client sends the ECH.
Server receives the ECH, and attempts to decrypt the inner client hello with the corresponding ephemeral private key, if it succeeds it then completes the server hello and key exchange as normal.
if it fails to decrypt the inner client hello with its own private key that means the public key used, was stale, outdated, bad etc.. . The client then uses the outer client hello and it checks, do I have server-ech.com, yes I do have a cert for that, let me just finish the key exchange and server hello with that instead, and btw, here is my REAL public key for example.com so you can use it for future ECHs. the public key is going to be encrypted for sure (that was something not clear in the article) because its TLS 1.3 and things are immediately encrypted from the server. The client then finishes the handshake and finds out the public key and immediately terminate the connection and re-establishes it with brand new encrypted inner client hello that uses the new server public key which we know this time its going to work ..
I can image getting into an infinite loop if the server accidentally gave a wrong public key. also some cons of ECH comes to mind is the larger client hello + the additional backend management to host that client facing server and cert. Also another question, what If I’m hosting 100 domains on my single IP address, do I get a single ephmeral public key for ECH? or is it per domain? and how does my server know what private key to use to decrypt, does it just try them one by one? or is there an indication to which public key was used to encrypt the inner client hello.. (I’m guessing there is )
https://blog.cloudflare.com/encrypted-client-hello/
Support this podcast: https://anchor.fm/hnasr/support
MySQL is clustered by default on the primary key which means inserts have to be ordered, let us discuss why UUID (random in nature) has bad performance in MySQL and whether postgres wins here. We will also explain why Sequential Writes are Faster than Random in MYSQL and
https://www.percona.com/blog/2019/11/22/uuids-are-popular-but-bad-for-performance-lets-discuss/
Support this podcast: https://anchor.fm/hnasr/support
This is a very interesting article that I encourage you to read it as it has lots of useful lessons in postgres. Using partial indexes, full vacuum, dropping unused indexes and much more helped this company save 70G worth of disk space.
https://hakibenita.com/postgresql-unused-index-size
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss my approach of learning new technology and how I break it down so I understand it. Hope it helps
Support this podcast: https://anchor.fm/hnasr/support
The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data in a distributed file system. IPFS uses content-addressing to uniquely identify each file in a global namespace connecting all computing devices.
Intro 0:00
Why IPFS? 2:00
Explain the original web model and the limitation
-
Content addressing instead of location addressing
-
decentralized content distributed among peers
Content 3:30
-
Content is hashed as CID
-
Content is immutable each update generates new CID
-
Content addressing
Routing 4:30
-
Distributed Hash Table (DHTs) maps CID / Peer IP address
-
DHT server hosts content and DHT
Publishing Content 6:30
-
New Content that you want to share on ipfs
-
hash the content creating new CID
-
Update your local DHT CID / your ip address
-
DHT will be updated to all the content peer (NOT the CONTENT)
-
People searching for your CID will be connected to you and only you.
Consuming Content 8:48
-
ipfs client (dht client) want to consume Ipfs://cid/
-
ipfs client consults its local DHT table to see where this CID is located, gets back a collection of IP addresses
-
client connects to some or all the peers found hosting that CID
-
client downloads chunks of the content from each peer so it speeds up
-
Once the client has the content it is now also updating its local DHT table that it now also hosts that CID (if it supports being a DHT server)
-
New updated DHT is propogated across peer
IPFS Overview (Digrams) 11:30
Demo 13:45
More Information 18:30
Immutable Content
-
if Content gets updated changes URI how do I inform the user?
-
hash the public key of the user instead and share that
Brand new Client/server
-
I know nothing about the network (Bootstraping)
-
you will be bootstrapped with a collection of ip addresses to start you up.
More
-
IPFS gateway
-
IP Name server
-
Solve content
-
Deleting Content( once other node hosts it no way to delete it from their network)
NAT traversal
Resources
https://datatracker.ietf.org/meeting/interim-2020-dinrg-01/materials/slides-interim-2020-dinrg-01-sessa-an-overview-of-the-interplanetary-file-system-ipfs.pdf
https://www.youtube.com/watch?v=K4Usud4g4iY&feature=youtu.be&t=1008
https://twitter.com/hnasr/status/1353548949945163776?s=21
https://docs.ipfs.io/conce
🎙️Listen to the Backend Engineering Podcast
https://husseinnasser.com/podcast
🏭 Backend Engineering Videos
https://backend.husseinnasser.com
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
🏰 Load Balancing and Proxies Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC
🏛️ Software Archtiecture Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9
📩 Messaging Systems
https://www.youtube.com/playlist?list=PLQnljOFTspQVcumYRWE2w9kVxxIXy_AMo
Become a Member
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support me on PayPal
https://bit.ly/33ENps4
Join our Thriving Backend Community on Discord
https://discord.com/invite/CsFbFce
Stay Awesome,
Hussein
Support this podcast: https://anchor.fm/hnasr/support
It looks like digital certificates and other certificate authorities issued by Spanish certificate authority Camerfirma will stop working in Chrome 90, in April. https://www.zdnet.com/article/google-bans-another-misbehaving-ca-from-chrome/ https://wiki.mozilla.org/CA:Camerfirma_Issues
Support this podcast: https://anchor.fm/hnasr/support
I explain why and when SELECT * can become expensive.
Support this podcast: https://anchor.fm/hnasr/support
David Schuts, a security researcher earned $5000 dollars in Google VRP by finding a Backend YouTube API that leaks Private Video Thumbnails. let us discuss how did he do that. Resources https://bugs.xdavidhu.me/google/2021/01/11/stealing-your-private-videos-one-frame-at-a-time/ Twitter @xdavidhu https://twitter.com/xdavidhu
Support this podcast: https://anchor.fm/hnasr/support
A vulnerability in Google Feedback component in postMessage allowed this security researcher to find a way to hijack private screenshots https://blog.geekycat.in/google-vrp-hijacking-your-screenshots/ https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage
Support this podcast: https://anchor.fm/hnasr/support
Brave supports IPFS (InterPlanetary File System) which is a protocol designed to replaced HTTP as a decentralized alternative. This allows users to host and consume Resources https://brave.com/brave-integrates-ipfs/ https://ipfs.io/#how
Support this podcast: https://anchor.fm/hnasr/support
The current FTP implementation in Google Chrome has no support for encrypted connections (FTPS), nor proxies. Usage of FTP in the browser is sufficiently low that it is no longer viable to invest in improving the existing FTP client. In addition more capable FTP clients are available on all affected platforms. Google Chrome 72+ removed support for fetching document subresources over FTP and rendering of top level FTP resources. Currently navigating to FTP URLs result in showing a directory listing or a download depending on the type of resource. A bug in Google Chrome 74+ resulted in dropping support for accessing FTP URLs over HTTP proxies. Proxy support for FTP was removed entirely in Google Chrome 76. Remaining capabilities of Google Chrome’s FTP implementation are restricted to either displaying a directory listing or downloading a resource over unencrypted connections. We would like to deprecate and remove this remaining functionality rather than maintain an insecure FTP implementation.
Resources
https://bugs.chromium.org/p/chromium/issues/detail?id=333943
https://docs.google.com/document/d/1JUra5HnsbR_xmtQctkb2iVxRPuhPWhMB5M_zpbuGxTY/edit#
https://www.chromestatus.com/feature/6246151319715840
Support this podcast: https://anchor.fm/hnasr/support
On Jan 4th 2021, Slack experienced a global outage that prevented customers from using the service for nearly 5 hours.
Slack has released the Root cause analysis incident report which I’m going to summarize in the first part of this video. After that Ill provide a lengthy deep dive of the incident so make sure to stick around for that.
If you are new here, I make backend engineering videos and also cover software news, so make sure to Like comment and subscribe if you would like to see more plus it really helps the channel, lets jump into it.
So This is an approximation of Slack’s architecture based on what was the described in the reports. Clients connects to load balancers, load balancers distribute requests to backend servers and backend servers finally make requests to database servers which is powered by mysql through vitess sharding. All of those are connected by routers in cross boundary network.
Around 6AM jan 4 , the cross network boundary routers setting between LB and backend and backend to DB started to drop packets.
This lead to the load balancers slowly marking backends as unhealthy and removing them from the fleet Which compounded the amount of requests
The number of failed requests eventually triggered the provisioning service to start spinning an absurdly large number of backend servers
However the provisioning service couldn’t keep up with the huge demand and shortly started to time out for the same networking reasons and eventually ran out of maximum open file handles.
Eventually Slack’s cloud provider increased the networking capacity and backend servers went back to normal around 11 AM PST
This was a summary of the slack outage, Now set back, grab your favorite beverage and lets go through the detailed incident report!
0:00 Outage Summary
2:00 Detailed Analysis Starts
5:20 The Root Cause
30:00 Corrective Actions
Support this podcast: https://anchor.fm/hnasr/support
In this video I go through the new most exciting features in HAProxy, one of my favorite proxies.
HAProxy 2.3 adds exciting features such as forwarding, prioritizing, and translating of messages sent over the Syslog Protocol on both UDP and TCP, and OpenTracing SPOA, Stats Contexts, SSL/TLS enhancements, an improved cache, and changes in the connection layer that lay the foundation for support for HTTP/3 / QUIC.
Resources
https://www.haproxy.com/blog/announcing-haproxy-2-3/
0:00 Intro
2:00 Connection Improvements
5:40 Load Balancing
11:36 Cache
15:00 TLS Enhancements
Support this podcast: https://anchor.fm/hnasr/support
In this video I go through the new features in Apache Kafka 2.7, it is very interesting to see the amount of work Apache Kafka is doing to get closer to removing ZooKeeper
* [KAFKA-9893] - Configurable TCP connection timeout and improve the initial metadata fetch * [KAFKA-9915] - Throttle Create Topic, Create Partition and Delete Topic Operations * [KAFKA-10054] - Add TRACE-level end-to-end latency metrics to Streams * [KAFKA-10259] - KIP-554: Add Broker-side SCRAM Config API * [KAFKA-10338] - Support PEM format for SSL certificates and private key https://dist.apache.org/repos/dist/release/kafka/2.7.0/RELEASE_NOTES.html https://blogs.apache.org/kafka/entry/what-s-new-in-apache4 Confluence Mobile - Apache Software Foundation
Support this podcast: https://anchor.fm/hnasr/support
I discussed this in many of my videos, the need for a database that natively supports a multiplexing protocol such as QUIC, gRPC or HTTP/2 in order to allow multiple isolated clients to make requests to the database without taking the overhead of establishing multiple connections.
Resources https://www.infoq.com/news/2021/01/eventstoredb/ https://developers.eventstore.com/clients/dotnet/5.0/streams/#writing-to-a-stream
Support this podcast: https://anchor.fm/hnasr/support
A lot of you guys ask me this question. “I have experience but not sure how to show it, how do I build my backend portfolio such that I can get hired in my dream job. “ Building a backend portfolio takes time and effort and In this video I will be discussing 9 tools that you can add to your backend portfolio 0:00 Intro
Live Projects 1:50 System Design Documents 3:45 Architectural/System Design Diagrams 5:45 UX/UI (in case of frontend) 7:13 Papers 8:30 Books 9:50 Blog articles 10:55 Videos 12:44 Podcast 14:45 Summary 15:45
Support this podcast: https://anchor.fm/hnasr/support
WhatApp has updated their terms of usage and privacy policy which caused many users to move to other platforms. This video will be a detailed report of their privacy policy, what they collect and what they can collect and see.
https://www.whatsapp.com/legal/privacy-policy
https://cdn.arstechnica.net/wp-content/uploads/2021/01/Image-from-iOS.png
cards
1:30 end to end
16:30 Samesite cookie
0:00 WhatsApp New Privacy
4:00 Your Account Information
5:30 Your Messages
12:15 Your Connections
13:00 Automatically Collected Information
17:45 Summary
Support this podcast: https://anchor.fm/hnasr/support
NodeJS Jan 2021 released its security update and its time to go through them! Resources https://nodejs.org/en/blog/vulnerability/january-2021-security-releases/use-after-free in TLSWrap (High) (CVE-2020-8265) HTTP Request Smuggling in nodejs (Low) (CVE-2020-8287) OpenSSL - EDIPARTYNAME NULL pointer de-reference (CVE-2020-1971)
Support this podcast: https://anchor.fm/hnasr/support
On Jan 4th 2021 7:14 PST All Slack services went down. This video is an early report of the incident and speculation of what might have caused this outage. We still don’t know what caused the outage, we will wait for the full incident report from slack and I'll make a video once that's up. https://status.slack.com/
Support this podcast: https://anchor.fm/hnasr/support
The SolarWinds hack is one of the largest highly coordinated and intelligent attempt to hit enterprise companies. In this video, I briefly explain how smart this is.
Support this podcast: https://anchor.fm/hnasr/support
While working on a Postgres docker container executing some queries I noticed that my index-only scan query is hitting the heap which it shouldn't. After digging deep I found that it's the shared memory that docker allocates by default. Defaults are never enough, very interesting train of thought that I thought I’d share with you The Blog I found that helped me find it https://blog.makandra.com/2018/11/investigating-slow-postgres-index-only-scans/
Support this podcast: https://anchor.fm/hnasr/support
Let us discuss what I'm excited for in Backend Tech in 2021 and answer your great questions
Support this podcast: https://anchor.fm/hnasr/support
In this video, I go through my process of how I design and architect full software from A-Z. This is part of a Twitter thread that you guys seem to enjoy so I decided to make a video on the topic. Although the spec I generate is usually Backend oriented this is applicable for all software. Twitter thread https://twitter.com/hnasr/status/1339021983195918337?s=20
Support this podcast: https://anchor.fm/hnasr/support
In this video, I go through how I overcome procrastination as a software engineer. What is Procrastination? 0:00How to Defeat * Reward based system - a reward after achieving 1:20 discipline, remembering why started this, your goal 3:16 Professional - I need to do the work and ship 6:10
Support this podcast: https://anchor.fm/hnasr/support
2020 retrospective
- Intro 0:00
- Goals 0:50
- Teaching vs Documenting 4:30
- Channel Growth 7:40
- 2021 10:50
Support this podcast: https://anchor.fm/hnasr/support
0:00 Intro
1:00 Summary of the Outage
4:00 Detailed Analysis of the Incident Report
On Dec 14 2020 Google across the globe suffered from an outage that lasted 45 minutes nobody could access most of Google services.
Google has released a detailed incident report discussing the outage, what caused it, technical details on their internal service architecture and what did they do to mitigate and prevent this from happening in this in the future
In this video, I want to take a few minutes to summarize the report and then go into a detailed analysis. You can find youtube chapters to jump to the interesting part of the video. pick your favorite drink, sit back relax, and enjoy. Let's get started.
let's start with an overview of how the google id service works, the client connects to Google authentication service to get authenticated or retrieve account information
The account information is stored in a distributed manner between the different service ids for redundancy.
when an update is made to an account on the leader node, the existing data in all nodes are marked as outdated, this is done for security reasons. Let’s say you updated your credit card info, privated your profile or deleted a comment, it is extremely dangerous to serve that outdated information. This was the key to the outage.
The updated account is then replicated based on Paxos Consensus protocol.
The user id service has a storage quota controlled by an automated quota management solution when the storage usage of the service changes.
the quota is maintained accordingly either reduced or increased based on the demand ..
So What Exactly Happened that caused the outage?
In October 2020, google migrated their quota management to a new system and registered the id service with the new system.
however some parts of the old system remained hooked up specifically the parts regarding the reading of the service usage. And because the service is registered to the new system, the old qouta system reported 0 usage as it should. So when the new quota manement asked its service for its usage it was incorrectly reporting 0.
Nothing happened for a while since there was a grace period, but that period expired on December
Thats when the new quota system kicked and saw the id service with 0 usage and started reducing the qouta for the id service down .. you are not using it why waste?
The quota kept reducing until the service had no space left.
This has caused updates to the leader node to fail, which caused all data to go out of date in all nodes which in turn escalated globally to what we have seen.
Resource
https://status.cloud.google.com/incident/zall/20013
Support this podcast: https://anchor.fm/hnasr/support
The Backend Engineering Show Live with Hussein Nasser episode 10 we discuss many great questions!! Indexing Woes, The Secret to Backend Interviews, What is on my Bookshelf? Backend Engineering Show
Support this podcast: https://anchor.fm/hnasr/support
Exposed Postgres instances are being ssh into and used as a botnet to mine bitcoin, in this video we explain how does that happens. the trick is the COPY FROM PROGRAM command
Support this podcast: https://anchor.fm/hnasr/support
At 3:47 am PST almost all google services went down including, gmail, youtube, drive, docs, meet, nest , google maps and many more. It took close to an hour to bring them back up. We still don’t know what caused this outage, in this video we will try to make sense from what we have gathered so far. A detailed analysis video will follow once we get a response from google
symptoms
- Could not sign in to google (account not found)
- Could not authenticate if you already have a token
- Services not require authentication also fails to retrieve certain account information (profile, YouTube comments)
- Guess -> Borg Service that provide authentication, authorization went down, a fix? Storage qouta issue?
https://twitter.com/googlecloud/status/1338493015145504770
https://www.tomsguide.com/news/gmail-and-youtube-down-several-google-services-are-not-working-latest-updates
Google down? Realtime status, issues and outages for the UK | Downdetector
Google Workspace Status Dashboard
Support this podcast: https://anchor.fm/hnasr/support
When the private key of a matching public key that belong to a certificate is leaked, an attacker can intercept server hello, use their own dh parameters sign it with the stolen private key and ship it to the client effectively doing MITM. This is extremely dangerous and we have no way in the client to know a MITM has happened.
That is why a certificate sometimes has to be revoked, and in this video I’m going to discuss those revocation techniques.
0:00 How Certificate Works
3:00 Certificate Revocation List
4:10 OCSP
7:00 OCSP Stapling
Support this podcast: https://anchor.fm/hnasr/support
In The Backend Engineering Show Live, we discuss Impostor syndrome and Staying Motivated in software engineering field.
Support this podcast: https://anchor.fm/hnasr/support
Oblivious DoH is a technology that separates IP addresses from queries, so that no single entity can see both at the same time.
Cloudflare, Apple & Fastly worked on this and did a good write-up of the tech, we discuss it in this video
https://blog.cloudflare.com/oblivious-dns/
https://blog.cloudflare.com/oblivious-dns/
Support this podcast: https://anchor.fm/hnasr/support
Oracle introduces a Game Changer Feature in MySQL that allows for OLAP & OLTP workloads in a single database. This is huge let us discuss
https://www.oracle.com/emea/news/announcement/oracle-announces-mysql-database-service-with-integrated-analytics-engine-2020-12-03.html
https://dev.mysql.com/doc/mysql-analytics/en/mysql-analytics-introduction.html
0:00 Intro
1:40 History of ETL
7:00 How Kafka Helped Data Warehouse
8:20 How RAPID Solves this
11:14 MySQL Database Service Analytics Engine (RAPID) Architecture
14:00 Loading Data
18:00 Summary
Support this podcast: https://anchor.fm/hnasr/support
In The Backend Engineering Show Live we will have a casual Q&A around QUIC Outline HTTP/1.1 Trouble HTTP/2 Trouble QUIC Handshake QUIC 0RTT HPACK vs QPACK Why HTTP/3
Support this podcast: https://anchor.fm/hnasr/support
In AWS re-invent, Amazon announced open sourcing Babelfish for PostgreSQL, a SQL Server-compatible end-point for PostgreSQL to make PostgreSQL fluent in understanding communication from apps written for SQL Server. Let us discuss what is this technology and whether if its gonna really move developers away form Microsoft SQL Server to Postgres
Resources
https://aws.amazon.com/blogs/opensource/want-more-postgresql-you-just-might-like-babelfish/
Chapters
0:00 Intro
1:30 Postgres vs SQLServer
5:20 What is Babelfish?
9:40 Why Babelfish May not Work
10:06 Will Babelfish Includes everything?
11:46 BabelFish is an Extra Layer
13:35 What REALLY is Babelfish?
15:00 Performance
Support this podcast: https://anchor.fm/hnasr/support
Attackers have been disguising trojans and other malicious codes in post-install NPM packages and developers have been targeted. This is another incident from NPM. NPM needs to step up and solve this problem https://www.zdnet.com/article/malicious-npm-packages-caught-installing-remote-access-trojans/
Support this podcast: https://anchor.fm/hnasr/support
AWS US east-1 experienced an outage Nov-25-2020. Amazon has updated us with summary detailing what exactly happened to amazon Kinesis that caused the outage let us discuss it
0:00 Intro
1:00 Tldr (diagram)
7:30 Detailed Analysis of What Happened
25:00 Why Cognito Went Down
31:20 Why CloudWatch Went Down
33:20 Why Lambda and AutoScaling Went Down
35:50 Why EventBridge, Elastic Kubernetes and Container Service Went Down
38:00 Why Service Status Went Down
40:00 Summary
https://aws.amazon.com/message/11201/
Support this podcast: https://anchor.fm/hnasr/support
AWS US east-1 experienced an outage yesterday, let us discuss what could have been the problem and what amazon did to solve it
—
Latest Update (6:23 PM PST): We’d like to provide an update on the issue affecting the Kinesis Data Streams API, and other dependent services, within the US-EAST-1 Region. We have now fully mitigated the impact to the subsystem within Kinesis that is responsible for the processing of incoming requests and are no longer seeing increased error rates or latencies. However, we are not yet taking the full traffic load and are working to relax request throttles on the service. Over the next few hours we expect to relax these throttles to previous levels. We expect customers to begin seeing recovery as these throttles are relaxed over this timeframe.
Resources
https://www.datacenterdynamics.com/en/news/aws-us-east-1-region-suffers-errors-and-outages-impacting-its-status-page/
https://disqus.com/by/disqus_DZeJlmjjGx/
https://downdetector.com/status/amazon/
https://www.youtube.com/watch?v=I9v-fCz0HZY&feature=youtu.be
Support this podcast: https://anchor.fm/hnasr/support
In this video I go through the three type of caching and how it is kept in sync.
0:00 Intro
0:30 What is Caching?
1:20 Spatial Cache
3:30 Temporal Cache
5:00 Distributed cache
6:30 Write-Through Cache
8:00 Write-Back Cache
Support this podcast: https://anchor.fm/hnasr/support
According to Sumo Logic's research, Redis is now officially the most popular database in 2020 on AWS cloud deployment. Let us discuss some of the reasons why the in-memory database became so popular
https://www.theregister.com/2020/11/23/redis_the_most_popular_db_on_aws/
Support this podcast: https://anchor.fm/hnasr/support
Felix Wilhelm of Google Project Zero found an injection Vulnerability affecting GitHub Actions and Workflow Commands specifically related to setting malicious environment variables by parsing STDOUT
Resources
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
https://bugs.chromium.org/p/project-zero/issues/detail?id=2070&can=2&q=&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary&cells=ids
https://www.zdnet.com/article/google-to-github-times-up-this-unfixed-high-severity-security-bug-affects-developers/
Support this podcast: https://anchor.fm/hnasr/support
A botnot caught scanning the web for .ENV file and harvesting Credentials, API Keys and Passwords. Let us discuss https://www.zdnet.com/article/botnets-have-been-silently-mass-scanning-the-internet-for-unsecured-env-files/
Support this podcast: https://anchor.fm/hnasr/support
The Envoy Proxy fixed two zero day vulnerabilities, from Envoy groups :
We are announcing the fixes for two zero days that were identified today:
- Crash in UDP proxy when datagram size is > 1500. This can happen if either MTU > 1500 or if fragmented datagrams are forwarded and reassembled: https://github.com/envoyproxy/envoy/pull/14122. This issue was already under embargo and a new issue was opened in public GitHub.
- Proxy proto downstream address not restored correctly for non-HTTP connections: https://github.com/envoyproxy/envoy/pull/14131. This issue was opened publicly recently but the security implications were not clear at the time. This will affect logging and network level RBAC for non-HTTP network connections.
Resources
https://groups.google.com/g/envoy-security-announce/c/aqtBt5VUor0
0:00
0:20 UDP Proxy Crash
2:15 Incorrect Downstream Remote Address
Support this podcast: https://anchor.fm/hnasr/support
In this live stream we have a Q&A about Communication Protocols in the Backend , enjoy.
We Talk about Masque, WebTransport, WebSockets, TCP, UDP and more
Support this podcast: https://anchor.fm/hnasr/support
A group of researchers from UC Riverside and Tsinghua University announced a new attack against the Domain Name System (DNS) called SAD DNS (Side channel AttackeD DNS). In this video I explain this attack
0:00 Intro
1:00 What is DNS?
3:10 Original DNS Poisoning
6:30 DNS Poisoning with Fragmentation Attack
9:30 ICMP Explained
13:00 DNS Poisoning with ICMP Error Messages
Resources
https://blog.cloudflare.com/sad-dns-explained/
https://www.saddns.net/
https://bit.ly/3lHTn45
https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol
Support this podcast: https://anchor.fm/hnasr/support
There is a new Protocol called WebTransport, it sets to solve some limitations in WebSockets, the question is will this completely replaces WebSockets? I’ll leave this question to you guys. Let us discuss
Resources
https://www.youtube.com/watch?v=jTBM9CDO_Wk&feature=youtu.be
https://datatracker.ietf.org/doc/draft-kinnear-webtransport-http2/
https://datatracker.ietf.org/doc/draft-vvv-webtransport-http3/
https://datatracker.ietf.org/doc/draft-vvv-webtransport-quic/
Support this podcast: https://anchor.fm/hnasr/support
FireFox Enables HTTPS Only Mode, let us discuss
https://blog.mozilla.org/security/2020/11/17/firefox-83-introduces-https-only-mode/
What does it mean?
- Death of HSTS?
- No more plugins
- Will it ever become default? (Government sites unencrypted, backward compatible)
Support this podcast: https://anchor.fm/hnasr/support
A Node.js application that allows an attacker to trigger a DNS request for a host of their choice could trigger a Denial of service by getting the application to resolve a DNS record with a larger number of responses. (CVE-2020-8277) I discuss this attack in this video and whether you should fix it. Impacts: * Versions 12.16.3 and higher on the 12.x release line * Versions 14.13.0 and higher on the 14.x release line * All versions of the 15.x release line Resources https://nodejs.org/en/blog/vulnerability/november-2020-security-releases/#:~:text=Denial%20of%20Service%20through%20DNS,a%20larger%20number%20of%20responses. Code Fix https://github.com/nodejs/node/commit/022899e1d5
Support this podcast: https://anchor.fm/hnasr/support
In this Livestream we discuss the following topic
Facebook moving to QUIC
https://engineering.fb.com/2020/10/21/networking-traffic/how-facebook-is-bringing-quic-to-billions/
Multiplexed Application Substrate over QUIC Encryption (masque)
https://datatracker.ietf.org/wg/masque/about/
KIP500, Kafka removing ZooKeeper
https://www.confluent.io/blog/how-to-prepare-for-kip-500-kafka-zookeeper-removal-guide/
DotNET 5
https://devblogs.microsoft.com/dotnet/announcing-net-5-0/
Support this podcast: https://anchor.fm/hnasr/support
HTTP CONNECT Method allows the client to create a tunnel through a proxy to forward any free-form content through it. Let us discuss why do the pros and cons of this
0:00 Intro
1:45 HTTP Proxy
5:50 HTTPS Proxy
9:40 HTTP CONNECT
14:15 HTTP CONNECT Chaining
16:10 Pros & Cons of CONNECT
23:20 MASQUE
Resources
https://tools.ietf.org/html/rfc7231#section-4.3.6
Multiplexed Application Substrate over QUIC Encryption (masque)
https://datatracker.ietf.org/wg/masque/about/
What if you want to connect to the secure site?
cards
18:40 http/2 clear smuggling
https://www.youtube.com/watch?v=B2VEQ3jFq6Q
17:40 layer 4 proxy
https://www.youtube.com/watch?v=aKMLgFVxZYk
16:50 WebSockets
https://www.youtube.com/playlist?list=PLQnljOFTspQUGjfGdg8UvL3D_K9ACL6Qh
21:00 HTTP/2
https://www.youtube.com/watch?v=fVKPrDrEwTI
Support this podcast: https://anchor.fm/hnasr/support
HTTP/2 Push is being removed since it is very difficult to implement and has no added value. Let us discuss
Resource https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/vOWBKZGoAQAJ
Video https://www.youtube.com/watch?v=uAfNRJJ_BrA
Support this podcast: https://anchor.fm/hnasr/support
An article from 2016 which caused lots of discussions in the software engineering community. We bring it back and open old wounds and discuss it again.
0:00 Intro
3:00 Problems with Architecture of Postgres
4:00 Postgres on-Disk Format
9:45 Replication
13:19 Write Amplification
16:44 Replication Bandwidth
21:16 Data Corruption
24:00 Replica MVCC
31:30 Postgres Upgrades
33:00 MySQL on-Disk Format
37:00 MySQL Replication
40:00 Connection Handling
https://eng.uber.com/postgres-to-mysql-migration/
https://news.ycombinator.com/item?id=12166585
Support this podcast: https://anchor.fm/hnasr/support
WebRTC (Web Real-Time Communication) is a free, open-source project that provides web browsers and mobile applications with real-time communication (RTC) via simple application programming interfaces (APIs).
In this video I go through webrt and discuss all the concepts of WebRTC in details. We will learn about NAT, STUN, TURN, ICE, SDP, Signaling and we will show a demo too! Finally we will talk about the pros & cons
0:00 Intro
3:44 WebRTC Overview
11:17 NAT
16:54 NAT Translation Methods
26:20 STUN
33:30 TURN
35:00 ICE
38:00 SDP
40:52 Signaling
43:30 WebRTC Demo
1:00:00 WebRTC Pros & Cons
1:04:00 Bonus WebRTC Content !
tags
webrtc, Web Real-Time Communication, webrtc video, webrtc samples, webrtc tutorial, webrtc , interactive connectivity establishment ice , session description protocol, session initiation protocol, peer to peer, getUserMedia, RTCPeerConnection
Support this podcast: https://anchor.fm/hnasr/support
Live Q&A discussion of different backend engineering topics
Support this podcast: https://anchor.fm/hnasr/support
Facebook move to QUIC from TCP was not smooth but they did see some improvement in all their apps. Let us discuss this
Article
https://engineering.fb.com/networking-traffic/how-facebook-is-bringing-quic-to-billions/
0:00 Intro
3:00 What is QUIC?
10:45 Facebook Backend
14:30 FaceBook Frontend
15:20 GraphQL
17:00 The Trouble with QUIC
23:00 Static & Video Content
25:15 Instagram App
26:00 QUIC Future
Support this podcast: https://anchor.fm/hnasr/support
SonaType detected a Malware in NPM registry imitating to be Twilio package that opens a reverse connection to a remote server and allows attacker to access your local machine content. Let us discuss
Since this command is unix specific it won’t work on Windows
https://blog.sonatype.com/twilio-npm-is-brandjacking-malware-in-disguise
Resources
SSH Tunneling https://youtu.be/N8f5zv9UUMI
Ngrok https://www.youtube.com/watch?v=pR2qNnVIuKE
Support this podcast: https://anchor.fm/hnasr/support
According to ZDNET "Chrome will soon have its own dedicated certificate root store" Let us discuss what that might mean to privacy https://www.zdnet.com/article/chrome-will-soon-have-its-own-dedicated-certificate-root-store/
Support this podcast: https://anchor.fm/hnasr/support
Ever heard of HTTP Smuggling? will this is smuggling a TCP packet into an HTTP body so that it can be interpreted by the router to open internal ports to your machine. NAT Slipstreaming was discovered by Samy Kamkar, Article and research by @SamyKamkar https://samy.pl/slipstream/
Support this podcast: https://anchor.fm/hnasr/support
Let us have a casual chat about TLS, Security, Certificates and more
Support this podcast: https://anchor.fm/hnasr/support
In this live stream I discuss all about Database ACID one by one and we also answer interesting questions!
Enjoy!
Watch stream here https://www.youtube.com/watch?v=QCKZ3VZ87Qo&feature=youtu.be
Support this podcast: https://anchor.fm/hnasr/support
My Thoughts on Full Stack Engineering
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss the new Uber Backend Architecture that they deployed to process payments and jobs and orders. https://youtu.be/mL0fzj7e6WU Revolutionizing Money Movements at Scale with Strong Data Consistency https://eng.uber.com/money-scale-strong-data/
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain the differences between Column vs Row Oriented Database Storage how efficient each method is, and their pros & cons
0:00 Intro
2:50 Row-Oriented Database
15:30 Column-Oriented Database
26:30 Pros & Cons
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain how Network Engineers can move to be a Backend Engineer by capitalizing on their skills in networking. Network Engineers can build great and improve the communication protocols that Backend Engineers use for service to service communication. Network Engineers can also be specialized in Proxies, Reverse Proxies, Load Balancers and Caching Layers.
Support this podcast: https://anchor.fm/hnasr/support
Software Engineering is overwhelming and hard, I discuss how to ease up that burden and make it fun here.
0:00 Intro
2:20 Learning Software Engineering
17:55 Bugs
23:30 Design Activities
28:50 Summary
🎙️Listen to the Backend Engineering Podcast
https://husseinnasser.com/podcast
🏭 Backend Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
🏰 Load Balancing and Proxies Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC
🏛️ Software Archtiecture Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9
📩 Messaging Systems
https://www.youtube.com/playlist?list=PLQnljOFTspQVcumYRWE2w9kVxxIXy_AMo
Become a Member
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support me on PayPal
https://bit.ly/33ENps4
Stay Awesome,
Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this live stream we will keep the theme about WebSockets, scaling, layer 7 proxying vs layer 4 proxying in websockets , interesting problems and just chatting!
0:00 Intro
13:00 Stream Starts
21:30 WebSockets
1:08:00 Securing WebSockets
1:13:00 Scaling WebSockets
Watch on YouTube
https://www.youtube.com/watch?v=brKJFkLQWYo&feature=youtu.be
Support this podcast: https://anchor.fm/hnasr/support
In this video, I will go through Database Partitioning and explain it in details, pros and cons with a demo using PostgreSQL.
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain how both Postgres and MySQL store their indexes and their effect on reads vs writes. Let us discuss
0:00 Intro
1:00 Tables
2:00 Indexes
3:20 Indexing in Postgres
5:00 Indexing in MySQL
6:35 What Happens on Update on Postgres
7:20 What Happens on Update on MySQL
9:00 Reads on Postgres
9:40 Reads on MySQL
Support this podcast: https://anchor.fm/hnasr/support
In this Video I discuss Discord WebRTC and Voice chat backend architecture, it is a very interesting article, let us discuss it
https://blog.discord.com/how-discord-handles-two-and-half-million-concurrent-voice-users-using-webrtc-ce01c3187429
0:00 Intro
3:30 WebRTC As a Choice?
8:00 ReWriting WebRTC
11:20 Backend Architecture
20:00 Failover
26:00 My Thoughts
Support this podcast: https://anchor.fm/hnasr/support
PostgreSQL 13 Has Some Performance Boosts! Let us discuss it!
Postgres 13 has been released and it has some interesting features how about we discuss it!
Resources
Press-release https://www.postgresql.org/about/featurematrix/detail/341/
Feature Matrix https://www.postgresql.org/about/featurematrix/
“The PostgreSQL Global Development Group today announced the release of PostgreSQL 13, the latest version of the world’s most advanced open source database.
PostgreSQL 13 includes significant improvements to its indexing and lookup system that benefit large databases, including space savings and performance gains for indexes, faster response times for queries that use aggregates or partitions, better query planning when using enhanced statistics, and more.
Along with highly requested features like parallelized vacuuming and incremental sorting, PostgreSQL 13 provides a better data management experience for workloads big and small, with optimizations for daily administration, more conveniences for application developers, and security enhancements.
"PostgreSQL 13 showcases the collaboration and dedication of our global community in furthering the abilities of the world's most advanced open source relational database," said Peter Eisentraut, a PostgreSQL Core Team member. "The innovations that each release brings along with its reputation for reliability and stability is the reason why more people choose to use PostgreSQL for their applications."
PostgreSQL, an innovative data management system known for its reliability and robustness, benefits from over 25 years of open source development from a global developer community and has become the preferred open source relational database for organizations of all sizes.”
Support this podcast: https://anchor.fm/hnasr/support
In this Live stream, I discuss HTTP and answer interesting questions about HTTP, TLS, UDP, QUIC, WebSockets and more from the community
watch the vod here https://www.youtube.com/watch?v=J6G8DdLgdJ4
Support this podcast: https://anchor.fm/hnasr/support
In this Article Stanislav Vishnevskiy elegantly discusses why Discord moved from MongoDB to Apache Cassandra, the challenges they faced, limitations of both Mongo & Cassandra. Well written article let us discuss
https://blog.discord.com/how-discord-stores-billions-of-messages-7fa6ec7ee4c7
Support this podcast: https://anchor.fm/hnasr/support
I stumbled upon this interesting article titled Scaling with common sense, the author goes into different topics of the bad practices of pre-mature scaling and optimization specifically with regards to microservices and k8. Let us discuss
Resources
https://zerodha.tech/blog/scaling-with-common-sense/
0:00 Intro
3:00 Comparisons are almost always meaningless.
5:30 Scaling starts with well built software.
8:50 Eat healthy and exercise daily.
10:15 KISS, don’t be afraid, and boring better cool.
12:00 The bottleneck is almost always the database.
13:40 RDBMS works, almost always.
15:00 Everyone forgets to index.
17:30 Don’t use an RDBMS. What?
19:40 Networking/IO is really hard. Network as little as possible.
21:20 Connections are hard. Connect little, pool much.
25:00 Latency is THE metric.
26:10 The Internet is the Wild Wild West.
28:40 Caching is a silver bullet, almost.
29:00 Dumb caching is best caching.
29:40 Some application state may not be bad.
31:20 HTTP APIs can be E-Tagged (304) too.
34:12 Allocation is expensive.
37:40 Multi-threading and concurrency are necessary, but hard.
38:30 Some technologies are genuinely slow. Use fast technologies.
39:30 Scaling horizontally, vertically, and “enterprisely”.
40:30 Human impediment.
42:20 My Thoughts on Microservices
🎙️Listen to the Backend Engineering Podcast
https://husseinnasser.com/podcast
🏭 Backend Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
🏰 Load Balancing and Proxies Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC
🏛️ Software Archtiecture Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9
📩 Messaging Systems
https://www.youtube.com/playlist?list=PLQnljOFTspQVcumYRWE2w9kVxxIXy_AMo
Become a Member
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support me on PayPal
https://bit.ly/33ENps4
Stay Awesome,
Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain how I was browsing a website (displate) and was served the exact same thing on my Instagram feed
Support this podcast: https://anchor.fm/hnasr/support
Watch the Video here
https://youtu.be/76sgBHUl7iI
Alessandro Ghedini wrote an interesting article discussing how DNS queries can help speed up HTTPS and HTTP/3 communication let us discuss this article
https://blog.cloudflare.com/speeding-up-https-and-http-3-negotiation-with-dns/
3:30 TLS video https://www.youtube.com/watch?v=AlE5X1NlHgg
4:57 HSTS https://www.youtube.com/watch?v=kYhMnw4aJTw
8:15 http/2 c smuggling https://www.youtube.com/watch?v=B2VEQ3jFq6Q
15:30 ESNI https://www.youtube.com/watch?v=t0zlO5-NWFU
Support this podcast: https://anchor.fm/hnasr/support
On September 28 2020 Microsoft 365 Service went down, what caused it? and what did Microsoft did to solve it.
Resources
https://twitter.com/msft365status/status/1310696819135901696?s=21
https://status.office.com/
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_account-mso_imobile-mso_o365b/error-aadsts90033/d2ba3ef2-fe85-411d-b4fe-5c44df1e121c
https://answers.microsoft.com/en-us/msoffice/forum/msoffice_o365admin-mso_other-mso_o365b/aadsts90033-a-transient-error-has-occurred-please/8117f18e-9de9-46d7-9505-1708b2a1c732
https://docs.microsoft.com/en-us/answers/questions/35944/aadsts90033-a-transient-error-has-occurred-please.html
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain how communication protocols are built on top of each other and how each protocol can be stateful or stateless. We need to understand this in order to know which protocol to use to build the right backend.
Support this podcast: https://anchor.fm/hnasr/support
It looks like Russia is submitting a proposal to block TLS 1.3, ESNI, DoH , DoT, let us discuss Resources https://www.zdnet.com/article/russia-wants-to-ban-the-use-of-secure-protocols-such-as-tls-1-3-doh-dot-esni/ https://en.wikipedia.org/wiki/SORM
Support this podcast: https://anchor.fm/hnasr/support
What scared me about this bug is how EASY it is to execute, no MITM, no special software.. its all exploiting of existing software.. Let us discuss The SSDP engine in Firefox for Android (68.11.0 and below) can be tricked into triggering Android intent URIs with zero user interaction. This attack can be leveraged by attackers on the same WiFi network and manifests as applications on the target device suddenly launching, without the users' permission, and conducting activities allowed by the intent. Resources https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol https://gitlab.com/gitlab-com/gl-security/security-operations/gl-redteam/red-team-tech-notes/-/tree/master/firefox-android-2020 https://twitter.com/init_string https://twitter.com/LukasStefanko/status/1307013106615418883
Support this podcast: https://anchor.fm/hnasr/support
Google Bot is now attempting to use HTTP/2 to crawl the web
Ever since mainstream browsers started supporting the next major revision of HTTP, HTTP/2 or h2 for short, web professionals asked us whether Googlebot can crawl over the upgraded, more modern version of the protocol.
Today we're announcing that starting mid November 2020, Googlebot will support crawling over HTTP/2 for select sites.
Article
https://webmasters.googleblog.com/2020/09/googlebot-will-soon-speak-http2.html
-
Intro 0:00
-
What is a Crawler 1:06
-
Current Crawler uses h1 2:00
-
Crawler now uses h2 4:40
-
Less connections, slightly high CPU usage 6:30
-
Opting out of h2 crawling 9:00
-
FAQ 11:40
Http/2 playlist 0;50 https://www.youtube.com/playlist?list=PLQnljOFTspQWbBegaU790WhH7gNKcMAl-
http/2 not cheap 7;00 https://www.youtube.com/watch?v=GriONb4EfPY
lucid chart 9;50 https://www.youtube.com/watch?v=gejfT1h6LBo
h2c smuggling 18:30 https://www.youtube.com/watch?v=B2VEQ3jFq6Q
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain how I got better at communicating my thoughts, opinions and ideas and how making content on YouTube Channel actually helped. I still have a long way but I found that making content and continuously trying to get better at delivering the core point helps.
Support this podcast: https://anchor.fm/hnasr/support
There was a recent article titled Stop Using React which spawned lots of discussions. I want to give my thoughts on this article and React in general. Resources https://dev.to/ender_minyard/why-you-should-stop-using-react-g7c
https://timkadlec.com/remembers/2020-04-21-the-cost-of-javascript-frameworks/ 0:00 Intro
3:20 Its Slow
8:30 Its expensive
12:00 its inaccessible
14:00 React goes against the web
18:00 made by facebook
21:00 my thoughts
Support this podcast: https://anchor.fm/hnasr/support
In this video, I discuss 3 methods to work with tables of billion rows. This discussion is inspired by a comment on my YouTube video on Twitter System Design right here https://www.youtube.com/watch?v=gfq-LG9ZJQA&lc=UgyYbm5889dW0XtKhsV4AaABAg
Chapters
Intro 0:00
-
Brute Force Distributed Processing 2:30
-
Working with a Subset of table 3:35
2.1 Indexing 3:55
2.2 Partitioning 5:30
2.3 Sharding 7:30
- Avoid it all together (reshuffle the whole design) 9:10
Summary 11:30
Support this podcast: https://anchor.fm/hnasr/support
It seems like these incidents are very common and not sure why ElasticSearch in particular. Let us discuss https://www.zdnet.com/article/leaky-server-exposes-users-of-dating-site-network/
Support this podcast: https://anchor.fm/hnasr/support
Some of you asked me how to deal with stress at my work and my content creation here on YouTube I share my thoughts with you
Intro 0:00
Stress from Work 2:40
Feeling down with no clear reason 7:13
Feeling anxious overwhelmed with stuff to learn 14:55
Pandemic 21:14
Support this podcast: https://anchor.fm/hnasr/support
Unimog is a layer 4 load balancer built for Cloudflare scale. Cloudflare has written a great blog about it so let us discuss this technology. Video: https://youtu.be/Q0irm6xzNNk Resources https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/ 0:00 Intro 3:33 Layer 4 vs Layer 7 Load Balancers 7:00 Anycast 13:45 Packet Forwarding 23:30 XDP and Network stack 26:45 Maintaining established connection 31:00 Edge Computing 32:00 UDP Routing 33:00 Unimog Summary 34:00 Open Source Software 36:00 K8 Rant 40:00 Conclusion Cards 6:30 L4 vs L7 proxying https://www.youtube.com/watch?v=aKMLgFVxZYk 12:30 vip https://www.youtube.com/watch?v=85XY7H2JPbs 13:30 tcp handshake https://www.youtube.com/watch?v=bW_BILl7n0Y&t=5s
Support this podcast: https://anchor.fm/hnasr/support
Jake Miller a security researcher discovered a serious flaw in proxies that allow h2c clear text upgrade and bypass proxy rules. Let us discuss
Thanks to @Textras for sending this article!
https://twitter.com/thebumblesec/status/1303305853525725184?s=21
https://labs.bishopfox.com/tech-blog/h2c-smuggling-request-smuggling-via-http/2-cleartext-h2c?hs_amp=true
Support this podcast: https://anchor.fm/hnasr/support
Apache Kafka is an interesting software, every design decision the team makes perfect sense. I decided to dive deep into the discussion of the consumer group concept which is underrated and talk more about it.
0:00 Intro
1:24 Messaging Systems Explained
3:30 Partitioning
4:30 Pub/Sub vs Queue
6:55 Consumer Group
10:00 Parallelism in Consumer Group
10:30 Partition awareness in Consumer Group
11:30 Achieving Pub/Sub with Consumer Group
14:00 Head of Line blocking in Kafka
Support this podcast: https://anchor.fm/hnasr/support
Someone asked me a question and I felt its interesting to make a video about, is there a limit to the maximum number of TCP connections a Client can make to the server? If there is what is it? and how does that make sense in all the configurations?
0:00 Intro
1:00 Is there a Max Connection Limit?
4:30 64K Connection Limit Explained
7:20 Max Connections on Reverse Proxies and Max Connections
14:30 How does Router get around Max Connections?
7:00 3 million whatsapp https://www.youtube.com/watch?v=vQ5o4wPvUXg
10:25 envoy https://www.youtube.com/watch?v=40gKzHQWgP0&
12:50 google https://www.youtube.com/watch?v=CUiBVTcgvBU
14:00 Active Active https://www.youtube.com/watch?v=d-Bfi5qywFo
Support this podcast: https://anchor.fm/hnasr/support
Doordash the food delivery service has built an asynchronous task processing backend with Celery and RabbitMQ. They are having lots of outages and problems. Let us discuss how they solved their problem by moving to Apache Kafka.
Very well written article.
Resource
https://doordash.engineering/2020/09/03/eliminating-task-processing-outages-with-kafka/
https://www.rabbitmq.com/connections.html#high-connection-churn
Support this podcast: https://anchor.fm/hnasr/support
Application-LayerProtocol Negotiation (ALPN) is a Transport Layer Security (TLS) extension that allows the application layer to negotiate which protocol should be performed over a secure connection in a manner that avoids additional round trips and which is independent of the application-layer protocols. It is needed by secure HTTP/2 connections, which improves the compression of web pages and reduces their latency compared to HTTP/1.x. The ALPN and HTTP/2 standards emerged from development work done by Google on the now withdrawn SPDY protocol.
https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation
1:30 TCP Handshake
1:40 TLS
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain when to use TCP vs UDP as a communication protocol on your backend app. I go through the advantages and disadvantages of UDP I also discuss the protocol within the context of Chatting System, Multiplayer game, and building a browser and a web server 0:00 Intro 2:00 UDP 3:00 TCP 6:00 UDP vs TCP for Building a Chatting System 9:20 UDP vs TCP for Building a Multiplayer game 15:30 UDP vs TCP for Building a Browser and WebServer 19:11 Summary
Support this podcast: https://anchor.fm/hnasr/support
HTTP/2 is a protocol that allows multiplexing which can be very beneficial however HTTP/2 is not always cheap and might not be a good choice for your backend.
-
Intro 0:00
-
What is HTTP/2 ? 1:30
-
HTTP/2 Pros 5:10
-
HTTP/2 Advantages on Browsers 5:30
-
HTTP/2 Advantages on Reverse Proxy Connection Pooling 9:20
-
HTTP/2 Problem 11:00
Google Talk https://www.youtube.com/watch?v=xxN4FfwaANk
Support this podcast: https://anchor.fm/hnasr/support
Envoy is an open-source L7 proxy and communication bus Originally built at Lyft to move their architecture away from a monolith.
In this video, I want to go through the following
-
What is Envoy? 0:00
-
Current & Desired Architecture 0:48
-
Envoy Architeture 3:00
-
DownStream/Upstream 7:30
-
Clusters 9:19
-
Listeners 10:50
-
Network Filters 11:50
-
Connection Pools 13:45
-
Threading Model 18:34
-
Example 21:25
-
Show the 4 apps 24:30
-
Install Envoy Brew 26:00
-
https://www.getenvoy.io/install/envoy/macos/
-
Envoy as a Layer 7 Proxy 27:30
-
Proxy to all 4 backend NodeJS services 28:00
-
Split load to multiple backends (app1/app2) 40:00
-
Block certain requests (/admin) 45:30
-
Envoy as a Layer 4 Proxy (tcp router) 47:50
-
Create DNS record 54:00
-
Enable HTTPS on Envoy (lets encrypt) 55:30
-
Enable HTTP/2 on Envoy 1:03:00
-
Disable 1.1/1.0 Enable TLS 1.2 and TLS 1.3 ONLY on Envoy 1:04:30
-
SSL Labs test 1:06:40
-
Summary 1:07:24
Config
https://github.com/hnasr/javascript_playground/tree/master/envoy
Resources
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/intro/terminology
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/connection_pooling#arch-overview-conn-pool
🏭 Backend Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT
💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2
🛰 Network Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr
🏰 Load Balancing and Proxies Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC
🐘 Postgres Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj
🚢Docker
https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r
🧮 Programming Pattern Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe
🛡 Web Security Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv
🦠 HTTP Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO
🐍 Python Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX
🔆 Javascript Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM
👾Discord Server https://discord.gg/CsFbFce
Become a Member
https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Support me on PayPal
https://bit.ly/33ENps4
Become a Patreon
https://www.patreon.com/join/hnasr?
Stay Awesome,
Hussein
Support this podcast: https://anchor.fm/hnasr/support
Envoy Proxy is an L3/L4 Proxy that is designed to be service mesh, In this video, I discuss my initial thoughts about the product, design choices, and much more. The actual full video on Envoy will be coming soon stay tuned.
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain Postgres Vacuum while I walk my dog.
Support this podcast: https://anchor.fm/hnasr/support
Cloudflare is doing fantastic job to the web community security and backend engineering. This latest fix is awesome it is the ability to auto-tune window size buffer when it comes to uploading HTTP/2 traffic. Article https://blog.cloudflare.com/delivering-http-2-upload-speed-improvements/ 4:15 slow tcp start https://www.youtube.com/watch?v=rgPcxg8gjho&t=1s 5:40 HTTP crash course https://www.youtube.com/watch?v=0OrmKCB0UrQ&t=4s 6:20 TCP crash course https://www.youtube.com/watch?v=qqRYkcta6IE 12:20 Bandwidth bideo https://www.youtube.com/watch?v=6Tf80mbhyAQ
Support this podcast: https://anchor.fm/hnasr/support
SameSite Cookie Lax is interesting and we are finding new exceptions everyday. Let us discuss this one where lax cookies will be sent on POST request as long as the cookies are fresh (2 minutes)
Resources
https://www.chromestatus.com/feature/5088147346030592
Support this podcast: https://anchor.fm/hnasr/support
This code was introduced for a user experience ending up taking 50% of the traffic on DNS Root server. Sorry I was touching my hair a lot just took a shower lol. With regards to this article I want to ask you guys a question, Chrome put this feature in order to improve the user experience but it ended up having a huge cost. Did you ever make a choice between performance and user experience? which one usually wins for you? would love to know your opinion Resources https://arstechnica.com/gadgets/2020/08/a-chrome-feature-is-creating-enormous-load-on-global-root-dns-servers/ https://docs.microsoft.com/en-us/deployedge/microsoft-edge-policies#dnsinterceptionchecksenabled https://news.ycombinator.com/item?id=24231857 https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/
Support this podcast: https://anchor.fm/hnasr/support
Windows 95 was a great operating system, wrote so many apps on top of it and played so many games too. Join me as I discuss this
https://www.theverge.com/21398999/windows-95-anniversary-release-date-history
Support this podcast: https://anchor.fm/hnasr/support
GraphQL was born to solve a major limitation in REST API, but the cost of GraphQL and barrier to entry is high. Vulcain addresses REST limitations by introducing HTTP/2 push. Is a simpler alternative? let us discuss Learn about Vulcain here https://github.com/dunglas/vulcain
Support this podcast: https://anchor.fm/hnasr/support
Chrome is enabling Raw TCP and UDP from the Browser, this is big news! let us discuss the implication, security and benefit for us backend engineers.
resources
https://www.theregister.com/2020/08/22/chromium_devs_raw_sockets/
raw tcp spec https://github.com/WICG/raw-sockets
Support this podcast: https://anchor.fm/hnasr/support
In this video, I discuss the different concurrency control at database transactions, specifically the pessimistic vs optimistic concurrency control. and the pros and cons of each. 0:00 Intro 3:00 concurrency Control 5:30 Pessimistic concurrency Control 9:20 Optimistic concurrency Control Resources https://en.wikipedia.org/wiki/Optimistic_concurrency_control https://www.baeldung.com/java-jpa-transaction-locks https://docs.oracle.com/javaee/7/api/javax/persistence/OptimisticLockException.html https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use https://www.2ndquadrant.com/en/blog/postgresql-anti-patterns-read-modify-write-cycles/
Support this podcast: https://anchor.fm/hnasr/support
Jenkins has just released a statement that there is a potential bug (CVE-2019-17638) where an attacker can steal content from other legitimate requests. In this video, I describe the bug and why being a web server is difficult. 2;00 HTTP Smuggling https://www.youtube.com/watch?v=PFllH0QccCs 7;50 multi-Threading https://www.youtube.com/watch?v=0vFgKr5bjWI&t=1s Resources https://nvd.nist.gov/vuln/detail/CVE-2019-17638 https://en.wikipedia.org/wiki/Jetty_(web_server) https://www.jenkins.io/security/advisory/2020-08-17/
Support this podcast: https://anchor.fm/hnasr/support
Some of you asked me to talk about how I learned to speak good English on my YouTube videos. I wanted to make a video on the fact that It wasn't always that easy and I struggled a lot and still struggling with English. I have immigrated to the United State in 2015 In this video, I want to explain my struggle with the English language as an Arabic native speaker and how I got better but still, I need lots of work. Speaking Tech English is definitely easier than Social.
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain in details what are third party cookies and how do they work and explain the same site property that google changed 0;30 SameSite 6;00 CORS 6;22 Content Security Policy https://www.youtube.com/watch?v=nHOuakyHX1E https://blog.chromium.org/2020/01/building-more-private-web-path-towards.html
Support this podcast: https://anchor.fm/hnasr/support
What if questions sometimes cripple the system design for backend application and complicate the end product. I discuss this in this video. Stay Awesome Hussein Nasser
Support this podcast: https://anchor.fm/hnasr/support
Light video today discussing my interviewing skills for software engineering positions. I always ask this open ended question and allow the candidate to go free.
Support this podcast: https://anchor.fm/hnasr/support
YAGNI stands for You aren’t gonna need it and its a pillar in extreme programming, in this video I discuss this philosophy within the context of Backend Engineering. https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it * Extreme Programming Rob Jefferies * You Aren’t Gonna Need it .. true but only if the design is well defined * But I am going to need it * Waterfall vs Agile
Support this podcast: https://anchor.fm/hnasr/support
SNI or server name indication is a TLS Extention that indicate which server/host/domain the client want to communicate with. This is to allow for hosting of multiple websites on the same public static ip address. For the longest time all ISPs used SNI to block hosts and websites, China is now blocking the encrypted version SNI.
0:00 Intro
2:00 DNS and DOH
3:30 SNI
6:30 ESNI
11:00 The Block
The ESNI and DOH stops this but China want
https://www.zdnet.com/article/china-is-now-blocking-all-encrypted-https-traffic-using-tls-1-3-and-esni/
https://tools.ietf.org/html/draft-ietf-tls-esni-07#section-3.2
Support this podcast: https://anchor.fm/hnasr/support
HSBC moving from 65 relational databases to a single Global MongoDB, that might be true but it's misleading as not all systems are moved.
Resources
https://diginomica.com/hsbc-moves-65-relational-databases-one-global-mongodb-database
Why some devs don't use MongoDB
https://news.ycombinator.com/item?id=23507197
https://news.ycombinator.com/item?id=19497817
https://news.ycombinator.com/item?id=18366385
https://news.ycombinator.com/item?id=23270429
Support this podcast: https://anchor.fm/hnasr/support
SameSite Cookie attribute has been introduced to secure the web and only send cookies within a trusted and safe context. SameSite Cookies Video https://www.youtube.com/watch?v=aUF2QCEudPo
Support this podcast: https://anchor.fm/hnasr/support
A great change by Chrome team, downloading files on HTTP insecure channels is insecure. Let us discuss Resource https://www.zdnet.com/article/google-to-block-some-http-file-downloads-starting-with-chrome-83/
Support this podcast: https://anchor.fm/hnasr/support
Homomorphic encryption is a form of encryption allowing one to perform calculations on encrypted data without decrypting it first. The result of the computation is on an encrypted form, when decrypted the output is the same as if the operations had been performed on the unencrypted data. In this video I go through what homomorphic encryption is and how it will change software engineering forever.
0:00 Intro
2:25 What is Encryption?
3:55 Why we can’t always encrypt?
- TLS terminator proxies to looks
- Search and Analyse data
- Database indexing, functions
8:30 Meet Homomorphic encryption
-Perform operations on encrypted data
-We decrypt the data to process it..
-Imagine no more TLS termination! In proxies
13:20 IBM FHE toolkit code demo
Country csv database, encrypted and then search
21:00 Pros & Cons of Homomorphic Encryption
Resources
https://github.com/IBM/fhe-toolkit-linux/blob/master/GettingStarted.md
https://www.youtube.com/playlist?list=PL0VD16H1q5IOEQuRdgRVt1M8uQSbpVzTb
https://arstechnica.com/gadgets/2020/07/ibm-completes-successful-field-trials-on-fully-homomorphic-encryption/?comments=1&start=80
https://github.com/IBM/fhe-toolkit-linux/blob/master/GettingStarted.md
cards
1;05 encryption video https://www.youtube.com/watch?v=Z3FwixsBE94
3;45 tls https://www.youtube.com/playlist?list=PLQnljOFTspQW4yHuqp_Opv853-G_wAiH-
7;41 steve gipson https://www.youtube.com/watch?v=UKvK76Rnqus
Support this podcast: https://anchor.fm/hnasr/support
Dropbox has fully migrated their proxying needs from nginx to envoy proxy. They wrote this detailed article about the reasons and motivations and problems faced during migration. It is an interesting read. Let us discuss https://dropbox.tech/infrastructure/how-we-migrated-dropbox-from-nginx-to-envoy Migrating Dropbox from Nginx to Envoy | Hacker News 0:00 Intro 4:20 What is this Article about? 6:10 Performance 11:15 Security 14:28 Missing Features in NginX 23:24 Migration was NOT Seamless 33:00 Summary
Support this podcast: https://anchor.fm/hnasr/support
The twitter hackers got caught and the case is closed, what have we learned? what really happened? and how can we prevent such attacks in the future, can homomorphic encryption help?
Resource
https://www.theverge.com/2020/7/31/21349920/twitter-hack-arrest-florida-teen-fbi-irs-secret-service
0:00 Intro
2:00 Summary of July 15
3:30 How the attack really happened?
8:00 How the attackers got caught?
10:45 How could this be prevented?
12:15 Can homomorphic encryption help?
Support this podcast: https://anchor.fm/hnasr/support
This is an analysis of the #percona benchmark article comparing MySQL & mariaDB performance with regards to SSD disks with NVMe vs SATA controllers. Pretty neat 0:00 Intro 1:00 MariaDB vs MySQL 2:15 SATA vs NVMe 4:30 SATA Benchmark 7:30 NVMe Benchmark 10:00 SSD & B-Trees 11:20 Best Practices mySQL for SSDs Resources https://www.percona.com/blog/2020/07/29/checkpointing-in-mysql-and-mariadb/ https://www.percona.com/blog/2020/07/30/how-mysql-and-mariadb-perform-on-nvme-storage/?utm_campaign=2020%20Blog%20Q3&utm_content=135945936&utm_medium=social&utm_source=twitter&hss_channel=tw-35373186 https://www.samsung.com/semiconductor/global.semi.static/best-practices-for-mysql-with-ssds-0.pdf 🏭 Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
Bob Diachenko discovered an attack on MongoDB and ElasticSearch clustered that are unsecured. We discuss this attack in detail and how we as Backend Engineers can secure our databases. 0:00 The Meow Attack again MongoDB & ElasticSearch 1:43 How does it work? 5:00 Scope of the Attack 6:00 How Backup & MVCC Help 8:30 What does “Unsecure” mean? 11:00 Protecting Database Instances
Support this podcast: https://anchor.fm/hnasr/support
My progress of researching webRTC
Support this podcast: https://anchor.fm/hnasr/support
This is a podcast I did with @Adarsh Menon where I discuss my journey into Backend Engineering and some lessons learned during the course of my 20+ years engineering journey. Enjoy 0:00 Intro 2:45 Podcast Starts 3:15 How did you get into programming? 10:15 What problems do you solve at Esri ? 14:55 Generalist or Specialist ? 24:45 Advice to people starting out 33:15 On being Humble 47:05 YouTube advice for tech YouTubers 53:45 Thoughts on starting a company 56:45 Advice to 22 year old Hussein 🏭 Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
Was reading this article and it is interesting how relatable to backend engineering and security and how many times I made this mistake before. In this video I discuss how it is not a good idea to ignore certificate validation which can lead MITM attacks. This article shows an ASUS router that does not verify TLS certificate which is a flaw discovered by Martin Rakhmanov a security researcher. 0:00 Intro 2:00 Validate Certificate 12:18 How to mitigate 18:00 Avoiding MITM Resources https://www.techradar.com/news/this-router-is-vulnerable-to-fake-updates-and-cross-site-scripting-attacks 🏭 Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
WhatsApp is a chatting application written in Erlang. Let us have a discussion on how WhatsApp managed to run 3 million TCP connections on each FreeBSD server. WhatsApp has the following metrics 42 Billion messages a day 1 Billion users 3 Million connections!! 0:00 Intro 2:00 How WhatsApp reached 1,2 then 3 Million Connection 7:00 How Many Processes? 10:00 Server Side Load Balancing 13:50 Client Side Load Balancing Resources https://blog.whatsapp.com/1-million-is-so-2011 https://blog.whatsapp.com/on-e-millio-n https://developers.facebook.com/videos/f8-2016/a-look-at-whatsapp-engineering-for-success-at-scale/ 🏭 Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this video I go through why TLS 1.0 and TLS 1.1 should go away. Resources https://threatpost.com/riskrecon-the-tls-1-2-deadline-is-looming-do-you-have-your-act-together/157296/ https://www.zdnet.com/article/chrome-84-released-for-blocking-notification-popups-on-spammy-sites/ https://www.theregister.com/2020/07/20/microsoft_roundup/
Support this podcast: https://anchor.fm/hnasr/support
Github security team has found a remote execution code in Node.JS library changelog. In this video I describe the bug and go through the code Resources https://portswigger.net/daily-swig/github-security-team-finds-remote-code-execution-bug-in-popular-node-js-changelog-library https://github.com/conventional-changelog/standard-version/pull/351/files https://github.com/advisories/GHSA-7xcx-6wjh-7xp2
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss the VPN Leak of 1.2 TB of user logs data, IP addresses, password and much more Resources https://www.theregister.com/2020/07/17/ufo_vpn_database/ https://www.comparitech.com/blog/vpn-privacy/ufo-vpn-data-exposure/
Support this podcast: https://anchor.fm/hnasr/support
In July / 17 Cloud Flare had a 27 minutes outage, we discuss this outage what caused it and my thoughts on this .. https://blog.cloudflare.com/cloudflare-outage-on-july-17-2020/
Support this podcast: https://anchor.fm/hnasr/support
A hacker used Twitter’s own ‘admin’ tool to spread cryptocurrency scam. In this video I discuss this attack
Support this podcast: https://anchor.fm/hnasr/support
Server-Sent Events or SSE is when the server sends events to the client in a unidirectional manner. In this video I explain Server-Sent Events and compare it to websockets and HTTP and Long Polling. Source Code https://github.com/hnasr/javascript_playground/tree/master/server-sent-events Resources https://developer.mozilla.org/en-US/docs/Web/API/EventSource 0:00 Intro 1:50 HTTP 1.0/1.1 3:40 WebSockets 5:00 Server Sent Events 7:30 SSE Use Cases 9:00 SSE Code Example 18:00 SSE Pros & Cons 25:20 Do You Need SSE? 28:30 Summary
Support this podcast: https://anchor.fm/hnasr/support
In this video I go through all possible ways the US can use to block TikTok? 0:00 Intro 0:22 App Stores 1:30 DNS 2:20 ISP Level Block 3:30 DOH/ DOT 5:00 SNI 5:50 VPN
Support this podcast: https://anchor.fm/hnasr/support
This is a question from one of you guys that I thought I'd answer in its own video since its loaded. Q/A - Shark Beak I currently have the same setup for my side project. What do you think about having a 'create table if not exist' running on startup that creates this table? Good/bad? It is always a good idea to have a specific database user for each route with specific permissions and use connection pooling as much as possible.
Support this podcast: https://anchor.fm/hnasr/support
ZeroMQ is an Open Source Messaging Library designed for a high-performance asynchronous messaging library. In this video I discuss this tech and build a simple queue with this tech
0:00 Intro
1:48 What is ZeroMQ?
4:48 Messaging Patterns
6:42 Socket Types
8:55 Simple Queue
11:00 Code
23:20 ZeroMQ Pros & Cons
29:30 Summary
Source Code
https://github.com/hnasr/javascript_playground/tree/master/zeromq-simplequeue
Resources
https://github.com/booksbyus/zguide/tree/master/examples/Node.js
https://en.wikipedia.org/wiki/ZeroMQ
https://blog.scottlogic.com/2015/03/20/ZeroMQ-Quick-Intro.html
http://zguide.zeromq.org/page:chapter3#advanced-request-reply
Outline
- What is ZeroMQ?
- Message library
- Message Patterns
- Broker less
- Simple you build the components that you need
- Sockets Types
- REQ
- REP
- PUSH
- PULL
- ROUTER
- DEALER
- Message PatternS
- Synchronous Request/Response
- Asynchronous Request/Response
- Publish/Subscribe
- Push/Pull
- Exclusive Pair
- Example! (Simple Queue (Push Pull))
- Pros & Cons
- Pros
- Simple (meh)
- Efficient lightweight
- Great for small use cases
- Cons
- You have to write customize
- If you are building a large distributed message queue then you need to implement all features
- Feels over-engineered Could be simpler.
- Pros
Support this podcast: https://anchor.fm/hnasr/support
Discussing Layer 7 Reverse Proxy D=DOS Mitigation (Security Now Video by Steve Gibson )
Support this podcast: https://anchor.fm/hnasr/support
Google Chrome and Firefox to Join Apple’s Safari in One Year Certificate Validity (My opinion)
Support this podcast: https://anchor.fm/hnasr/support
TCP Fast Open Spec https://tools.ietf.org/html/rfc7413#section-1
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss what is the TCP Slow Start and its effect on performance of backend applications, proxies and even frontend applications.
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain why we can't run unencrypted HTTP/2 or HTTP/3 without enabling TLS. This is because of Protocol Ossification.
Support this podcast: https://anchor.fm/hnasr/support
Article: Why Turning on HTTP/2 Was a Mistake - Lucidchart - https://www.lucidchart.com/techblog/2019/04/10/why-turning-on-http2-was-a-mistake/ In this video I discuss this article and my opinion. That is not a limitation of HTTP/2 but of the application that couldn't handle the request. It is like driving a volvo all your life and then switching to a Ferrari and saying it was a mistake because its too fast. I disagree with the solutions of throttling the LB and I think the app should either be architected to not send this much requests if possible or just add more servers since HTTP is stateless you should be able to scale. HTTP/2 however does use more cpu it is dealing with many streams. The article doesn’t explain if it was H2 all the way though or not. 0:00 Intro 1:17 HTTP/1.1 Current Architecture 4:00 What happened when They Enabled HTTP/2 AT LB 7:00 Why I disagree with the throttling 8:00 Proposed Solutions 12:15 Why HTTP/2 can be CPU intensive Card at minute 3 playlist http2 🏭 Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this video I have a conversation with you on how we one break it into backend engineering answer most of your questions, what should you write in a CV, what recruiters expect for backend engineers etc.. Question: Hey Hussein, I hope you are doing well, Are there any tips or tricks I can do to make it easier to break into the industry as a back-end developer? i mean what are recruiters looking for in a CV? i would be glad if you made a video about that.
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss multicast DNS. Wikipedia defines multicast dns In computer networking, the multicast DNS (mDNS) protocol resolves hostnames to IP addresses within small networks that do not include a local name server. 0:00 Intro 0:30 DNS Explained in LAN 4:00 Multicast DNS
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain the different cases where we overengineer software especially in two pieces, software code, and system design and architecture. Chapters 0:00 Intro 1:45 OverEngineering in Software Development 7:15 OverEngineering System Design
Support this podcast: https://anchor.fm/hnasr/support
We know ISP can block you from going to YouTube all together but can they block you from watching a single youtube video? I discuss that in this video Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain what happened to services such as stripe & roku which failed to establish TLS sessions because the ROOT certificate AddTrust External CA Root has expired. This is a bug in openSSL and other software the perform this kind of validation. I explain what happened in details.. Resources https://twitter.com/sleevi_/status/1266647545675210753 https://tools.ietf.org/html/rfc4158 https://ohdear.app/blog/resolving-the-addtrust-external-ca-root-certificate-expiration
Support this podcast: https://anchor.fm/hnasr/support
Sharing a Single TCP Connection whether this is HTTP, WebSockets or just RAW TCP protocol between multi-threading or multi-processes application is bound to cause bad undesirable effects. I explain this in this video and give example of how QUIC & HTTP/2 have solved this problem. Pooling 5:00 QUICK Database 6:30 HTTP/2 Playlist 7:00
Support this podcast: https://anchor.fm/hnasr/support
in this video I explain how database systems store their data on Disk both f from SSD (solid-state drives) and HDD (HarD disk drive). There are many factors that go into that discussion so I tried to cover the high level here. Reseources https://www.extremetech.com/extreme/210492-extremetech-explains-how-do-ssds-work https://www.percona.com/blog/2019/12/18/give-love-to-your-ssds-reduce-innodb_io_capacity_max/
Support this podcast: https://anchor.fm/hnasr/support
Understand Fundamentals not Tools (e.g. WebSockets vs Socket.io, Javascript vs React, Python vs Django)
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss all about software documentation.Good doc from bad doc and how to differentiate and is documentation the only source of learning about a particular tech?
Support this podcast: https://anchor.fm/hnasr/support
In this podcast I discuss 7 Tips To Optimize Your Backend API (Without Caching)
1) The serialization representation (XML/JSON, pbf) 0:40
2) Preheating connections
3) H1 vs H2
4)TCP meltdown (big distance)
5) Proxies
6) large payload (select * )
7) Client side processing (transfomring the work)
Support this podcast: https://anchor.fm/hnasr/support
In this video, I would like to discuss 10 performance tweaks and tips that you can apply to your frontend application to improve its performance and efficiency. These tips are applicable to any programming language on Web, mobile, or desktop application. Chapters 0:00 Intro 0:40 Optimistic queries 1:57 Paging 3:00 Lazy Loading 4:00 Request what you Need 6:53 Connection State 10:00 LRU Cache 11:10 Group Notifications 12:30 Avoid Expensive Queries Even At Expense of Bad UX 14:00 Design your UX So you Minimize Requests
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain the difference between Agile & Water, the pros & cons and more.
Chapters
0:00 Intro
1:00 Waterfall
5:00 Agile
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain how WebSockets work in the new HTTP/2 protocol. This is explained in the Bootstrapping WebSockets with HTTP/2 in RFC8441 https://tools.ietf.org/html/rfc8441 https://link.medium.com/v5sB9nbUp6 1:00 HTTP2 5:50 HTTP/2 limitations
Support this podcast: https://anchor.fm/hnasr/support
Metroexpress lane is implementing a Machine Learning Occupancy Detection System which I found interesting. In this video I discuss machine learning, supervised learning, labeling and much more.
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain the multi-tenancy architecture. The basic idea is to have a single instance of your application to serve multiple tenants or customers and the properties are this. This is as opposed to isolated or dedicated infrastructure.
- Shared Instance
- One database hosting multiple customers
- Isolation at the application level
- Can be multi-processes and multi-instances as long as they are pooled
- Stateless
Tags
Multitenancy, software Multitenancy, Multitenancy explained, Multitenancy architecture
https://www.zdnet.com/article/defining-the-true-meaning-of-cloud/
Support this podcast: https://anchor.fm/hnasr/support
Algorithms & Data Structures are critical to Backend Engineering however it really depends on what kind of application and infrastructure you are building. In this video I want to go through the following 1 Backend Engineers are two types - Integrating Existing Backend - Core Backend Example Building a CRUD API? Online Cinema system, URL shortener, You will pick up a database and write your logic Building a social network? * are you gonna be integrator use a ready made graph database? * Are you gonna use a off the shelf database and write your logic in the application? * Are you gonna build your own graph database platform? * Any of these scenarios you will run into problems slow performance and you need to understand why Building a monitoring system? are you gonna integrate an existing database ? or build your own? 2. Be Pragmatic (Algorithms are not always the solution) * Most performance issues are not algorithm problems, they are just bad bugs. and misuse .. paging We are a sorted 100 items takes 1 minute to sort and return.. merge sort or heap or quick sort won’t help you 3. Always keep learning to be open to learn new Algorithms
Support this podcast: https://anchor.fm/hnasr/support
In this video I want to talk about my preferred method of learning backend engineering technologies, I prefer podcasts and youtube videos some people prefer books.
- My First Programming Book
- My Problem with Learning in Books
- My Preferred Method of learning (Podcasts)
- YouTube Videos
- Problem is Biased ..
- Details in WIkipedia then
Recommended Podcasts
https://softwareengineeringdaily.com/
https://www.dataengineeringpodcast.com/
https://changelog.com/podcast
Recommended YouTube Channels
https://www.youtube.com/user/TechGuyWeb
https://www.youtube.com/user/99baddawg
https://www.youtube.com/channel/UCRPMAqdtSgd0Ipeef7iFsKw
https://www.youtube.com/channel/UCn1XnDWhsLS5URXTi5wtFTA
Support this podcast: https://anchor.fm/hnasr/support
Message Queues system like RabbitMQ and Kafka are amazing technologies but when should you actually use a message queue? I discuss this in this video.
Support this podcast: https://anchor.fm/hnasr/support
In this video I talk about RabbitMQ Channels compared to HTTP/2 Streams and how QUIC helps mitigate some of the major limitations in Channels and Streams.
Chapters
0:00 Intro
0:10 RabbitMQ Channels
3:10 HTTP/2 Streams
6:00 How QUIC Helps
Support this podcast: https://anchor.fm/hnasr/support
In this video I discuss why QUIC will make a great communication protocol for databases and how it solves a critical problem with stateless web applications. Web applications uses database connection pooling to establish database connections on the backend. But that creates other sorts of problems.
Timecodes
0:00 Intro
0:20 Database Communication Protocols
2:00 Problem with Sharing Database Connections
6:50 How QUIC streams can help Databases
Support this podcast: https://anchor.fm/hnasr/support
Database Application level sharding is the process of splitting a table into multiple database instances in order to distribute the load. However, Sharding a database is an expensive operation (maintainability and overhead) and I suggest you do that only when you absolutely need to. That means when your single instance database can no longer serve queries with minimum latency. So I suggest you monitor that and only shard if necessary.. I rather do replication make master / backup and make requests read from replica than Sharding just because it is easier.. good question... nice idea for a video
5:00 Partition horizontally
7:45 Replication (Master/backup) Scale reads
11:00 Scale writes by region
12:30 Sharding
16:40 Sharding in YouTube
Support this podcast: https://anchor.fm/hnasr/support
In this video I give some advice to junior backend engineers who just started their new jobs and feel overwhelmed. Backend engineering jobs can be overwhelming at the start, there are so much to learn, so much to read, documentation, tests, code in multiple programming languages. Here are some advice for Junior backend engineers.
0:00 Intro
1:20 Take your time
6:05 Stay hungry keep learning
8:36 Assume your collages know something you don’t
12:42 Don’t bad mouth other people code even if its bad
17:19 Question everything
Support this podcast: https://anchor.fm/hnasr/support
Creating a software design is an art here is why
Support this podcast: https://anchor.fm/hnasr/support
Software Testing is Art, In this video, I discuss what makes a Good Software Tester. How can you become a better software tester that people line up to give pick you?
Organizations build software products. But the software is useless if it is filled with bugs. A bug discovered after the product is shipped costs the organization a lot of money. A bug that is discovered before shipping saves the organization time and resources. So organizations line up to find good software engineers that are great in troubleshooting and testing.
Support this podcast: https://anchor.fm/hnasr/support
JSON Web Token (JWT, sometimes pronounced JOT) an internet standard for creating JSON-based access tokens that assert some number of claims. The tokens are signed either using a private secret or a public/private key.
In this video I want to discuss the difference between JWT and Session Based Auth, will show examples with Node JS and Postgres. Finally I’ll discuss the pros and cons of JWT.
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain the main difference between asynchronous execution, multithreading and multiprocessing programming. There are advantages and disadvantages of each approach.
- Synchronous 0:30
- Multithreading a process have many threads shared resources 3:20
- Async io single thread 6:00
- Multiprocessing 11:00
Threads are evil
https://web.stanford.edu/~ouster/cgi-bin/papers/threads.pdf
sync vs async, multithreading vs multiprocessing, multithreading async, threading
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain End to End encryption within the context of WhatsApp. I explain how encryption and TLS works then the problem of having a centerlized server decrypting the traffic, I then talk about how end to end encryption (e2e) can help mitgate that and finally I explain the problems with e2e encryption
- Classic Encryption Example 1:00
- End to end encryption 3:25
- Problem with E2E 7:30
-
- Trust? Fingerprints/CA (QR whatspp)
encryption 00:30
encryption, e2e encryption, end to end encryption, whatsapp end to end encryption, how e2e works, e2e limitations
Support this podcast: https://anchor.fm/hnasr/support
- Introduced long time a go
- Unlimited forwarding then limited Up to 5 times
- Double forwarded too many times (last year)
- Double forwarded messages can only be forwarded once or even none
Forwarding feature
- end to end encryption
- Sent/delivered/read
https://www.cnbc.com/2020/04/07/whatsapp-limits-message-forwards-to-combat-coronavirus-misinformation.html
Support this podcast: https://anchor.fm/hnasr/support
In this video, I explain the difference between a proxy and a reverse proxy. This is a refreshed version of the proxy video I made. What is a Proxy ? Proxy use cases - Logging - Anonymity - Geofencing - Caching - Block sites (office) - Enable Polyglot What is Reverse Proxy? Reverse Proxy Example Reverse Proxy Use Cases - Caching (Varnish) - Load Balancing - Ingress (/pics) - Canary Deployment Many types of proxy check it here Q and A * can we use proxy and reverse proxy together? * Can I use Proxy to hide my identity instead of a VPN? * Can I use proxy just for HTTP? tags: proxy vs reverse proxy, reverse proxy benefits, what is a reverse proxy, what is a proxy, what is the difference between proxy and reverse proxy Sidecar proxy 4:14 Load balance 7:40 Proxy Reverse Proxy 5:00 🏭 Software Architecture Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this video I explore what is a distributed transaction but first I explain what is a transaction, then why we invented distributed transactions and finally discuss proposed solutions to implement distributed transactions Cards 2:40 ACID https://www.youtube.com/watch?v=pomxJOFVcQs 7:30 Microservices https://www.youtube.com/watch?v=T-m7ZFxeg1A Transactions 2:30 Compensating Edits 6:40 Atomic Clocks 11:50 Event Sourcing 13:30 Mini-Monolith 15:20 Resources https://www.youtube.com/watch?v=YPbGW3Fnmbc https://www.youtube.com/watch?v=S4FnmSeRpAY https://www.youtube.com/watch?v=epOLEdaPSLQ https://softwareengineeringdaily.com/2018/12/19/linkerd-service-mesh-with-william-morgan/ 🏭 Software Architecture Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🛰 Network Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr 🏰 Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC 🐘 Postgres Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj 🚢Docker https://www.youtube.com/playlist?list=PLQnljOFTspQWsD-rakNw1C20c1JI8UR1r 🧮 Programming Pattern Videos https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe 🛡 Web Security Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv 🦠 HTTP Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO 🐍 Python Videos https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX 🔆 Javascript Videos https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM 👾Discord Server https://discord.gg/CsFbFce Support me on PayPal https://bit.ly/33ENps4 Become a Patreon https://www.patreon.com/join/hnasr? Stay Awesome, Hussein
Support this podcast: https://anchor.fm/hnasr/support
A very necessary change and good step made by Google Chrome Team to rollback the same site cookie change
Blog
https://blog.chromium.org/2020/04/temporarily-rolling-back-samesite.html
Firefox re-enables
https://www.youtube.com/watch?v=sh3TPId35Ec
SameSite Cookie Attribute
https://www.youtube.com/watch?v=aUF2QCEudPo
Support this podcast: https://anchor.fm/hnasr/support
In this video I explain why we invented bloom filters and where you can use it to make your queries more efficent.
Support this podcast: https://anchor.fm/hnasr/support
On-Demand TLS is a new feature developed by Caddy Web Server that allows TLS certificate to be generated on the first request, I explain this tech and their pros and cons in this video.
Support this podcast: https://anchor.fm/hnasr/support
Lazy Loading is a technique where a piece of data is being loaded when needed instead prior. This ensures Fast startup times but can delay requests.
In this video I’ll show both Eager loading and the lazy loading with example Node JS
Support this podcast: https://anchor.fm/hnasr/support
In this podcast I discuss the good, the bad and the ugly about gRPC. No technology is perfect.
Support this podcast: https://anchor.fm/hnasr/support
THIS IS MAIN reason gRPC was invented 8:40 -> 16:40 (8 minutes)
In this video I discuss the reasoning and all the problems and limitations that lead to the invention of gRPC.
Support this podcast: https://anchor.fm/hnasr/support
Firefox has disabled TLS 1.0 and TLS 1.1 to improve your website connections. Sites that don't support TLS version 1.2 will now show an error page. We reverted the change for an undetermined amount of time to better enable access to critical government sites sharing COVID19 information.
https://www.youtube.com/watch?v=grVVuGnN9IE
https://www.mozilla.org/en-US/firefox/74.0/releasenotes/
https://twitter.com/cramforce/status/1242515799215988737?s=21
Support this podcast: https://anchor.fm/hnasr/support
Copy and Paste Code, Tutorial Hell, Frontend Frameworks, GitHub Pull request Trolling and more
Support this podcast: https://anchor.fm/hnasr/support
New software chat series let me know if you like it
Support this podcast: https://anchor.fm/hnasr/support
In this video, I discuss the path of becoming a backend engineer through concepts and fundamentals. These are not tools 🧰 these are backend concepts and fundamentals technologies. - Communication Protocols 3:30 - TCP,UDP - QUIC - HTTP - WebSockets - gRPC - Web Servers 8:40 - How web servers work? - Dynamic vs Static Content - E-Tags - HTTP protocol - Database Engineering 11:00 - Relational vs NoSQL - ACID - Proxies (Reverse Proxies, Load balancer) 13:31 - What is difference between Proxy vs Reverse Proxy - Layer 7 Proxy vs Layer 4 Proxy - Reverse Proxy applications - Load Balancing algorithms - Caching 15:40 - When to use Caching - Message queue, Pub/Sub 16:36 - When to use PubSub messaging first queue. - Web Frameworks (API authoring) 18:30 - Express, Django, Node JS - Message Formats (JSON, protobuf) 19:24 - JSON & protobuf - Security 20:50 - TLS, Encryption, Firewalls
Support this podcast: https://anchor.fm/hnasr/support
FireFox DNS over HTTPS is a great step to a more secure web however it is absolutely useless without this ESNI. I also discuss doh in this video, esni and why this might not be a good idea
https://blog.mozilla.org/blog/2020/02/25/firefox-continues-push-to-bring-dns-over-https-by-default-for-us-users/
Support this podcast: https://anchor.fm/hnasr/support
In this video I want to talk over the active active active vs active passive cluster configuration for high availability. We will also explain the pros & cons of using an active-active cluster vs using an active-passive cluster.
Support this podcast: https://anchor.fm/hnasr/support
A sidecar proxy is an application design pattern which abstracts certain networking features, such as inter-service communications, monitoring and security, timeouts, retries, communication protocols, away from the main architecture to ease the maintenance of the application as a whole.
In this video I’d like to talk about how we classically do things:
- Library Pattern
- getComments HTTP1:1/JSON
- Add features retries/timeout /hardcode the server name what it changed/add new server to load balance between them/credentials
- Sidecar pattern
- getComments HTTP/2
- Sidecar proxy/container
Examples
- Microservices (Linkerd, Envoy, Istio)
Pros
- Language agnostic (polyglot)
- Protocol upgrade
- Security
- Tracing and Monitoring
- Service Discovery
- Caching
Cons
- Complexity
- Latency
- SideCar Proxy must be layer 7
Library 2:25
Sidecar: 7:40
Example: 13:00 (https://www.youtube.com/watch?v=cp3Ku1XeOn8)
Pros & Cons : 16:15
Cards
6:22 HTTP/2
10;50 Reverse Proxy / proxy
example 13:00 (https://www.youtube.com/watch?v=cp3Ku1XeOn8)
16:00 Layer 4 vs Layer 7 load balancer
Support this podcast: https://anchor.fm/hnasr/support
MySQL 8.0 supports atomic Data Definition Language (DDL) statements. This feature is referred to as atomic DDL. An atomic DDL statement combines the data dictionary updates, storage engine operations, and binary log writes associated with a DDL operation into a single, atomic transaction. The transaction is either committed, with applicable changes persisted to the data dictionary, storage engine, and binary log, or is rolled back, even if the server halts during the operation.
I discuss mySQL 8.0 atomic ddl compared to Postgres transactional ddl
Support this podcast: https://anchor.fm/hnasr/support
gRPC (gRPC Remote Procedure Calls[1]) is an open source remote procedure call (RPC) system initially developed at Google in 2015[2]. It uses HTTP/2 for transport, Protocol Buffers as the message format.
In this video I want to explore gRPC, go through examples, pros and cons of gRPC.
Client/ Server communication
- SOAP
- HTTP (REST)
- WebSockets
Client Libraries
gRPC
gRPC Demo
- todos
gRPC Pros and Cons
Pros
- Fast two/uni and request
- Unform
- One library to rule them all
- Progress feedback( long synchronous requests) drop pluggable wait...)
- cancel request
- All benefits of H2 and Protobuff
Cons
- schema based (not everyone wants schema)
- Thick client - limited languages - Proxies still don’t understand it
- Still young
- Error handling
- No native browser support
- Timeouts, circuit breaker just like any RPC (pub/sub rules in this case)
Can you create your own protocol?
- Spotify example with Hermes
Support this podcast: https://anchor.fm/hnasr/support
Firefox version 73.1 fixes a bug that prevented users from signing into Royal Canadian Bank (Firefox fixed a bug that prevented Canadians from signing in to their bank royal Canadian bank) what caused it? Lets get into it
* Users will sign in and will get blank page.
* Cause Beacon API and HTTP/2
- Bug has been there forever
Sources
https://bugzilla.mozilla.org/show_bug.cgi?id=1613943#c16 🏭 Software Architecture Videos
Support this podcast: https://anchor.fm/hnasr/support
Some cool new features are coming to the Javascript language namely Optional chaining and nullish coalescing how useful are they? lets discuss
So Optional chaining and nullish coalescing are coming to the Javascript language. I think it is already in the Javascript Engines (V8, SpiderMonkey, JavascriptCore and Chakra) which will be picked up by other clients such as browsers, node js , dino other stuff .. Cool stuff lets get into it.
- Show them (chrome devtools)
- Where is it available?
- Node JS not yet (13.9 V7.9) (https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V13.md#13.9.0)
Resources
V8 release v8.0 · V8
Optional Chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
Nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator
Support this podcast: https://anchor.fm/hnasr/support
We are one step closer to a more secure web. Firefox disabled TLS 1.0 1.1 It’s a big change that I am very happy about but not everyone is. Let's discuss .
Resources
https://hacks.mozilla.org/2020/02/its-the-boot-for-tls-1-0-and-tls-1-1/
https://www.openssl.org/~bodo/ssl-poodle.pdf
Firefox disabled TLS 1.0 1.1 It’s a big change that I am very happy about but not everyone is. Lets discuss
Resources
https://hacks.mozilla.org/2020/02/its-the-boot-for-tls-1-0-and-tls-1-1/
https://www.openssl.org/~bodo/ssl-poodle.pdf
Support this podcast: https://anchor.fm/hnasr/support
Google engineer Alyssa Wilk discovers a critical security bug related to whitespaces in header values. Envoy and Node JS fix this by releasing a security patch. let us discuss it. tags: envoy, envoy proxy, envoy security, envoy proxy whitespace, Alyssa Wilk
Support this podcast: https://anchor.fm/hnasr/support
Node JS Releases a Fix for an HTTP Request Smuggling Vulnerability, we discuss the attack and the fix in detail.
Support this podcast: https://anchor.fm/hnasr/support
In April 2019 Google blocked sign-ins from Embedded Browsers to avoid man in the middle phishing attacks. In this video, we discuss why.
https://security.googleblog.com/2019/04/better-protection-against-man-in-middle.html?m=1
Support this podcast: https://anchor.fm/hnasr/support
PostgresSQL version 12 has been released, let's go through the features that I think are most interesting and cool. #softwarenews Feature Matrix https://www.postgresql.org/about/featurematrix/ - Allow adding columns to Index (GIST) https://www.postgresql.org/about/featurematrix/detail/314/ - COPY FROM WHERE COPY FROM ... WHERE - More native support of JSON objects https://www.postgresql.org/docs/12/functions-json.html#FUNCTIONS-SQLJSON-PATH - Reindex concurrently (slow but allows writes) https://www.postgresql.org/docs/12/sql-reindex.html#SQL-REINDEX-CONCURRENTLY - Performance on large partitioned tables - Stored Generated Columns
Support this podcast: https://anchor.fm/hnasr/support
Google just released the latest version of Chrome (80) and one of the interesting features making a big change to the default cookies that might actually prevent CSRF forever. Let’s discuss this. #softwarenews *
Same Site Attribute * Break some apps * Devs must explicitly set None;secure * Will this end CSRF
Resources https://youtu.be/GPz7onXjP_4
https://www.chromestatus.com/feature/5088147346030592
News Theme 2 by Audionautix is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) Artist: http://audionautix.com/
Support this podcast: https://anchor.fm/hnasr/support
Load balancing is process of balancing incoming requests to multiple machines, processes or services. In this video we will explain two types of load balancers, layer 4 and layer 7.
Layer 4 - haproxy, NLB
Pros
- great for simple packet-level load balancing
-
Fast and efficient doesn’t look at the data
-
More secure as it cant really look at your packets. So if it was compromised no one can look
- Uses NAT - One connection between client and server NATed
Cons
- Can't do smart load balancing based on the content, such as switch request based on the requested media type
- Cant do microservices with this type
- Has to be sticky as it is a stateful protocol (all segments)
Layer 7 (Nginx , haproxy)
This type of proxy actually looks at the content and have more context, it knows you are visiting the /users resources so it may forward it to a different server. Essential and Great for microservices , it knows the content is video:image etc..
Support this podcast: https://anchor.fm/hnasr/support
NginX is an open source web server written in C and can also be used as a reverse proxy and a load balancer.
In this video, I want to go through the following topics in NginX
- What is NginX? 2:25
- Current & Desired Architecture 4:58
- Layer 4 and Layer 7 Proxying in Nginx 8:40
- Example 10:25
- Install Nginx (mac) 13:30
- Nginx as a Web Server 15:00 (webserver.conf)
- Static context location root 20:00
- regular expression 27:00
- proxy pass 30:30
- Nginx as a Layer 7 Proxy 33:30
- Proxy to 4 backend NodeJS services (docker) 37:00
- IP_Hash load balaning 43:00
- Split load to multiple backends (app1/app2) 46:00
- Block certain requests (/admin) 49:00
- NGINX as a Layer 4 Proxy 51:30
- Create DNS record 1:01:08
- Enable HTTPS on Nginx (lets encrypt) 1:05:08
- Enable TLS 1.3 on Nginx 1:14:00
- Enable HTTP/2 on NGINX 1:17:10
- Summary 1:20:10
Cards
3:40 proxy vs reverse proxy https://www.youtube.com/watch?v=ozhe__GdWC8&t=1s
8:50 OSI Model https://www.youtube.com/watch?v=7IS7gigunyI
10:00 L4 vs l7 load balancer https://www.youtube.com/watch?v=aKMLgFVxZYk
34:00 spin up docker app https://www.youtube.com/watch?v=vmSMrQ8Ev9w&t=14s
44:26 stateless vs stateful https://www.youtube.com/watch?v=nFPzI_Qg3FU
1:16:30 TLS video https://www.youtube.com/watch?v=AlE5X1NlHgg&t=8s
Support this podcast: https://anchor.fm/hnasr/support
In this podcast I explain what database partitioning is and illustrate the difference between Horizontal vs Vertical Partitioning, benefits and much more. Why Partitioning?
Support this podcast: https://anchor.fm/hnasr/support
ARP
Address Resolution Protocol is a protocol that maps IP address to a mac address so hosts can be addressed. I want to make a video explaining ARP because it is starting to surface a lot in different videos that I’m making such as man in the middle, load balancing, and security.
Support this podcast: https://anchor.fm/hnasr/support
In this video I want to go through what really happens under the hood when you type google.com and you hit enter in a browser. This is inspired by alex’s github page below, it is a great detailed description of what happens. I did however add more details to certain areas and I removed some stuff like keyboard events and low level operating system like that. So if you are interested stay tuned!
https://github.com/alex/what-happens-when
5:30 HSTS https://youtu.be/kYhMnw4aJTw
19:30 tcp vs udp
https://youtu.be/qqRYkcta6IE
24:42 tls
https://youtu.be/AlE5X1NlHgg
40:56 mimesniffing
https://youtu.be/eq6R6dxRuiU
Initial typing
- lookup for most visited sites or an actual search
Google.com Enter
- parse is it a url or search term? If search do a search if url visit website
Which protocol? which port ?
HSTS?
HTTPS
or HTTP?
DNS
cached?
Hosts?
DoH?
Lookup ?
TCP
ip / port ?
arp?
NAT?
proxy?
HTTP 1.1 ? More than one connection
TLS
version? ciphers? Alpn? SNI ? H2 ? H3? Quic?
GET / - ?
Headers compress
cookies?
binary stream?
HTML?
H2 push?
HTML Parse?
Make multiple requests css? JS? Multiple streams?
If H1 then pipeline?
Support this podcast: https://anchor.fm/hnasr/support
Failover is the technique of switching to a redundant backup machine when a certain node goes down. This is a very common implementation for achieving high availability and it is often mixed with different load balancing techniques such as layer 4 and layer 7 balancing. In this video i want to go through following * What is Failover? 1:47 * ARP - Address Resolution Protocol 3:00 * VIP & VRRP 5:40 * High-availability Example 12:12 Cards 4:00 ARP 12:50 docker javascript 18:00 OSI Model
Support this podcast: https://anchor.fm/hnasr/support
HAProxy is free, open source software written in C that provides a high availability layer 4 and layer 7 load balancing and proxying . It has a reputation for being fast and efficient (in terms of processor and memory usage). In this video I want discuss the following
Current & Desired Architecture 2:30 HAProxy Architecture 5:50 HAProxy as TCP Proxy & HTTP Proxy (Layer 4 vs Layer 7) 17:00 ACL (Access Control Lists) 19:20 TLS Termination vs TLS Pass Through 20:40 Example 24:23
Spin up the services 25:51 Install HAProxy - 28:00
HAProxy configuration 29:11 ACL Conditional 39:00
ACL Reject URL 48:00
Enable HTTPS HAProxy 53:00 Enable HTTP/2 on HAProxy 1:05:30
Summary
Cards Docker Javascript node 4:00 Varnish 15:46 NAT 23:30 Docker Javascript node 26:00 Encryption 56:00 TLS 56:10 HTTP2 1:08:40 Source Code for Application HAProxy config https://github.com/hnasr/javascript_playground/tree/master/proxy Docker application https://github.com/hnasr/javascript_playground/tree/master/docker resources https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/ https://www.haproxy.com/documentation/aloha/10-0/traffic-management/lb-layer7/acls/#predefined-acls https://certbot.eff.org/lets-encrypt/osx-nginx
Support this podcast: https://anchor.fm/hnasr/support
HSTS or HTTP Strict Transport Security is a mechanism to force clients to communicate in HTTPS with a web server if both support this protocol.
In this podcast, I talk through this tech.
Support this podcast: https://anchor.fm/hnasr/support
Database engines or storage engines or sometimes even called embedded databases is software library that a database management software uses to store data on disk and do CRUD (create update delete)
Embedded means move everything in one software no network client server.
In this video I want to go through the few popular database engines, explain the differences between them and finally I want to spin up a database and change its engine and show the different features on each engine
Timecodes
What is a database Engine 3:00
myISAM 9:43
Aria 16:30
InnoDB 19:00
XtraDB 25:30
LevelDB 27:40
RocksDB 34:00
SQLite 38:11
BerkelyDB 42:00
Demo! 47:11
Cards
ACID 4:30
mysql/javascript 56:17
Resources
https://youtu.be/V_C-T5S-w8g
https://mariadb.com/kb/en/library/changes-improvements-in-mariadb-102/
https://mariadb.com/kb/en/library/why-does-mariadb-102-use-innodb-instead-of-xtradb/
https://github.com/facebook/rocksdb/wiki/Features-Not-in-LevelDB
https://mariadb.com/kb/en/library/aria-storage-engine/
Best of MyISAM and InnoDB
What is the performance impact of using CHAR vs VARCHAR on a fixed-size field?
MySQL :: MySQL 8.0 Reference Manual :: 15.6.2.1 Clustered and Secondary Indexes
Why Uber Engineering Switched from Postgres to MySQL | Uber Engineering Blog
Support this podcast: https://anchor.fm/hnasr/support
Database Sharding Crash Course (with Postgres examples)
Database sharding is process of segmenting the data into partitions that are spread on multiple database instances to speed up queries and scale the system.
What is sharding?
sharing key / partition key
Consistent Hashing
Horizontal partitioning vs Sharding
Example
Pros and cons
What is Sharding? 1:30
Consistent Hashing 4:50
Horizontal partitioning vs Sharding 7:36
Example 8:45
Spin up Docker Postgres Shards 10:02
Write to the shard 17:25
Read from the Shard 39:20
Pros & Cons 51:10
Cards
Postgres pgadmin Docker 8:54
Postgres Javascript 18:18
URL vs Query param 22:30
CORS 29:30
sql injection 42:40
Source Code
https://github.com/hnasr/javascript_playground/tree/master/sharding
Docker commands (including pgadmin)
https://github.com/hnasr/javascript_playground/blob/master/sharding/shards/commands.txt
Dockerfile & init.sql
https://github.com/hnasr/javascript_playground/tree/master/sharding/shards
Horizontal partitioning vs Sharding
HP same database instance so you can still join
sharding across instances (different server)
Pros
Extreme scale rdbms
Optimal and Smaller index size
Cons
Transactions across shards problem
Rollbacks
Schema changes
Complex client (aware of the shard)
Joins
Has to be something you know in the query
Example
URL shortener
create table
CREATE TABLE public.test1
(
id serial NOT NULL primary key,
url text,
url_id character(5)
)
Spin up 3 instances
p1
P2
P3
post
get
Support this podcast: https://anchor.fm/hnasr/support
Varnish is a reverse proxy web accelerator Written in C designed to improve HTTP performance by using server side caching. In this video I’m going to go through how Varnish works, we will spin up a Varnish docker container and finally talk about the pros and cons of this tech. Agenda
Classic HTTP Architecture How Varnish Works? HTTP Setup - Varnish HTTPS Setup - Varnish with Caddy as TLS Terminator. Varnish Pros & Cons Summary
Pros
Cache & prefetching documents Resolve DNS hostnames in documents Rewrite scripts to optimize code Load Balancing Backend Connection Pooling Varnish Modules (VMODS, e.g. rewrite headers) Edge side includes (personalized dynamic page)
Cons
Cache Invalidation Only works on unencrypted HTTP For HTTPS frontend, terminate TLS For HTTPS backend, no solution in Varnish open source Can’t cache POST requests (e.g. GraphQL queries) HTTP/2 backends won’t benefit
docker commands docker run --name varnish -p 8080:80 varnish docker cp default.vcl varnish:/etc/varnish default.vcl vcl 4.0; backend default { .host = "husseinmac"; .port = "2015"; } timecodes Classic HTTP Architecture : 1:53 How Varnish Works 3:50 Demo HTTP 7:50 Demo HTTPS 18:23 Varnish Pros 28:43 Varnish Cons 39:26 Cards: Connection Pooling 2:15 https://www.youtube.com/watch?v=GTeCtIoV2Tw Layer 4 vs Layer 7 Load balancing 6:25 https://www.youtube.com/watch?v=aKMLgFVxZYk GET vs POST 6:48 https://www.youtube.com/watch?v=K8HJ6DN23zI Postgres NodeJS Chrome 8:23 https://www.youtube.com/watch?v=0JNq46eFuOM Redis 46:13 https://www.youtube.com/watch?v=sVCZo5B8ghE TLS Termination 49:35 https://www.youtube.com/watch?v=H0bkLsUe3no tags: varnish http accelerator, varnish pros and cons, varnish with https, varnish disadvantages Resources edge side includes https://www.fastly.com/blog/using-esi-part-1-simple-edge-side-include Cache invalidation varnish https://www.smashingmagazine.com/2014/04/cache-invalidation-strategies-with-varnish-cache/ ——
Support this podcast: https://anchor.fm/hnasr/support
short is a major revision of the hyper text transfer protocol that improves the performance of the web. It uses a binary protocol and multiplexing. In this video I want to go through how HTTP/2 works, its pros and cons and show the performance difference between HTTP 1.1 and HTTP/2 Http/1 request has to wait for a response And if you used up your connection response you can send anything else. Source Code: https://github.com/hnasr/javascript_playground/tree/master/staticpage So hack ! 6 tcp connections Pros One connection Multipex Push ( have to enable) Hpack (compress) Cons Abuse push Proxies that dont support it might slow down Load balancer layer 7 (a layer 7 load balancer that doesnt support h2 will slow down when your web server supports it) Cards 1:25 HTTP crash course 12:30 TLS Time codes HTTP 1.1 2:15 HTTP/2 5:50 HTTP/2 With Push 8:25 HTTP/2 Pros 9:48 HTTP/2 Cons 14:10 Demo 17:15 +----------+--------+---------+---------+ | Protocol | Online | Fast 3G | Slow 3G | +----------+--------+---------+---------+ | HTTP 1.1 | 94 ms | 10 s | 36 s | +----------+--------+---------+---------+ | HTTP/2 | 171 ms | 2.7 s | 10 s | +----------+--------+---------+---------+
Support this podcast: https://anchor.fm/hnasr/support
Sorry about the free form audio and low quality thought I post something here its been while
Support this podcast: https://anchor.fm/hnasr/support
Tor or (The Onion Router) is a free and open source project for allowing anonymous communication.
In this video I want to discuss this technology and explain how it works.
- What can sniffers really see?
- Why Tor? Why don’t we just use a VPN?
- How Tor Works?
- More about Tor
So if you are interested stay tuned.
Cards:
6:00 TLS
10:53 symmetric key encryption
TimeCodes:
Normal Request 1:30
What sniffers see 4:30
Why Tor and not VPN? 6:00
How Tor Works 8:50
More tor info 21:00
Tor Directory
Relay node
Exit node
Guard node
Support this podcast: https://anchor.fm/hnasr/support
Apache Kafka is a distributed stream processing software developed by LinkedIn and written in Scala and Java.
In this video I want to focus on the basics on Kafka, talk about how it works give finally spin up a kafka cluster and write a producer and a consumer.
Kafka Components
_ producer
_consumer
_topic
_partition
How Kafka works
Example ( Ride sharing )
Spin Kafka cluster
Write a producer
Write a consumer
Kafka Applications
Pros and Cons
Support this podcast: https://anchor.fm/hnasr/support
Tom Scott Full Video https://youtu.be/WVDQEoe6ZWY Tom Scott is a YouTuber with around 2M subscribers who discusses and specialized in computer security. I always enjoy his videos specially those on computerphile. He recently made a very interesting video titled This video is sponsor by —- We all probably know this is a reaction video to NordVPN Hack. He is really smart and not just because of his accent I wanted to make a video to elaborate on some of the statements that Tom made in his video and how that actually work. Cards 5:30 TLS https://www.youtube.com/watch?v=AlE5X1NlHgg 12:24 TLS Termination https://www.youtube.com/watch?v=H0bkLsUe3no Forged certificate https://slate.com/technology/2016/12/how-the-2011-hack-of-diginotar-changed-the-internets-infrastructure.html
Support this podcast: https://anchor.fm/hnasr/support
RabbitMQ
RabbitMQ is an open source distributed message queue written in Erlang and supports many communication protocols. It was trying to solve the spaghetti 🍝 mesh architecture where every client is communicating with other client in System by introducing an extra layer in the middle.
(slide)
In this video we will explain basic components of RabbitMQ Such as AMQP, channel, queue, publisher, consumer and some stuff, we will also learn how to spin up a RabbitMQ server and we will finally write some code to develop a publisher client that post messages to RabbitMQ. We will also write the consumer piece which will be the code that reads rabbitMQ. Finally I will talk about my personal thoughts on this tech.
- RabbitMQ Components(slide)
- *Publisher
- *Consumer
- *Connection
- *Channel
- *Queue
- Spin RabbitMQ server with Docker
- Write a Publisher client NodeJs
- Write a Consumer client Nodejs
- My Thoughts about this tech
- Summary
timecodes
components 2:00
spin up docker rabbit 8:30
Write a Publisher client NodeJs 11:00
Write a consumer client NodeJs 20:30
my thoughts 33:50
Source Code: https://github.com/hnasr/javascript_playground/tree/master/rabbitmq
Example
Schedule async job
Exchange
Queues
Publisher
Consumer
AMQP
Channel
Connection
HTTP
AMQP
Uses Channels and Queues
Multiples channels into one connections
docker run --name rabbitmq -p 5672:5672 -d rabbitmq
docker run --name rabbitmq -p 5672:5672 -p 15672:15672 -d rabbitmq:3-management
HTTP
fetch("http://localhost:15672/api/vhosts”, {headers: {"Authorization" : Basic ${btoa('guest:guest')}}}).then(a=>a.json()).then(console.log)
fetch("http://localhost:15672/api/channels", {headers: {"Authorization" : Basic ${btoa('guest:guest')}}}).then(a=>a.json()).then(console.log)
fetch("http://localhost:15672/api/queues", {headers: {"Authorization" : Basic ${btoa('guest:guest')}}}).then(a=>a.json()).then(console.log)
https://www.squaremobius.net/amqp.node/channel_api.html#channel_bindExchange
https://www.rabbitmq.com/tutorials/tutorial-three-javascript.html
Support this podcast: https://anchor.fm/hnasr/support
Redis is a noSQL key-value store, in memory database first that exploded in popularity in the past few years. In this video slash course, we will explain what Redis is, talk about the in-memory model, the optional durability, replication, clustering publish subscribe and the protocol and finally we will go through examples. During the video I will add time codes to each topic so you guys can jump to the topic that interests you the most. Table of Content and TimeCodes Redis as a NoSQL In Memory Key-Value store 3:49Optional Durability 10:00Transport Protocol 15:10Pub/Sub 18:24Replication and Clustering 20:40Examples 23:41Spin Docker 26:10Command CLI 28:50Set key 30:30 get key 31:00Set key with expiry 31:10exists key 32:00del 32:30 append key 32:50 publish subscribe 33:30 Commands Docker run --name redis -p 6379:6379 redis docker exec -it rdb redis-cli Cards 4:25 ACID! Support me on PayPal https://bit.ly/33ENps4 Become A Patron https://www.patreon.com/join/hnasr?
Support this podcast: https://anchor.fm/hnasr/support
GraphQL Pros and Cons, examples and when to use over REST
GraphQL is an open source query language developed by facebook that allows clients to formulate queries to get different results. Its main goal is to combine multiple services into one endpoint. In this video we will discuss what is GraphQL, why facebook developed it, go through some examples using github GraphQL API, finally we will discuss the pros and cons and when you should use this technology.
- What is GraphQL?
- Examples
- Pros and Cons
- when to use REST vs GRAPHQL
What is GraphQL
- Schema
- Query language
- Nesting
- Mutation and subscription
Examples
Schema intro
Github API
Rest api
Pros
- Flexibility
- efficient response : payload back only get what you want of fields since you know the schema
- No round trips- Avoiding multiple round trips (HATEOS REST)
- Uniform single interface API endpoint
- Self documenting
Cons
- Complexity
- Typed system - ( use it to know if a type is available or not and fork logic) slows down adoption.. same as soap
- No Caching etag since always POST
- Error management non-standard for HTTP.
- Over engineering can lead to Inefficiency of the joins can lead to performance and DOS
Support this podcast: https://anchor.fm/hnasr/support
Consistency vs Eventual Consistency Consistency is the property of having expected result in view of the data or during reads while working with a database system. It is one of the ACID properties in relational databases. Eventual consistency is another term that was born recently specifcally as NOSQL databases got emerged. In this video we will talk discuss the difference the different kind of consistencies and we will explain what Eventual consistency and how both relational databases and NO SQL databases have this kind of consistency with some examples. Cache Leader Following Consistency in Data Your data broken into multiple normalized tables/collections is consistent. Consistency in Reads If you write a value new transactions will pick up that new value. If you do not have consistency in data you do not have eventual consistency, your data will not magically correct itself. If you have do not have consistency in reads you might have eventual consistency, you reads might eventually become consistent. Eventual Consistency means that your reads will become consistent as time pass time. This is true for both NOSQL and relational database system especially if you have leader/follower module. In this video we will talk about an example of eventual consistency and this is tolerable when it’s not. Eventually Consistency Benefits Twitter timeline (Eventual Consistency is good) Let’s say you tweeted something and you have a follower in Spain and a follower on New Zealand 🇳🇿 furthest two countries on Earth. Your Spain follower might see your tweet before your New Zealand one does. This depends on which datacenter your write goes to first. That eventual consistency is tolerable and its ok if New Zealand don’t see your tweets. Benefits Write scales much better, you can write to different locations and have them synced. Add more machine to scale to more and more users. Twitter privacy example (Eventual Consistency is bad) However take this scenario, you are Taylor Swift (Taylor swift wishes) a celebrity with 85 million followers. You tweeted something that you regretted later and decided to delete that tweet! That tweet better be deleted instantly to all your 85 million follows. Eventual consistency is not tolerable here otherwise people lose faith of the system. Well, you can always say well I’m gonna take a screenshot of Taylor Swift tweet. To which I would say I’ll give you another example, let’s say you changed your privacy setting to private and you tweeted something right after, that change should be immediately take effect and NO public user should see that tweet unless they are obviously in your followers. Problems: Users lose faith in the system. Cards 4:30 ACID https://www.youtube.com/watch?v=pomxJOFVcQs
Support this podcast: https://anchor.fm/hnasr/support
ACID ACID are four properties of relational database, they Atomocity, consistency, isolation and durability, and I think any one working with a relational database like postgres, mysql, sqlserver oracle, should understand these properties. In this video we will go through the four properties and explain why each is critical to make a relational database we will also talk about why some people are moving to NOSQL database Atomicity All or none. if a failure happened during transaction, db failure, or one of the queries failed. Example Isolation Concurrency, is transaction isolated from other inflight transactions? if a transaction is in flight does it see changes from other inflight transactions? Does is it see any changes? Does it only see committed changes. Does leading to inconsistent results. Problems arising from isolation (read phenomenons)
dirty reads Non repeatable reads Phantom reads
Isolation levels
Read uncommitted Read committed Repeatable read Serializable
Durability When I commit a transaction does my changes stay durable after the database restarts/crashes etc. See if your data still there. Consistency Consistency from referential integrity keys Does the number of likes on a picture = the number of rows that the picture got on another table? If a delete a picture does all the likes of that pictures go away on the other table. Consistency in reads If I committed something does everybody see it immediately or are they going to get an old value? Consistency in concurrency Is the view of a transaction in flight consistent? Are other inflight transactions making changes to the database affects that transaction view? Jump Codes 2:00 What is a Transaction? 4:30 Atomicity 7:00 Isolation * 9:30 Isolation - Read phenomena * 11:40 Dirty Reads 14:40 Non-repeatable Read 17:00 Phantom read 18:53 Isolation Levels*2 19:20 Read uncommitted 19:55 Read committed 21:05 Non-repeatable Read 23:40 Serializability 25:00 Isolation Levels vs Read phenomena Table 27:45 Consistency 28:30 Consistency in Data 33:50 Consistency in Reads 35:00 Eventual Consistency 40:30 Durability Cards 27:40 Exclusive lock vs shared lock
Support this podcast: https://anchor.fm/hnasr/support
REST stands for Representational state transfer its is an architecture that became very popular in build web APIs. It was the dissertation of Roy Fielding. In this video we discuss what makes an API RESTFUL, the REST APIs constrains, ill the show you an example of a RESTFUL api in github.
Representation and State transfer
Representational
The resource is a representation or meta data, but the actual backend could be something else and stored differently. An
Example, could be a user resource could be represented as a JSON object but it is stored on the backend as relation DBMS tables such as postgres.
State transfer
The application server is stateless, and when we want communicate we transfer the current state of with each request. Thus the state transfer.
Example, lets say you are uploading a 5MB file in 5 chunks each is 1 MB in size and assemble it on the backend. The REST api end point takes the content along with a upload sequence, then persist it on a storage backend such as S3. Each chunk request could hit a completely different stateless server and the transfer will work fine since we are transferring the state (upload sequence) with every request. The client maintains the state in this case.
Rest constraints
Client/server architecture
Is there separation of concern? Can you upgrade your server without upgrading client? Can you upgrade the server without upgrading the client?
Statelessness
Is your api stateless? Can you restart your backend server and clients of your api resume working normally without failing? Can you add a non sticky load balancer and transfer the load between the servers without the client breaking?
Cachablity
Can resources that can be cached be cached with your api? And is there a way to identify stale resources?
Layered systems
Can I insert gateways and proxies and firewalls silently without this architecture breaking? Load balancers
Uniform interface
Resource identification (uri)
Resource Representation (json)
HATEOAS
Hypermedia as an engine to application state
Initial link can link to the rest ( github)
Github
Emojis
Support this podcast: https://anchor.fm/hnasr/support
A proxy is a software that intercepts traffic and forward it to the destination on behave of the client. This extra layer provide several advantages such as caching, load balancing, content filtering and much more. Some implementations of proxy can be used by governments to spy on its citizens. We made a video about proxy vs reverse proxy check it out if you want to learn more about the difference. In this video we will explain the different types of HTTP proxies and the benefits and use cases of using each coming up.
Transparent proxy (gateway)
HTTP insecure proxy
Service Mesh Proxy
Transparent proxy
It is mostly used by the ISPs, clients don’t know they are connected to transparent proxy. The way it works is it looks at TCP/IP layer 4/3 and forward it to the destination, it might do some content filtering based on the IP address or the port so it blocks certain sites. But thats pretty much it. transparent proxy cannot know which pages are you viewing or your what youtube videos are you watching. It can block you from watching youtube all together but it cannot block you from watching lets say a specific youtube channel that is critical of the government ISP is located at.
Transparent proxy doesn’t change the content.
HTTP Proxy (insecure)
This kind of proxy is used alot, especially in service meshes like linkerd. This kind of proxy have to be configured in the client to use it. Each request will always be targeted to the proxy IP address / port. So when want to make a GET request to husseinnasser.com, and you have a proxy configured, when you look at the TCP packet for that request the destination IP and port is those of the proxy. The proxy looks at the GET request and specifically the HOST header and establishes another TCP connection to the actual destination on husseinnasser.com. So this kind of proxy maintains two tcp connections. Client to proxy and proxy to destination. The proxy have access to the content, it can block the website. It can know what exact page you are viewing. It knows everything because HTTP is insecure. Assuming youtube uses just HTTP, if you have a proxy setup it can block a specific channel or even video from being viewed.
1:05 proxy vs reverse proxy https://www.youtube.com/watch?v=ozhe__GdWC8
2:50 TLS https://www.youtube.com/watch?v=AlE5X1NlHgg
Kazakhstan government is now intercepting all HTTPS traffic
https://www.zdnet.com/article/kazakhstan-government-is-now-intercepting-all-https-traffic/
Support this podcast: https://anchor.fm/hnasr/support
HTTP is a protocol for transferring web pages, text, media, binary files and much more. It stands for hyper text transfer protocol and It is what the Internet pretty much runs on. In this video we will learn how HTTP works, how it is secured with HTTPS, will also show how to spin up an Http web server, and we will also go through the evolution of HTTP starting from HTTP 1.0 to HTTP 1.1 to HTTP/2 and eventually HTTP/3 which is still experimental.
HTTP anatomy
Request (browser, web app)
- URL
- Method type
- Headers
- Body
Response (web server)
- Status code
- Headers
- Body
HTTP 1.0 over tcp
- Application Layer 7
- new connection with each request.
HTTP 1.1 over tcp
- Persisted connection
HTTP/2 over tcp
- Compression
- Multiplexing
- Server Push
- SPDY
- Mostly secure by default
- Negotiates protocol during TLS (NPN/ALPN)
HTTP/2 over QUIC ( HTTP/3)
- Replaces TCP with QUIC (udp with congestion control)
Support this podcast: https://anchor.fm/hnasr/support
NAT network address translation is a process of mapping an IP address or IP port pair to another IP address or IP: port. You might be wondering what a software engineer like me doing making a video on a low level networking concept? I have good reasons for that. NAT was originally designed to solve the ipv4 limited IP addresses, but since been used for port forwarding and layer 4 load balancing through the virtual ip address such as Haproxy thats why I decided to make a video about NAT from a software engineer view. In this video we will explain how NAT works and we will explain its applications.
Support this podcast: https://anchor.fm/hnasr/support
Tunneling protocol
Tcp tunneling Tunneling is the process of encapsulating content from a protocol A into another protocol B, usually because protocol A is blocked or unavailable. In this video we will explain how TCP tunneling works, the applications of TCP tunnels and the pros and cons. Coming up!
- TCP Tunneling
- Applications
- Pros and Cons
TCP Tunneling Here is how TCP Tunneling works.
Lets say your goal is to access a website that your ISP proxy blocks www.server2.com this is hosted on server2 on port 80. Lets say there is another Server1 that you have access to and Server1 have direct access to Server2. So if you can make Server1 make the request on your behave to Server2 and somehow deliver the results back to you, you just created a tunnel between You and Server1.
Here is how it actually works.
You create a legit tcp connection over a known protocol such as SSH between you and Server1. You then create a tcp packet that is intended for Sever2 so you tag it with Server2:80. Then you package that packet into another TCP packet intended for Server1! Huh ! Server1:22. You then forward the packet over, your ISP police will see that there is a packet intended to Server1 on port 22. Proxy approves and forwards it over not knowing that you are smuggling content in that packet. Also the proxy cant even look in the content because its encrypted with RSA. Server1 unpacks the package, decrypt and discover that its an other tcp packet. Here is where the shady stuff happen. Server1 now looks and see that the smuggled package is intended for Server2:80, created a connection and delivers the package it, it changes the source ip to its self and keeps track somehow of that. Once it receives the package it knows that this package has to go back to tunnel. The client now have access to the blocked site! What does this look like guys? Yes you guessed it its a VPN.
It’s literally like smuggling content inside a package 📦 that looks legitimate.
Server1 and Server2 can be the same server
There are many types of tunneling
Local port forwarding: Remote connection, Socks Proxy: forward pretty much anything (VPN) Reverse Tunneling : Expose local web server publically
Applications VPN Securing an insecure connection Anonymity Bypass firewall SOCKS 4 proxy redirect all your traffic regardless of the port to an internal proxy instead which tunnels it. Dynamic port forwarding
Pros Secure connection Access blocked services Anonymity Expose internal traffic
Cons TCP meltdown (TCP over TCP) Slow retransmission Stateful
Local port forwarding Just one app gets forwarded when the local port is requested
Socks All apps goes through the proxy
Http tunneling
TCP VS UDP 1:00 11:00 OSI model 15:40 private vs public ip 18:35 proxy vs reverse proxy 24:30 TLS
11:20 local 16:20 reverse 20:40 socks
Support this podcast: https://anchor.fm/hnasr/support
TLS which stands for transport layer security is a protocol for securing communication between client and server. Specifically for HTTPS. Thats what the S is stands for.
In this video, we will learnq how insecure vanilla HTTP works, HTTPS, then we will learn how HTTPS is possible via the transport layer security and finally we will talk about the improvements in 1.3 that was published August 2018.
Vanilla HTTP
HTTPS
TLS 1.2 handshake
TLS 1.3 enhancements
Vanilla HTTP
Before we discuss TLS, HTTPS or anything else lets go through how HTTP request work. You can type in the browser www.husseinnasser.com , the OSI magic kicks in, client figures out the IP address of husseinnasser.com by calling the DNS which uses UDP. Then HTTP application layer makes a GET / request passes in the IP address and port 80 (default for insecure http). This creates an underlying TCP connection. GET / string among other stuff into the packet and send it over. TCP does its thing server receives GET / calls the appropriate process at the backend which could be just return index.html sets content type text/html and sends back big response for client. All of this obviously is plain text no encryption any kind and if you watched the OSI video we made you can tell that people can sniff/snoop packets and get packets they aren’t supposed to get
HTTPS
Works by negotiating a symmetric key so they can both secure messages. Watch the video we did on encryption. Before we jump to GET request there must be a handshake 🤝 that must occur between the client and server. The tricky part is exchanging that key. Same thing as above except port is 443 instead of 80. Remember once we lose the TCP connection we will have to renegotiate the key. But beauty of this is HTTP is stateless so it remains working just fine.
Tls handshake 🤝
The original TLS handshake involves 4 roundtrips. A client hello which the client includes which encryption algorithms it supports (Both symmteric and asymmetric). The server receives the request then replies back with the server certificate which includes the server public key and also the encryptions that they will change to. The client receives the server hello, generates the premaster key, encrypts it with the server’s public key then send it over. The Server decrypts the message, gets the premaster generates the symmetric key finally tells the client that we are good to go.
Tls 1.3
TLS 1.3 involves much shorter and much secure communication using only deffie hellman as key exchange and just two round trips.
Support this podcast: https://anchor.fm/hnasr/support
Encryption is the process of scrambling data to protect personal files, secure communication, hide identities and much more.
In this video we will learn about the different type of encryptions we will talk about symmetric encryption, asymmetrical encryption, where they are used for and the pros and cons of each one.
- Symmetric encryption
- Asymmetrical encrypt
- Pros and cons of sym va asym
Symmetric encryption
Might as well just call it classic encryption I would argue and i think this is the first encryption known to us. I have some thing I dont want anyone to see I use a lock key to lock it. Only I can open it unless I have a lock.
The same key you use to encrypt is the same key to Decrypt.
Examples
Examples of popular symmetric-key algorithms include
AES
Twofish
Serpent
DES
Twofish, Serpent, AES (Rijndael), Blowfish
CAST5, Kuznyechik, RC4, DES, 3DES, Skipjack, Safer+/++ (Bluetooth), and IDEA
Asymmetrical encryptions
We had symmetric encryptions for a long time, then internet came and networking and we needed to encrypt messages going back and forth. We said cool lets use AES. Then we said wait a second.. the other computer doesnt really have my key so we need to encrypt it..
Also called Public key encryption
1977
Rivest–Shamir–Adleman (RSA)
- Diffie–Hellman key exchange protocol
- DSS (Digital Signature Standard), which incorporates the Digital Signature Algorithm
- ElGamal
- Various elliptic curve techniques
- Various password-authenticated key agreement techniques
- Paillier cryptosystem
- RSA encryption algorithm (PKCS#1)
- Cramer–Shoup cryptosystem
- YAK authenticated key agreement protocol
Support this podcast: https://anchor.fm/hnasr/support
Bandwidth explained from software engineer point of view
Bandwidth is measured by how many bits a device is allowed to send/receive in a second. It ranges from your internal network starting from network card all the way to your ISP Internet speed. In this video we will discuss the definition of bandwidth upload vs download speed, the different usage patterns for normal web browsing, streaming, gaming, media production and cloud web servers etc and finally we will show an example of the whole thing.
When your ISP gives you 80Mbs download/1Mbs upload bandwidth, this means 80 mega bits download which means your router (and all your devices) can receive (download) a total of 80 mega bits in a second at a time, and it can send (upload) 1 mega bit in a second. With your ISP can you pick and choose your plan that fits your usage.
Usage patterns
Web Browsing
Most people download far more than they upload. So the download bandwidth matter more than upload bandwidth. For simple web browsing the download incoming traffic is slightly larger than the outgoing one. You make a GET request to pull index.html, you get back the html content. Yeah there are some images but still not much of incoming.
Streaming
Think about it when you watch a movie in Netflix you make a small request usually in few hundred bytes. But the data you receive (download) is huge. So streaming is constant large incoming data. While still outgoing traffic is minimum.
Gaming
The bandwidth for gamers is really interesting specially who play online and it really depends on the game. a game might use the server authoritative model, where all the online players send their inputs and receive the final game state changes from the server. In that case the there is a constant upload, still minor but the download bandwidth is larger since state is usually bigger. Games can also use the lock state model or the client authoritative model, where all the clients send their inputs the server and the server just send back all the input from all players back to everyone so everyone calculate the game state locally. So in this efficient model, both the upload and download is quite small and its good for countries with low bandwidth.
Some games require cloud saving which means upload speed is recommended.
In general latency is much better in gaming.
Media production
Youtuber and you make 5 videos a day so you need to upload alot of data to the youtube server. So upload speed really matter, and download also matter because you will also consume alot.
Web servers
Netflix (which is hosted in amazon) they upload data far more than they download. So their upload bandwidth is more than their download bandwidth. All those movies and tv shows pushed to consumers.
Example
Assume you have bandwidth of 80mb/s download speed (which is 10 Mega bytes per second) and 1mb/s upload (which is 125 KB per second). You are connected to a web server which has 80Gb/s upload and 80Gb/s download bandwidth, which is 10Gigabyte per second. Lets say you want to download a 60GB file or movie (no streaming). It will take the server 6 seconds to upload that file however it will take you 1.7 hours to download it. Your ISP That is not entirely true though for TCP, which implement congestion control. UDP does not implement congestion control.
Support this podcast: https://anchor.fm/hnasr/support
Denial of Service attacks
Dos attacks (denial of service) are type of attack on a server to prevent users from consuming a particular service, usually this is an HTTP web server. This could happen by either saturating the bandwidth of the pipe going to the server or by bringing the server down to its knees so it stops taking requests all together. In this video we will learn about 3 different types of DOS attacks and explain each one by example.
Bandwidth based DOS 2:15
- Dos Sending Huge amount of data to a server with lower bandwidth from a client with higher bandwidth which ends up saturating the server pipe and queue up future requests, new requests will have to wait or perhaps denied service. Example, the attacker have 100mb/s bandwidth (upload) the server has 10Mb/s download. If the attacker starts sending 100 mb worth of data to the server, it will take it 1 second to leave the pipe. However, The server can only download 10 mb each second for processing because thats its bandwidth, so it needs 10 seconds to completely download that 100mb and process. In this 10 seconds the server is fully busy serving just 1 client. Other requests will not be able to even reach the server, they might get queued and they may never be executed. Thus denied service. It is important to know that the server must have an end point that actually accept such large data. Like upload file with no limit. Another example, is UDP where there is no connection.
- Ddos this previous scenario is less likely since servers usually has much more bandwidth than a single computer. A common attack is to do a DOS in distributed manner. Assume a server with 1 Gb and client with 10 mb/s no matter how much data the client can send it can only send 10mb per second, and the server can go through them real quick. Example, the client sends 1GB, it will leave the client’s pipe into 100 (10mb) means the client will take 100 seconds just to upload all the data because it can only sends 10 mb each seconds. And the server is processing it so fast it each second and it will still have enough bandwidth to process other requests(1000-10). But imagine 100 users with 10 mb connection each, all coordinate to send 1 Gb worth of data to the server at the same time (critical that its in the same time) 100x10 each second they can send 1 Gb in total to the server, the server can only process 1 GB per second so the server will not be able to process any other requests because its bandwidth is saturated processing this 1 GB from different place. Make it 200 users and you just clogged the pipe.
Max connections based DOS 10:13
Another type of denial of service attack is by somehow force the server to reach its max connections. The web server usually sets a maximum number of tcp connections so that it doesn’t run out of memory. an attacker can perform a DOS attack to force the server to reach its max connection. once it does, it wont accept any more connections thus deny service of future requests. However it is not easy, web servers have good preventive measures to minimize unnecessary tcp connections. So you cannot just establish a connection and ghost the server. This isn’t your ex boyfriend. Server has good timeouts for connections that are idle, terminated or potentially harmful. However one possible attack is to establish a connection but send the data slowly so when the server tries to timeout it immediately reset the timeout and keep the connection alive! Assuming the max tcp connection is 200, Run your script 200 times and you just created 200 connections to the server so no new connection can connect.
Vulnerability based DOS 16:30
Support this podcast: https://anchor.fm/hnasr/support
TCP and UDP are communication protocols that allows us to send and receive data in a network. We have both for a reason since each has its advantages and disadvantages. In this video we will talk about two protocols, pros and cons of each one and will write tcp and udp server with nodejs showing you these in ACTION. coming up
Code!
https://github.com/hnasr/javascript_playground/tree/master/tcp
Jump Codes 🏷
tcp pros cons(4:12)
Tcp demo (19:00)
Udp pros cons (24:25)
Udp demo (31:25)
Summary (35:40)
Explain TCP
Pros:
acknolwedgment,
garenteed delivery,
connection based
congestion control
ordered packets
Cons
Larger packets
More bandwidth since more headers.
Slower (because of retransmission)
stateful once connection is lost its lost, cannot resume it.
TCP Example (Code)
telnet 127.0.0.1 8080
Explain UDP
Pros
Statless,
Less bandwidth
smaller packets
Faster to transmit
Cons
no ack,
no garenteed delivery,
no congestion control
no ordered packets
UDP client
echo "foo" | nc -w1 -u 127.0.0.1 41234
2:24 osi model tag
14:20 stateful. S stateless video
Support this podcast: https://anchor.fm/hnasr/support
Get and POST are the most popular http methods used on the web. Each carries its own differences and properties. It can confusing to get to choose when to use POST over GET. In this podcast we will explain the differences, use cases and the benefits of using GET and POST.
Support this podcast: https://anchor.fm/hnasr/support
Microservices (Explained by Example) Microservices technology is a new pattern of software engineering that has been popularized recently. In this video we will explain what microservices are, their pros and cons by example. A lot of companies have moved in the early 2010 such as twitter and netflex to the microservices architecture. Microservices technology is a pattern where you can break down big application into smaller services and what previously is a function call is now a network call GET or POST through HTTP. In order to explain what microservices are, we need to explain how a traditional classical application looks like. Here is an example .. Instagram View Picture, list comments, likes and Picture and Location. Picture Likes Comments Picture API (Thumbnails) Likes API Comments API Most popular comments Pros: Polyglot architecture Easy scaling for microservices that needs scaling. Better Team management, each microservice is a team Easier to innovate certain areas. Each microservice can pick their own database Scale busy services instead of entire system Cons: Very complicated to implement, network call, service discovery Very Difficult to debug Hard to find where the fault is Network calls fails adds complexity. 5:00 6:50 proxy vs reverse
Support this podcast: https://anchor.fm/hnasr/support
Attribute Rules are scripts that can be authored and placed in the geodatabase to ensure data integrity lets discuss them.
Support this podcast: https://anchor.fm/hnasr/support
In this episode we will talk about the difference between public ip and private ip address and casually talk about routers, modems, LAN, WAN and port forwarding.
Support this podcast: https://anchor.fm/hnasr/support
Join me in the upcoming Esri dev summit! Ill be presenting attribute rules and the utility network.
Support this podcast: https://anchor.fm/hnasr/support
It is a long weekend with the president day on Monday. Which means it is another good opportunity to create more art. Whether that is a new ArcGIS pro Add-in or a new map, a painting, a new piece of poetry or music. This also means more chances to get discovered, which means more chances to get criticized. I get criticized all the time now that I have bigger exposure and this is our topic for this podcast coming up,
Support this podcast: https://anchor.fm/hnasr/support
This is originally a video that I converted into a podcast I think it is still useful https://www.youtube.com/watch?v=8D1NAezC-Dk
Notifications have become part of our daily life. We receive them almost every minute on our phones, tablets, and computers. As software engineers, it is good to understand how notifications work, and what different type of notifications there are (push and pull notifications), the advantages and disadvantages of each. In this video I'll explain how the push notification and pull notification works and the stateful vs stateless nature of each and how it affects the architecture decision of our application that we are building.
Online diagram tool used in this video:
Http://www.gliffy.com
Become a better software engineer by joining IGeometry checkout the other software engineering content on this channel
-Hussein
Support this podcast: https://anchor.fm/hnasr/support
Soap (Simple Object Access Protocol) has been a popular messaging protocol in the early 2000s. It uses XML as a format with well defined schema, and your choice of stransport protocol. Despite it still being used people have been moving towards #REST architecture and more so #gRPC recently. In this episode we discuss why people moved from #SOAP and what are the advantages and disadvantages of SOAP. Stay Awesome! Hussein Nasser
Support this podcast: https://anchor.fm/hnasr/support
In this episode we answer a question from Anthony. How do you know when to focus on building a geodatabase (let's say SQL server database) rather than just let the data sit in the shapefile?
I would say if you are planning to have multiple users accessing and editing the data, also if your data grew too large shape files might not be scalable to read. Also if you are planning to use the capabilities of Geodatabase such as domains, subtypes, attribute rules, versioning, archiving etc not to mention sharing as well. if you wont plan to use any of the above then shape file is a good choice. If you are planning to just maintain a shape file data of 10k or 100k features and you edit it yourself locally it works!
Support this podcast: https://anchor.fm/hnasr/support
When you publish a service you can specify how many processes (min / max) your service can use. As requests are being served memory is allocated, state is changed, some processes could go corrupt Recycling is the process destroying and re-spawning processes that are running to maintain a good memory footprint.
Here is a good read on recycling http://enterprise.arcgis.com/en/server/latest/publish-services/windows/tuning-and-configuring-services.htm
Enjoy Hussein
Support this podcast: https://anchor.fm/hnasr/support
I don’t have the equipments. There are people doing that already. And my personal favorite, I don’t know enough. Excuses are always there, In this podcast I talk about my personal story of how I got started.
You don't need fancy equipments to share your skills and knowledge. In this podcast we discuss how with just a screen sharing app (Quicktime) and this microphone I was able to grow my youtube software engineering channel to 2000 subs and more than 1 million watch hours.
Support this podcast: https://anchor.fm/hnasr/support
A podcast about the difference between a layer and its data source. Enjoy!
Support this podcast: https://anchor.fm/hnasr/support
Vikrant asked a question on Linked:
Dear Hussein, I want to learn programming for GIS. Is python useful? I am not very good in programming. Kindly guide. And thanks for the posts..
Programming for GIS really depends on what you are trying to do and how do you want to advance your career. In this podcast we will discuss four kind of areas where you can programming for GIS.
GIS Admin Programming
GIS Desktop Programming
GIS Web Programming
GIS Server Programming
Enjoy
Support this podcast: https://anchor.fm/hnasr/support
If you ever go a website and there some images you liked to reference in your page or blog. You would copy the link, and put in your blog.
Only to find out that the image looks broken?
You copy the link again and paste it in URL and it opens fine?
This is an effect of CORS or cross origin resource sharing.
https://husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
ArcGIS Server is a technology that allows you to pull your heavy work load on the backend and isolate the clients from complexity of the database by exposing HTTP services instead of low level TCP database. You can also extend the server capability by writing your own extensions.
In this episode ArcGIS Server talk we will talk about two ways you can extend your ArcGIS server, Server Object Extensions and Server Object Interceptors.
Server Object Extension
Extends the MapServer to allow you to write your own logic that cannot be easily done by built - in Feature Server or Map server. Or it is more performant to do it.
Users create new excavation I want to auto-calculate the price of the excavation based on the soil type, whether there are trees, rocks.
Server Object Interceptors
You don’t write new thing you instead intercept existing calls to query and applyEdits to inject your own logic to pre or post processing. Lets say every-time any application tries to write to the service, you want to verify that the edit is permissible. Include water marks to all images that are downloaded.
Learn more about SOE vs SOI here
http://enterprise.arcgis.com/en/server/latest/publish-services/linux/about-extending-services.htm
Support this podcast: https://anchor.fm/hnasr/support
Any content served through HTTP “should” include meta data about its type. This is so the browser/client knows what to do with the content it receives. For example, if the content type header is an image the browser will preview it, if it is HTML it will render the markup and execute any javascript code.
Content type however is optional and web masters sometimes don’t set it, which leave the browsers wondering about the content type it is consuming. So browsers had to implement parsing and “sniffing” techniques to detect the type of content when a content type header was not served.
However, this caused security problems and attacks that we explain in this video! So to prevent sniffing, web servers can return X-Content-Type-Options: nosniff which opts out browsers from sniffing the content.
Media type: https://en.wikipedia.org/wiki/Media_type#Common_examples
Cheers!
Hussein Nasser
Support this podcast: https://anchor.fm/hnasr/support
Thank you so much guys for a great year! I love you.
-Hussein
2018 was a great year:
Created, edited and posted 120 software engineering youtube videos with over 1.1 Million impressions
Published 75 podcasts with over 8k plays
Self Published a new GIS book
Published two programming online courses #hustle
Support this podcast: https://anchor.fm/hnasr/support
This is the final episode of 2018. We will discuss the difference between Availability vs Scalability in software services. Availability
What does it mean for my software to be available? software, service, application is designed in way so it can be available to consumers despite situation Will your service be available if
Host Failure Maintenance Upgrading Security updates
Available doesn’t necessarily mean performant.
Scalability
What does it mean for my software to be scalable? The ability for the software to handle the increase of workload or data growth Assume photo sharing app, retrieving 1 photo with details. If in normal conditions you service can serve 100 requests per minute, will doubling resources double the output?u If the volume of data increases will your query still perform? A software is scalable when it is designed in a way so that adding more resources will handle more users
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss the concept of subtypes in the geodatabase, subtype layers and much more!
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss the difference between authentication and authorization.
Support this podcast: https://anchor.fm/hnasr/support
Hussein's Courses 📐⟶ https://www.husseinnasser.com/courses Hussein's Books 📒⟶ https://www.husseinnasser.com/books
In this episode of #softwaretalk I'm in beautiful San Francisco for the weekend. Our topic today is database locks.
Stay Awesome! Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss the ArcGIS scripting language Arcade. New to GIS? Checkout my books and courses here https://husseinnasser.com/books
Support this podcast: https://anchor.fm/hnasr/support
We discuss the difference between short and long transactions in the geodatabase
Support this podcast: https://anchor.fm/hnasr/support
We discuss consistency in this episode
Support this podcast: https://anchor.fm/hnasr/support
In this episode I talk about how the feature service came to be. From local shape files through the enterprise geodatabase to the beautiful HTTP protocol services. Https://husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
Hussein's Courses 📐⟶ https://www.husseinnasser.com/courses Hussein's Books 📒⟶ https://www.husseinnasser.com/books
I don’t usually do rants. f you are new software developer, pick a language and write software. There is no best programming language. Each programming language has its own of pros and cons.
Stay Awesome -Hussein
Support this podcast: https://anchor.fm/hnasr/support
In this episode I explain the backend development, its history, how it came into existence and where it is going. I also share with you my latest online course Python on the Backend. http://www.husseinnasser.com/courses
Support this podcast: https://anchor.fm/hnasr/support
We discuss the HTTP protocol and explain its elegance and how it is powering the microservices architecture.
Support this podcast: https://anchor.fm/hnasr/support
Hussein's Courses 📐⟶ www.husseinnasser.com/courses Hussein's Books 📒⟶ www.husseinnasser.com/books
ObjectId (among others) is a system field that is added to a geodatabase table. In this episode we discuss best practices in coding against this field. Checkout my new book (learn gis programming with ArcGIS online and Javascript 4.9 API) ! www.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss the difference between the HTTP client pull, server push and the long poll also known as #HTTP ASYNCHRONOUS polling.
Support this podcast: https://anchor.fm/hnasr/support
As a software engineer, I like to reverse engineer applications and see how it was made. This episode we discuss how YouTube handles audio vs video streaming. When you minimize the app,
Support this podcast: https://anchor.fm/hnasr/support
Port 6080 is interesting, we talk little bit about it in this episode.
Support this podcast: https://anchor.fm/hnasr/support
If you go to the geoprocessing service in Manager you might have noticed that there is an option to run asynchronously. In this episode we explain when you might want to do this!
Support this podcast: https://anchor.fm/hnasr/support
Sde intercept Any operation performed on ArcGIS ends up eventually to the geodatabase as a set of one or more queries. A “zoom-in” or pan operation translates to multiple queries to the underlying geodatabase.
SDE intercepts always you to intercept and take a look at those queries to diagnose slow operations in your enterprise geodatabase. It also helps pinpoint where the bottle neck is and how can you optimize your app or geodatabase. https://support.esri.com/en/technical-article/000010355
Support this podcast: https://anchor.fm/hnasr/support
Most of the time a Load balancer is a reverse proxy but a reverse proxy is not necessary a load balancer. Reverse proxy takes a request from client and forward it to “a server” doesn’t care of the server is overloaded or down or not just forwards it. Load balancer stores state about each server behind it, how overloaded each server is, and what server is down, and smartly forward request to a server to balance the load across all the group. So a load balancer is a special case of a reverse proxy but smarter!
Support this podcast: https://anchor.fm/hnasr/support
We discuss the max heap size property on the arcgis server admin. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
In this episode of arcgis server talk we discuss the database schema lock option in manager. Advantages and disadvantages of having your service acquire a lock.
Support this podcast: https://anchor.fm/hnasr/support
We discuss how split work, the split policy on the domain and the split editing model.
Support this podcast: https://anchor.fm/hnasr/support
In this episode of #softwaretalk we discuss the art of software troubleshooting. We can become better software engineers by aiming to excel at our craft. Not just find a bug, but track its cause. Not just find a bad performing workflow in the app but find out what causes the performance bottleneck. Do the due diligence.
Enjoy this episode guys www.husseinnasser.com
Support this podcast: https://anchor.fm/hnasr/support
In this episode, we talk about applyEdits method on the feature service REST API. Enjoy
Grab my Administering ArcGIS for Server book and learn more about Installing and configuring ArcGIS for Server to publish, optimize, and secure GIS services.
http://husseinnasser.com/books
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss how using the new ArcGIS Server version management service we can undo edits even after the service goes down!
Support this podcast: https://anchor.fm/hnasr/support
In software development, introducing a new feature or functionality always introduces bugs specially in a large software. In this episode we discuss should we fix all the bugs that get introduced from a new feature or install the feature and worry about the bugs later? #agile
Support this podcast: https://anchor.fm/hnasr/support
Throw back me using PHP and apache to build a stateful service.
Support this podcast: https://anchor.fm/hnasr/support
In this episode of #softwaretalk, we discuss the differences between the software product architect and a solutions architect. We start by defining the difference between a software product and a solution. Then we discuss the responsibilities of product architect vs solution architect.
If you are interested to be a solution or product architect or engineer you came to the right place.
Cheers Hussein
Support this podcast: https://anchor.fm/hnasr/support
Archiving is used to store or keep track of state of rows even after they are edited in older state. In this episode we discuss how can you do archiving in the geodatabase.
Support this podcast: https://anchor.fm/hnasr/support
In this episode of ArcGIS Server Talk we discuss logging.
Support this podcast: https://anchor.fm/hnasr/support
In this episode we explain the difference between ddl and dml, with respect to the geodatabase
Support this podcast: https://anchor.fm/hnasr/support
The communication between the client and the enterprise geodatabase is done through the spatial database engine. We explain that layer in this episode.
Support this podcast: https://anchor.fm/hnasr/support
Any modern software or application that we build must have a backend API (Application Programming Interface) whether design in C++ IMPL Interfaces Or HTTP REST Architecture, and a Front End User Experience to consume this API which could range from a mobile app to a voice device. This is an open discussion, when you as a software engineer want to design a new application, what do you start first? Front-End User Experience? Or the Back-end API? We discuss the pros and cons of both. I enjoyed making this episode!
Enjoy guys Hussein Nasser
Support this podcast: https://anchor.fm/hnasr/support
We explain how the asynchronous geoprocessing service works in ArcGIS. We also briefly explain the difference between synchronous vs asynchronous request. Enjoy !
Support this podcast: https://anchor.fm/hnasr/support
There is no secret, no magic potion, no pill or book or a “new” programming language that will make you a better software engineer. To be a better software engineer you have to write more software. Write silly software, write fart software, write stupid software, write bad performant software. Write enough software to give the edge to be better than other software engineers and software developers.
Support this podcast: https://anchor.fm/hnasr/support
Today’s topic is geodatabase domains.
Support this podcast: https://anchor.fm/hnasr/support
Welcome to the first episode of software engineering talk. This series we pick a topic and casually talk about it. In today’s episode, we try to answer the question is Coding easy?
Support this podcast: https://anchor.fm/hnasr/support
In this episode of software engineering by example we discuss how software engineers should be empathetic and never underestimate and brush a problem as simple. We have to give it some thoughts and think it all the way through.
Support this podcast: https://anchor.fm/hnasr/support
This episode I explain what is a stateful connection and how that compare to traditional vs the new branch versioning model. We also explain Pros and cons of the stateful connection (with a DMV example :) )
Support this podcast: https://anchor.fm/hnasr/support
We explain the difference between Isolation and High Isolation from a geodatabase point of view.
Support this podcast: https://anchor.fm/hnasr/support
We discuss briefly the 5 registration types in the enterprise geodatabase. Unversioned, unversioned with archiving, versioned, versioned with archiving and branched versioned.
Support this podcast: https://anchor.fm/hnasr/support
We discuss the concept of versioning (high isolation) in the geodatabase.
Support this podcast: https://anchor.fm/hnasr/support
We discuss the geodatabase connection which is called a workspace,
Support this podcast: https://anchor.fm/hnasr/support
We discuss the table artifact in the geodatabase.
Support this podcast: https://anchor.fm/hnasr/support
This is a new series that discusses the Esri geodatabase technology!
Support this podcast: https://anchor.fm/hnasr/support
Discussing Connection Pooling in ArcGIS Server
Support this podcast: https://anchor.fm/hnasr/support
Discussion about server site, port 6080,6443, configuration store and creating and joining existing site
Support this podcast: https://anchor.fm/hnasr/support
Discussing arcgis server technology part 1
Support this podcast: https://anchor.fm/hnasr/support
Discussion on Persisted storage in the disk while taking a walk in Bahrain 🇧🇭 . Enjoy.
Support this podcast: https://anchor.fm/hnasr/support
In this podcast I talk about my story. Where did I come from? What did I study? What jobs did I have? Why did I start my blog and Youtube Channel? How did I write my books? Why did I move to the United States? and much more..
Stay Awesome guys and Enjoy the podcast. Hussein
Support this podcast: https://anchor.fm/hnasr/support
Discussion on GIS geographic information system my career since 2005. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
We discuss we some people are intermediated by programming or find it hard to learn. Spoiler alert it is our fault as educators.
Support this podcast: https://anchor.fm/hnasr/support
This episode is pulled from my vlog right after I finished Gary vee book Crush it. Today I finished @garyvee book Crush it (Cash in on Your Passion). I read alot of books but dont necessary review them in my SM. However this book is different, though written 9 years ago and some technologies are dead but that in itself shows this guy jumps on bleeding edge technologies like there is no tomorrow which I love. Never mind, that book actually inspired me to twist Gary’s ideas in the software engineering realm. And that is the topic of the vlog, you are passionate about 8 bit video game development? Build a community around it. You really genuinely love the assembly language? Build software with assembly and make videos and show your love, you will be surprised how many engineers share your interest. Cant wait for Gary’s new book Crushing it! - I would really like to thank my YouTube mentor @robertoblake for introducing me to such passionate person.
Support this podcast: https://anchor.fm/hnasr/support
New technologies appear every day. We ask the question should software engineers learn new emerging technologies as soon as they come in? We answer yes and We bring in badass Gary vee as a successful example of someone who tackle any new social media app.
Support this podcast: https://anchor.fm/hnasr/support
We discuss how JSON became popular. It isn’t some fashion trend, its convenience. We also discuss how XML was invented as an interchange format.
Support this podcast: https://anchor.fm/hnasr/support
Discussion of latency across the entire software stack. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
We discuss relational databases. Their properties and scalability.
Support this podcast: https://anchor.fm/hnasr/support
I discovered git submodules today. I talk about how I found them and how they helped my code reusability.
Support this podcast: https://anchor.fm/hnasr/support
We answer the question can Python runs on the backend and serve REST end points
Support this podcast: https://anchor.fm/hnasr/support
We discuss memory leaks. What are they? How they happen? How dangerous can they be?
Support this podcast: https://anchor.fm/hnasr/support
We discuss how an engineer can save tremendous design and architecture efforts at the backend building the REST services by actually starting with the user experience. As usual with an example. Cheers y’all
Support this podcast: https://anchor.fm/hnasr/support
We talk about how the we are meant to create beautiful art and how the resistance (the dragon) is stopping us from putting that work out there.
Support this podcast: https://anchor.fm/hnasr/support
While building an Alien Invasion javascript game I wrote some bad code that froze my chrome. I discuss the unscalable piece of code and go through how I discovered it and how to fix it.
Support this podcast: https://anchor.fm/hnasr/support
With new technologies it becomes overwhelming for engineers to learn. We discuss one of the best ways to learn the new technologies by doing projects using them. Step out of your comfort zone and build apps using those tech!
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss why lot of people think Developers make unusable and lousy user experiences and interfaces. We take examples of Windows vs Mac and a throw back from a blog post I wrote 10 years ago. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
We discuss stateless and stateful client / server architecture using instagram as an example.
Support this podcast: https://anchor.fm/hnasr/support
We discuss how persistence at the client side. We bring Siri, Whatsapp and Twitter s examples of dealing with persistence of draft, lost and unsent messages .
Support this podcast: https://anchor.fm/hnasr/support
We discuss software scaling in three tiers. The front end, the backend and the middle. We take instagram as an example for all three tiers.
Support this podcast: https://anchor.fm/hnasr/support
This podcast is for software engineers that just started their career and facing pressure from outside world to divert their path. Keep doing what you love.
Support this podcast: https://anchor.fm/hnasr/support
Does the user experience derive the software architecture or vise versa? Can user experience help determine what backend database you end up using? We discuss both styles of engineering and talk about pros and cons. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
In this episode we discuss how can you make a change to your career. I also talk about my experience a little. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
We discuss two programming styles both have their advantages and disadvantages. Enjoy
Support this podcast: https://anchor.fm/hnasr/support
First podcast - intro - welcome to IG podcast
Support this podcast: https://anchor.fm/hnasr/support