Service Discovery_System Design Notes 7

I. Application Layer

In a simple 3-tier structure, the web service layer handles both requests and business functions:

A better structure is to separate the web layer from the application layer (also called platform layer):

Advantages include:

Can scale the application layer independently: allows adding machines independently, switching to dedicated machines, etc.
Reuse infrastructure: simplifies multi-end support, caching, database processing can all be reused
Makes organizations easier to scale: one team is responsible for implementing/optimizing the platform itself, other multiple teams use platform features for development

After separating the application layer, the next problem faced is how to divide responsibilities within the application layer, how to collaborate, which is the problem microservices architecture aims to solve

II. Microservices Architecture

Microservices architecture advocates designing applications as a series of loosely coupled fine-grained services, organized through lightweight communication protocols:

In a microservices architecture, an application is structured as a collection of loosely coupled services, which are fine-grained and the protocols are lightweight.

These services can all be deployed independently, scaled independently, each service has solid module boundaries, even allows using different programming languages to write different services, and can be managed by different teams:

P.S. For more information about microservices architecture, see Microservices Architecture

Under microservices architecture, applications are split into multiple services, each running in different processes (on different machines):

If each microservice only runs on a single machine, one microservice can find other dependent services through static configuration tables, then complete collaboration through inter-service communication

However, in actual scenarios, 1 microservice is usually deployed on multiple machines, and dynamically scales as needed (adding/removing machines), simple static configuration obviously cannot satisfy, thus needs a service registration query mechanism:

That is Service Discovery

III. Service Discovery

Client-Side Service Discovery

Client queries service registry, gets a series of addresses of target service, and selects one to initiate request based on load balancing strategy (i.e., client-side load balancing)

Among them, service registry is used to store all available service instances, and provides management (register/deregister) and query APIs:

The service registry is a database of available service instances. The service registry provides a management API and a query API. Service instances are registered with and deregistered from the service registry using the management API. The query API is used by system components to discover available service instances.

Specifically, when starting service instances, add its network location to registry, remove record when stopping service, and during service instance operation, periodically refresh registration information through heartbeat mechanism

P.S. For example Netflix Eureka provides REST API to register/deregister, query service instances:

Netflix Eureka provides a REST API for registering and querying service instances. A service instance registers its network location using a POST request. Every 30 seconds it must refresh its registration using a PUT request. A registration is removed by either using an HTTP DELETE request or by the instance registration timing out. As you might expect, a client can retrieve the registered service instances by using an HTTP GET request.

And supporting Netflix Ribbon, used as client-side load balancing

This pattern is relatively simple, and clients can make smarter (such as application-specific) load balancing decisions, but also has some disadvantages:

Each language used by client must be implemented once
Need to maintain a highly available registration service yourself
Service discovery related logic is all implemented in client, such as retry, causing client to be relatively heavy

DNS-SD

Specially, can use DNS as service registry, called DNS-SD (DNS-based Service Discovery)

Complete one-to-many mapping from service to instances through DNS SRV records:

SRV record (Service locator record): General service location record, specifies server where service is located (domain name and port number), mostly used for SIP (Session Initiation Protocol)

(Extracted from Resource Records | DNS_System Design Notes 3)

For example:

$ nslookup -query=SRV _http._tcp.backends.example.com 10.0.0.2
Server:		10.0.0.2
Address:	10.0.0.2#53

_http._tcp.backends.example.com	service = 0 2 8090 backend-0.example.com.
_http._tcp.backends.example.com	service = 0 1 8091 backend-1.example.com.
_http._tcp.backends.example.com	service = 10 1 8092 backend-2.example.com.

Although using DNS is simple and easy to operate, but limited by DNS update timeliness (cache problems)

Server-Side Service Discovery

Of course, query process can also be completed on server side:

Client requests target service through load balancer, load balancer queries registry to get a set of available instances, and selects one to initiate request based on load balancing strategy

P.S. For example AWS Elastic Load Balancer (ELB)

Under this pattern, clients no longer need to implement service query logic for various languages, different frameworks, simply initiate requests to load balancer, but if deployment platform doesn't provide this capability, need to build and maintain such a highly available system component yourself

IV. Service Registration and Deregistration

In Service Discovery, service instances must register to service registry, and deregister in time, divided into self-registration and third-party registration 2 patterns

Self-Registration Pattern

Under self-registration pattern, service instances are responsible for registering themselves to service registry, and deregistering from it, if necessary, also send heartbeat requests to stay active, avoid registration expiration

This method is relatively simple, doesn't rely on other system components, but service instances and service registration mechanism become coupled, resulting in registration logic needing to be implemented once in clients of various languages, different frameworks

P.S. Netflix OSS Eureka client adopts this pattern, Eureka client handles service instance registration and deregistration

Third-Party Registration Pattern

Service instances no longer responsible for registration/deregistration, handed over to service registrar to handle, decouples coupling relationship between service instances and registration mechanism. Registrar tracks service instance operation status by polling deployment platform or subscribing to events, registers when discovering new service instances, deregisters when discovering service instances stopped

P.S. Registrator adopts this pattern, supports automatic registration/deregistration of services deployed with Docker containers

Specially, deployment platform controls service instance startup and shutdown, it's most appropriate for it to complete registration, deregistration. In fact, deployment platforms like Kubernetes and Marathon all provide service registration, query capabilities. Specifically, use proxy services running on each node in cluster as load balancer in server-side Service Discovery, clients send requests to proxy, proxy service forwards to available instances on other nodes in cluster

V. Summary

Microservices architecture is responsible for splitting services and decoupling dependencies, while Service Discovery is used to solve communication problems between these services, allowing one microservice to find another

In implementation, divided into client-side Service Discovery and server-side Service Discovery two types, difference lies in whether query/selection logic is implemented on client side or server side. And service registration/deregistration can be completed by service itself (self-registration), or by deployment platform and other third parties (third-party registration)