PolyCode Technical Architecture Document

With Oauth 2.0, it’s important to understand the different terms. Here is a list of the most important of them :

There is mutliple flow to authenticate a user. Here is the most common ones, and his differences.

A access token permits to access to a protected resource. But, it’s not enough to access to all the resources. The authorization server can restrict the access, depending on the scope of the access token.

The score are asked by the client, in the query. Once the user is authenticated, the authorization server ask the user if he want to grant the requested scope, by displaying a consent page. If the user accept, the authorization server return the access token, with the requested scope.

Here is an explaination of the JWT parts :

The payload of a JWT contains claims. Claims are statements about an entity (typically, a user) and additional metadata.

There are three types of claims :

Here is an payload of an basic ID token used in OpenID Connect, with the definition of each claims :

Once the user is authenticated, the authorization server return an access token, valid for a certain amount of time. Once this token expires, the client can ask for a new one, using the refresh token. Here is a diagram of the flow to get a new token :

Here is the terminologies used in PKCE :

When the client send the authorization request to the authorization server, he also send the code challenge and the code challenge method (but not the code verifier).

Then, when the client send back the authorization code to the authorization server, he also send the code verifier. The authorization server then hash the code verifier and compare it to the code challenge. If they match, the authorization server send back the access token. Here is a diagram of this flow :

There is multiple scopes in OIDC. Let’s see the most common ones :

Here is the endpoints defines in OIDC :

KeyCloak is an open source identity and access management solution. It’s a single sign-on solution that allows you to manage users, applications, roles, groups, etc. It’s an Oauth 2.0, OpenID Connect and SAML 2.0 compliant solution.

With the graphical interface, you can manage easily and quickly your resources. It supports multiple factors authentication and social login, all with a lot of adapters to integrate it in your application.

Here is an image of the keycloak architecture :

In the case of PolyCode, we will use KeyCloak to manage the authentication and authorization of our users and services.

There is multiple other solutions to manage the authentication and authorization of your users and services. Because we will use KeyCloak for PolyCode, we have detailed it in the previous section.

Here is a list of other possible solutions :

Here is a diagram of the wanted architecture :

The goal is to have a KeyCloak instance, with a client for each service. Each service will have a client secret, and will use it to authenticate to KeyCloak.

The user, when he wants to access to some resources, will be redirected to KeyCloak. He will authenticate with his credentials (or create new), and will be redirected back to the PolyCode UI with a code. The UI will then exchange this code with KeyCloak, and will get an access token. This token will be used to authenticate to PolyCode, verify by KeyCloak against the Polycode API, and will be used to access to the resources.

To do this, we will follow multiple steps. This document is based on the official KeyCloak documentation.

First, we need to setup a KeyCloak instance. You can use the docker image, or install it on your server. You can run a KeyCloak instance in dev mode with the following command :

Once the KeyCloak instance is running, you can access it at localhost:8080 with the admin/admin credentials if you used the command above.

Once logged in, you can create a new realm. A realm is a group of users, applications, and clients. You can create multiple realms in KeyCloak, and each realm is isolated from the others. For PolyCode, we will create a polycode realm.

A client is an application that wants to use KeyCloak to authenticate users. You can create multiple clients in a realm. For PolyCode, we will create two clients :

Each client have multiple settings :

For better security, you can also uncheck the Implicit Flow option.

In our case, we don’t need to configure users, roles, groups, etc. because authorization will be managed by the API.

Now, we need to integrate KeyCloak in PolyCode. We will explain how to do this with a POC. Here is the project links :

To run the POC, you need first to run the backend, then the frontend, and finally the runner. The backend contains already a KeyCloak instance (not configured).

Then, you can access the polycode-keycloak application at localhost:3001.

Here is the deployment diagram of the future PolyCode authentication architecture :

All communications between those services are in HTTP, and are secured with TLS.

Here is some sequence diagrams to explain the differences between the authentication process modes.

Synchronous communication is a type of communication where the sender service waits for a response from the invoker service before continuing. There are two main mechanisms for synchronous communication: REST and RPC.

Asynchronous communication is a type of communication where the sender service does not wait for a response from the invoker service before continuing. Let’s speak about the most popular mechanisms for asynchronous communication : Message queues and Publish-subscribe.

An API gateway is a single entry point for clients to access the functionality of a distributed application. The API gateway is responsible for routing requests to the appropriate microservice, and for aggregating the results. It works as a reverse proxy, and it can be used to implement security, rate limiting load balancing and other functionality.

The following diagram shows a simple example of an API gateway:

In this example, the API gateway is responsible for routing requests to the appropriate microservice, and for aggregating the results.

The circuit breaker pattern is a mechanism that allows a service to fail fast and recover quickly. It is used to prevent cascading failures in a distributed system. A circuit breaker is a state machine that can be in one of two states: closed or open. When the circuit breaker is closed, the service can be called. When the circuit breaker is open, the service cannot be called. The circuit breaker can be in the open state for a fixed amount of time, or it can be in the open state until a certain number of calls have been made.

The following diagram shows a simple example of a closed circuit breaker:

In this example, the circuit breaker is closed, and the service can be called. Until the service succeeds, the circuit breaker remains closed.

The following diagram shows a simple example of an open circuit breaker:

In this example, the circuit breaker is open, and the service cannot be called. The circuit breaker will remain open for a fixed amount of time, or until a certain number of calls have been made. When the circuit breaker is open, the client receives an error message by the circuit breaker.

The bulkhead pattern is a mechanism that allows a service to limit the number of concurrent connections it can handle. It is used to prevent cascading failures in a distributed system. It applies when multiple applications needs to connect to a component by requesting a connection to that component.

The following diagram shows a simple example of the bulkhead pattern:

This pattern works as follows:

The sidecar pattern is a mechanism that allows a service to add additional functionality to a microservice, such as logging, monitoring, or authentication. It is typically implemented as a separate process that runs alongside the microservice, such as a service proxy.

The following diagram shows a simple example of the sidecar pattern:

With this pattern, each microservice is composed of two parts: the microservice and the sidecar. When a request is made from or to the microservice, the sidecar intercepts the request and adds additional functionality, then forwards the request. Let’s explain in detail how it works.

The following diagram shows the traffic flow when a request is made to the microservice, with a service proxy sidecar:

When a request is made to the microservice, the service proxy intercepts the request (and can adds additional functionalities), then forwards the request to the microservice. The microservice processes the request, and returns the response to the service proxy (and can too adds additional functionalities), which forwards it to the client.

A service mesh is a dedicated infrastructure layer for handling service-to-service communication. It works with a sidecar proxy that is deployed alongside each service. The sidecar proxy intercepts all network communication between microservices, and adds some functionnalities.

By this way, each microservice does not have to be coded with inter process communication logic, but only with the business logic. The service mesh is responsible for the inter process communication logic.

The most popular service mesh is Istio. It is an open source service mesh that provides traffic management, policy enforcement, and observability. Istio provide all following features:

Here is a schema of the Istio service mesh, with the envoy proxy sidecar:

A service registry is a key part of service discovery. It’s a database that contains the network location of all the services in a distributed system. He must be highly available. Some softwares implements this functionality, such as Consul, etcd, ZooKeeper, or Netflix Eureka.

Note that some systems such Kubernetes, Marathon and AWS have their own service registry implementation.

In those solutions, there are two common patterns for a service to register itself to the service registry: Self-registration and Third-party registration.

In client-side discovery, the client is responsible for discovering the location of the other services by query the service registry.

The following diagram shows a simple example of client-side discovery:

in this schema, each service, once started, will register itself to the service registry.

The client will then query periodically the service registry to get availiable locations of the services. If a service is no longer available, the service register will delete its list of available services, then after the next periodic updating of the customer, it will no longer try to connect to this service.

In server-side discovery, the server (load balancer) is responsible for discovering the location of the other services by query the service registry. A load balancer is a server that distributes network or application traffic across a number of services.

The following diagram shows a simple example of server-side discovery:

In this schema, as well for the client-side discovery, each service, once started, will register itself to the service registry.

The load balancer will then query the service registry periodically to get availiable locations of the services. If a service is no longer available, the service register will delete its list of available services, then after the next periodic updating of the load balancer, it will no longer try to connect to this service.

The server-side discovery has several benefits:

The mailer service is responsible to send emails to users or candidates. It is a service that is used by others. For example, when a user is invited in a team, the Account service will call the Mailer service to send the email.

This use case does not need a response from the mailer, because if a letter is not received, there is still a way to send it again. Especially since we would have no guarantee than the mail was well received and read.

Therefore, we will use asynchronous communication for this service. Since we only have this service that uses it, we will not use pub/sub pattern, but a simple queueing system.

In our previous example, the Account service will send a message to the event queue, and the Mailer service will consume this message, and send the email.

The benefits of this pattern are:

The drawbacks of this pattern are:

The chosen the queueing system RabbitMQ. There is no particular reason for this choice, except that it is the most widely deployed open source message broker.

Here is the schema of the Mailer microservice communication with RabbitMQ message broker:

For the other microservices, we will use synchronous communication. The reason is that we need a response from the service. For example, when a user query his profile data to the Account service, a call to the Authentication service is made to check token validity. In this case, we need a response before sending the data to the user.

For all our microservices, we will use a REST API. The reason is that it is a well-known pattern, and it is easy to use. We don’t need to use a more complex protocol such as gRPC, because we don’t need to optimize the communication between our microservices for now.

For the service discovery, we will use the third-party registration pattern. We will use the Consul service registry. The reason is that it is a well-known service registry. We will use the self-managed version of Consul (open source) with kubernetes, because the PolyCode application is not deployed in a cloud environment, but on private servers, at Polytech.

Here is the schema of the PolyCode service discovery with Consul:

In this schema, we can see that each microservice is deployed in a kubernetes pod. Each pod has the microservice and a sidecar container, that is responsible to register, deregister and query the service. The sidecar container is made up of two parts: the dataplane and an envoy proxy. The envoy proxy is responsible to communicate with the service, and make load-balacing, and the dataplane with the consul control plane. The consul control plane is responsible to keep the global state of the cluster.

Here is a sequence diagram of a basic request, from the Account service to the Authentication service, to verify the token validity:

The code review is a very important part of the development process. It is a way to ensure that the code is written in a secure way, and that it follows the best practices. It is also a way to ensure that the code is written in a way that is easy to maintain, and that it is easy to understand. It is always good to have another point of view on written code.

Examination of the code can be carried out by someone in the same company, or by external developers, with security audits.

Additionally, it is necessary to keep an eye on the dependencies used for the development of applications. security failures are frequently discovered, and security updates are then necessary.

There are tools to detect it automatically, such as Dependabot (GitHub), or Snyk.

Security testing is a way to ensure that the code written code works as expected. There are several ways to do it, and we will explain three of them.

If you use a gateway, you can encrypt the communication between the gateway and the web. This is the most easy way to add encryption. You only have to add a TLS certificate to the gateway, then all the outgoing communication will be encrypted. Note that all internal communication will not be encrypted.

Here is a diagram showing the gateway level encryption:

Here, the gateway provide a TLS certificate to the web. The web will then be able to communicate with him in a secure way.

You can also encrypt the communication between each microservice and the gateway (or the web, if you donc have a gateway). This is a more complex way, but it is also the most secure one.

Here is a diagram showing the sidecar level encryption:

Here, each microservice have a sidecar that provide a TLS certificate. With this approach, all communication is encrypted (internal and external). The only exception is the communication between the sidecar and the microservice, but this communication is not exposed to any other network.

It’s strongly recommended to use this approach, if possible.

Rate limiting is a way to limit the number of requests that can be sent to a microservice. It is a way to prevent a service from being overloaded, or to prevent a service from being attacked, such as a DDoS attack, or a brute force attack.

There are several ways to do it, such as the Nginx rate limiting module, or the Envoy rate limiting module.

Service authentication and authorization is a way to ensure that only known and authorized services can communicate with each other.

HashiCorp Vault is a tool for securely storing and accessing secrets. It provides a centralized platform for managing sensitive information. Vault can also be used to generate dynamic credentials for services and applications, and to control access to those secrets through role-based access controls.

Here is a diagram showing the flow of HashiCorp Vault:

As you can see, a service can get the secrets from Vault, using its own credentials. Vault will then check if the service is authorized to get the secrets with the RBAC, and if it is, it will return them.