Self-Hosted Server
Sometimes you may want to run your own Yorkie Server. This page explains how to run a Yorkie Server.
To run a Yorkie Server, you need to install the CLI. If you haven't installed it yet, please refer to CLI.
Running a Server via CLI
Let's start a Server with the CLI. You can start a Server with the following command:
```bash$ yorkie serverbackend created: id: c6ljcdl94818baveba8g, rpc: localhost:11101: db: memoryserving admin on 11103serving RPC on 11101serving profiling on 11102
The Server handles administrative requests such as project maintenance, and it continues to run until it's told to quit with termination commands such as Ctrl+C. The Server stores its data using an in-memory DB, which does not provide durability by default.
Persistence
If you start the Server with a MongoDB address, you can permanently save the data stored by Yorkie.
To start MongoDB, docker-compose -f docker/docker-compose.yml up --build -d
in the project root.
Then, start a Yorkie Server with --mongo-connection-uri
flag to connect the MongoDB.
$ yorkie server --mongo-connection-uri mongodb://localhost:27017MongoDB connected, URI: mongodb://localhost:27017, DB: yorkie-metabackend created: id: c6ljibt948102kkt9neg, rpc: localhost:11101: db: mongodb://localhost:27017serving profiling on 11102serving RPC on 11101
Server for Web
Server uses gRPC to provide an API that Clients can connect to. Since it is currently impossible to implement the HTTP/2 gRPC in some browsers, Envoy is required for web. For more details: gRPC-web
This page shows how to start the Server for web. Overall structure is as follows:
Browser Envoy Server┌────────┐ ┌──────────────┐ ┌───────────┐│gRPC-web├─HTTP1.1─┤gRPC-web Proxy├─HTTP2─┤gRPC Server│└────────┘ └──────────────┘ └───────────┘
Configuring Envoy by hand with its config file is cumbersome, but using Docker Compose makes it easy.
First, download all manifests files from docker folder. Then, execute the following command in the folder containing the downloaded file.
$ docker-compose up --build -dStarting yorkie ... doneStarting envoy ... done
This will launch Yorkie(Server) and envoy containers on your environment.
$ docker psIMAGE COMMAND PORTS NAMESgrpcweb:envoy "/usr/local/bin/envo…" 0.0.0.0:8080->9090/tcp envoyyorkieteam/yorkie:latest "yorkie server --ena…" 0.0.0.0:11101-11103->11101-11103/tcp yorkie
Then, the ports of the services are bound to the host environment.
- 8080: gRPC-web port for SDK
- 9090: gRPC-web port for Admin Server is used by Dashboard(Dashboard) or CLI
- 11101: gRPC port for SDK
- 11102: HTML port for Profiling Server
- 11103: gRPC port for Admin Server
Now, let's create a Client with address https://api.yorkie.dev
.
1const client = new yorkie.Client('https://api.yorkie.dev');2await client.activate();
Next, let's take a look at the JS SDK.
Monitoring
Server exports metrics under the /metrics
path on its profiling port.
The metrics can be fetched with curl:
$ curl http://localhost:11102/metricsyorkie_server_version{server_version="{{site.version}}"} 1# HELP yorkie_pushpull_received_changes_total The total count of changes included# TYPE yorkie_pushpull_received_changes_total counteryorkie_pushpull_received_changes_total 6...
This metrics can be collected from Prometheus.
Running Prometheus and Grafana is the easiest way to monitor Server's metrics.
First, download all manifests files from docker folder. Then, start the applications with docker-compose
:
$ docker-compose -f docker-compose-monitoring.yml up --build -dCreating prometheus ... doneCreating grafana ... done
Now, Prometheus will collect Server metrics every 10 seconds.
Grafana has a built-in Prometheus support; just add a Prometheus data source:
Name: prometheusType: PrometheusUrl: http://localhost:9090Access: proxy
Then, import the default Yorkie dashboard template and customize it. For instance, if Prometheus data source name is my-prometheus
, the datasource field values in JSON also need to be my-prometheus
.
Sample dashboard:

Cluster Mode
In a production environment, it is generally expected that more than one server handles requests. Even if a Server goes down, the other Servers must be able to handle the request. We can achieve that by setting up Servers with Cluster-Mode.
This page describes how to set up a cluster of Servers. An example of the cluster is as follows:
┌───────────┐│ ┌──────┐ │┌─┼─►│Server│ │ ┌───────┐│ │ └──▲───┘ │ ┌─►│MongoDB│┌──────┐ ┌────────┐ │ │ │ │ │ └───────┘│Client├─►│Load ├──┤ │ Broadcast ├─┤└──────┘ │Balancer│ │ │ DocEvents │ │ ┌────┐└────────┘ │ │ │ │ └─►│etcd││ │ ┌──▼───┐ │ └────┘└─┼─►│Server│ ││ └──────┘ │└───────────┘
- Load Balancer: It is responsible for distributing the load of the requests to the Servers.
- Broadcast Channel: It is responsible for broadcasting the events to all Servers.
- MongoDB: It stores the data of Yorkie.
- etcd: It is used to store the state of the cluster such as MemberMap, SubscriptionMap, etc.
First, we need components such as etcd, MongoDB, and Load Balancer in order to set up the cluster.
Various types of load balancer can be used for the cluster. For CodePair, We have configured the cluster on AWS, so we use ALB as Load Balancer.
For testing cluster mode, we can use Docker Compose to run applications such as etcd and MongoDB. To start them, type docker-compose -f docker/docker-compose-ci.yml up --build -d
in the project root.
If etcd and MongoDB are ready, we can run Servers in cluster mode. In the terminal, run the Server with flags for MongoDB and etcd.
$ yorkie server --mongodb-uri mongodb://localhost:27017 --etcd-endpoints http://localhost:2379MongoDB connected, URI: mongodb://localhost:27017, DB: yorkie-metaetcd connected, URI: [http://localhost:2379]backend created: id: c6n20nnblaroaopqdbp0, rpc: localhost:11101: db: mongodb://localhost:27017serving profiling on 11102serving RPC on 11101
Open a new terminal and run the Server by using xxx-port
flags to avoid ports conflicts.
$ yorkie server --rpc-port 21101 --profiling-port 21102 --mongo-connection-uri mongodb://localhost:27017 --etcd-endpoints http://localhost:2379MongoDB connected, URI: mongodb://localhost:27017, DB: yorkie-metaetcd connected, URI: [http://localhost:2379]backend created: id: c6n25o7blarohq3ve250, rpc: localhost:21101: db: mongodb://localhost:27017serving profiling on 21102serving RPC on 21101
Now both Servers are ready to receive requests from Clients.