CLI
CLI is a command-line interface for Yorkie. You can use CLI to manage your projects, documents or to start the server.
Getting started
Let's get started with the CLI. There are two ways to install the CLI: pre-built binaries and Homebrew.
Install pre-built binaries
The easiest way to install Yorkie is from pre-built binaries:
- Download the compressed archive file for your platform from Releases.
- Unpack the archive file. This results in a directory containing the binaries.
- Add the executable binaries to your PATH. For example, rename and/or move the binaries to a directory in your PATH (like /usr/local/bin), or add the directory created from the previous step to your PATH.
- Test if Yorkie is in your PATH in shell:
$ yorkie versionYorkie: 0.4.18Commit: ...Go: ...Build date: ...
Install via Homebrew
Homebrew is a free and open-source package management system for Mac OS X. Install the official Yorkie formula in Terminal.
To update Yorkie to the latest version, update Homebrew first.
$ brew update
Now, install Yorkie.
$ brew install yorkie
Verify that the installation worked by opening a new Terminal session and trying Yorkie's available subcommands.
$ yorkie versionYorkie: 0.4.18Commit: ...Go: ...Build date: ...
Commands
Yorkie is just a single command-line application yorkie, which takes a subcommand such as server.
The Yorkie CLI is a well-behaved command-line application. In erroneous cases, a non-zero exit status will be returned. It also responds to -h and ---help as you'd most likely expect.
To get help for any specific command, pass the -h flag to the relevant subcommand. For example, to see help about the server subcommand:
$ yorkie server --helpStart Yorkie serverUsage:yorkie server [options] [flags]Flags:--auth-webhook-cache-auth-ttl duration TTL value to set when caching authorized webhook response. (default 10s)--auth-webhook-cache-size int The cache size of the authorization webhook. (default 5000)--auth-webhook-cache-unauth-ttl duration TTL value to set when caching unauthorized webhook response. (default 10s)--auth-webhook-max-retries uint Maximum number of retries for an authorization webhook. (default 10)--auth-webhook-max-wait-interval duration Maximum wait interval for authorization webhook. (default 3s)--backend-admin-password string The password of the default admin. (default "admin")--backend-admin-token-duration duration The duration of the admin authentication token. (default 168h0m0s)--backend-admin-user string The name of the default admin user, who has full permissions. (default "admin")--backend-secret-key string The secret key for signing authentication tokens for admin users. (default "yorkie-secret")--backend-snapshot-interval int Interval of changes to create a snapshot. (default 1000)--backend-snapshot-threshold int Threshold that determines if changes should be sent with snapshot when the number of changes is greater than this value. (default 500)--backend-snapshot-with-purging-changes Whether to delete previous changes when the snapshot is created.--backend-use-default-project Whether to use the default project. Even if public key is not provided from the client, the default project will be used for the request. (default true)--client-deactivate-threshold string Deactivate threshold of clients in specific project for housekeeping. (default "24h")-c, --config string Config path--enable-pprof Enable runtime profiling data via HTTP server.--etcd-dial-timeout duration ETCD's dial timeout (default 5s)--etcd-endpoints strings Comma separated list of etcd endpoints--etcd-lock-lease-time duration ETCD's lease time for lock (default 30s)--etcd-password string ETCD's password--etcd-username string ETCD's user name-h, --help help for server--hostname string Yorkie Server Hostname--housekeeping-candidates-limit-per-project int candidates limit per project for a single housekeeping run (default 500)--housekeeping-interval duration housekeeping interval between housekeeping runs (default 30s)-l, --log-level string Log level: debug, info, warn, error, panic, fatal (default "info")--mongo-connection-timeout duration Mongo DB's connection timeout (default 5s)--mongo-connection-uri string MongoDB's connection URI--mongo-ping-timeout duration Mongo DB's ping timeout (default 5s)--mongo-yorkie-database string Yorkie's database name in MongoDB (default "yorkie-meta")--profiling-port int Profiling port (default 11102)--rpc-cert-file string RPC certification file's path--rpc-key-file string RPC key file's path--rpc-max-requests-bytes uint Maximum client request size in bytes the server will accept. (default 4194304)--rpc-port int RPC port (default 11101)
Login
To use admin commands, you need to log in to the server. You can log in to the server using the login subcommand:
$ yorkie login -u <username> -p <password> --rpc-addr api.yorkie.dev:443
Username and password are the same as the ones you used to log in to the dashboard.
If you are using the self-hosted server, default username and password are admin and admin.
You can log in to the server using yorkie login -u admin -p admin --insecure.
Once you log in to the server, the CLI stores the access token in the ~/.yorkie/config.json file.
You can check the context of the CLI using the context subcommand:
$ yorkie context lsCURRENT RPC ADDR INSECURE TOKENapi.yorkie.dev:443 false eyJhbGciOi...DuhaUgofYg* localhost:11101 true eyJhbGciOi...FUT3js73Mw
If you want to log out from the server, you can use the logout subcommand:
$ yorkie logout$ yorkie context lsCURRENT RPC ADDR INSECURE TOKEN* api.yorkie.dev:443 false eyJhbGciOi...DuhaUgofYg
Project
Project represents your service or application in Yorkie. Separate Projects can exist within a single application. You can add more Projects as needed, for example, if you want to manage auth webhook and documents for specific purposes.
To manage Projects, you can use the project subcommand:
$ yorkie projectManage projectsUsage:yorkie project [command]Available Commands:create Create a new projectls List all projectsupdate Update a projectFlags:-h, --help help for projectUse "yorkie project [command] --help" for more information about a command.
Listing Projects
You can list Projects on the server using ls.
$ yorkie project lsNAME PUBLIC KEY SECRET KEY AUTH WEBHOOK URL AUTH WEBHOOK METHODS CLIENT DEACTIVATE THRESHOLD CREATED ATdefault cfurlpq6qdop9v48o0u0 cfurlpq6qdop9v48o0ug [] 24h 15 minutes
Server creates a default project automatically. If you create a Client without apiKey, the Client is created in the default Project.
Creating a Project
You can create a new Project with a name using create.
$ yorkie project create test-project{"id":"63fdbebab48d9fd33996b38a","name":"test-project","owner":"","auth_webhook_url":"","auth_webhook_methods":null,"ClientDeactivateThreshold":"24h","public_key":"cfurtei6qdop9v48o0v0","secret_key":"cfurtei6qdop9v48o0vg","created_at":"2023-02-28T08:43:38.192448Z","updated_at":"2023-02-28T08:43:38.192448Z"}
If you create a Client with public_key of the Project as apiKey, you can manage the Client in the Project.
1const client = new yorkie.Client('https://api.yorkie.dev', {2 apiKey: 'cfurtei6qdop9v48o0v0', // public_key of the project3});
Then Documents attached to the Client are isolated and stored in the Project.
Updating the Project
You can update the Project on the server using update.
Usage:yorkie project update [name] [flags]Examples:yorkie project update name [options]Flags:--auth-webhook-url string authorization-webhook update url--client-deactivate-threshold string client deactivate threshold for housekeeping-h, --help help for update--name string new project name
You can update the name of the Project with --name flag.
$ yorkie project update test-project --name new-test-project{"id":"63fdc721b48d9fd33996b38b","name":"new-test-project","owner":"","auth_webhook_url":"","auth_webhook_methods":null,"ClientDeactivateThreshold":"24h","public_key":"cfuse8a6qdop9v48o100","secret_key":"cfuse8a6qdop9v48o10g","created_at":"2023-02-28T09:19:29.269875Z","updated_at":"2023-02-28T09:27:06.0588974Z"}
Document
Document is a data structure that can be synchronized between multiple clients. With document commands, you can manage documents in the project.
$ yorkie documentManage documentsUsage:yorkie document [command]Available Commands:ls List all documents in the projectremove Remove documents in the projectFlags:-h, --help help for documentUse "yorkie document [command] --help" for more information about a command.
Listing Documents
You can list documents in the project using ls.
Usage:yorkie document ls [project name] [flags]Flags:--forward Whether to search forward or backward-h, --help help for ls--previous-id string The previous document ID to start from--size int32 The number of document to output per page
$ yorkie document ls test-projectID KEY CREATED AT ACCESSED AT UPDATED AT SNAPSHOT6482cea1c38d5a06529bbaa7 test-document-5 ... ... ... ...6482cea0c38d5a06529bba96 test-document-4 ... ... ... ...6482ce9fc38d5a06529bba85 test-document-3 ... ... ... ...6482ce9dc38d5a06529bba69 test-document-2 ... ... ... ...6482ce97c38d5a06529bba1b test-document-1 ... ... ... ...
There are several flags to control the output of the command. With the flags you can paginate the output.
You can use --size flag to limit the number of documents to output per page.
$ yorkie document ls default --size 1ID KEY CREATED AT ACCESSED AT UPDATED AT SNAPSHOT6482cea1c38d5a06529bbaa7 test-document-5 ... ... ... ...
You can use --previous-id flag to start after a specific document.
$ yorkie document ls default --previous-id 6482ce9fc38d5a06529bba85ID KEY CREATED AT ACCESSED AT UPDATED AT SNAPSHOT6482ce9dc38d5a06529bba69 test-document-2 ... ... ... ...6482ce97c38d5a06529bba1b test-document-1 ... ... ... ...
You can use --forward flag to search forward or backward.
$ yorkie document ls default --forwardID KEY CREATED AT ACCESSED AT UPDATED AT SNAPSHOT6482ce97c38d5a06529bba1b test-document-1 ... ... ... ...6482ce9dc38d5a06529bba69 test-document-2 ... ... ... ...6482ce9fc38d5a06529bba85 test-document-3 ... ... ... ...6482cea0c38d5a06529bba96 test-document-4 ... ... ... ...6482cea1c38d5a06529bbaa7 test-document-5 ... ... ... ...
Removing a Document
You can remove a document from the project using remove.
Usage:yorkie document remove [project name] [document key] [flags]Examples:yorkie document remove sample-project sample-document [options]Flags:-h, --help help for remove
$ yorkie document remove test-project test-document-1
Auth Webhook
A webhook is an HTTP POST that is called when something happens. If the Auth Webhook has been configured, when a Server receives a request from a Client, the Server checks if the Client has been authorized for a certain Document by asking an outside service with a REST request.
This page shows how to set up an Auth Webhook. The overall flow is as follows:
(5) response the request (4) handle the request┌─────────────────┐ ┌──┐│ │ │ │ (3) response▼ │ ▼ │ - allowed┌──────┐ ┌┴─────┤ - reason ┌──────────────┐│Client├────────────►│Server│◄──────────────┤Outside Server│└──────┘ (1)request └────┬─┘ └──────────────┘- token │ ▲- dockey └──────────────────────┘(2) call webhook- token- dockey- verb: r or rw
First, We need to pass some tokens (that identify users in the service) when creating a Client:
1const client = new yorkie.Client('https://api.yorkie.dev', {2 token: SOME_TOKEN,3});
This token will be sent to the Server upon every request from the Client.
We can specify an Auth Webhook for the project by update command with the --auth-webhook-url flag:
$ yorkie project update [name] --auth-webhook-url http://localhost:3000/auth-hook
The Server who receives the token calls the given webhook URL before processing the requests.
Here is an example of the webhook requests:
1{2 "token": "SOME_TOKEN", // token passed by the client3 "method": "PushPull", // method: ActivateClient, DeactivateClient, AttachDocument, DetachDocument, WatchDocuments4 documentAttributes: [{5 key: "DOCUMENT_KEY", // document key6 verb: "r" // 'r' or 'rw'7 }]8}
The outside server should respond like this:
1{2 "allowed": true, // or false if the given token is not authorized for this document.3 "reason": "ok" // [optional] reason for this response.4}
If the server receives a response with allowed: true from the outside server, it handles the request normally, otherwise it sends the codes.Unauthenticated error to the client.
Client Deactivate Threshold
Client Deactivate Threshold is a time duration that determines the time for client deactivation of documents in projects. Deactivating the client and detaching it from documents is needed to increase the efficiency of GC removing CRDT tombstones. For more information about GC, please refer to Garbage Collection.
You can set client deactivate threshold of the project with --client-deactivate-threshold flag.
$ yorkie project update test-project --client-deactivate-threshold 1h30m20s{"id":"63fdc721b48d9fd33996b38b","name":"test-project","owner":"","auth_webhook_url":"","auth_webhook_methods":null,"ClientDeactivateThreshold":"1h30m20s","public_key":"cfuse8a6qdop9v48o100","secret_key":"cfuse8a6qdop9v48o10g","created_at":"2023-02-28T09:19:29.269875Z","updated_at":"2023-02-28T09:19:41.7616869Z"}
Setting the client deactivation threshold to 1h30m20s will deactivate clients that have been inactive for 1 hour, 30 minutes, and 20 seconds but still remained activated. This will help document GC by not counting deactivated client's document sequence.