所需知識

MongoDB Sharding基本組成知識，可以參考此篇: MongoDB Sharding with Docker
docker-compose.yaml 撰寫知識

yaml檔案撰寫環境

OS

Windows 10

Tools

npm
Visual Studio Code
Robo 3T 1.3 (1.4有些shard的指令沒辦法跑)

架設環境

Rocky-Linux 8.5

MongoDB Sharding架構

筆者使用1個mongos，1個僅有primary的cfg，2個PSA Shards。

mongos: 1
cfg: 1 (only primary, without secondary)
shard : 2 (PSA)

docker-compose.yaml 撰寫

image採用 MongoDB® Sharded packaged by Bitnami
使用筆者所寫之CLI程式 mongodb-sharding-bitnami-yaml-generator 產生docker-compose.yaml檔。

安裝docker-compose.yaml產生器

請確定環境已有npm來安裝產生器

npm install -g mongodb-sharding-bitnami-yaml-generator

撰寫docker-compose.yaml產生器設定檔

檔案名稱 mongo-sharding.config.json

{
    "generator": {
        "projectName": "", //e.g. `hello` 將會產生`hello-mongodb-monogos`service名稱，空白則產生`mongodb-mongos`
        "shardCount": 1, //要產生的shard數量
        "useSameShardConfig": true //是否所有shard都用與第一項一樣的設定(PSA,PSS)，如果為否，請在`shards`設定與`shardCount`數量一樣的設定
    },
    "mongos": {
        "replicaSetKey": "replicaSet123", //請設定至少5個字，並不包含特殊字元。
        "rootPassword": "password" //密碼
    },
    "shards": [
        {
            "secondaryCount": 1, //secondary的數量
            "enableArbiter": true //是否要有arbiter
        }
    ],
    "cfg": {
        "secondaryCount": 0 //secondary的數量
    }
}

執行docker-compose.yaml產生器

mongodb-sharding-bitnami-yaml-generator -c mongo-sharding.config.json -o docker-compose.yaml

產生之docker-compose.yaml

version: '3.4'
services:
  mongodb-mongos:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-mongos
      - MONGODB_SHARDING_MODE=mongos
      - MONGODB_CFG_PRIMARY_HOST=mongodb-cfg-primary
      - MONGODB_CFG_REPLICA_SET_NAME=replicaSet-cfg
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_ROOT_PASSWORD=password
    volumes:
      - ./mongodb-shard/mongos-data:/bitnami
    ports:
      - '27017:27017'
  mongodb-cfg-primary:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-cfg-primary
      - MONGODB_SHARDING_MODE=configsvr
      - MONGODB_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=primary
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-cfg
    volumes:
      - ./mongodb-shard/cfg-data:/bitnami
  mongodb-shard0-primary:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard0-primary
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=primary
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard0
    volumes:
      - ./mongodb-shard/shard0-primary-data:/bitnami
  mongodb-shard0-secondary0:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard0-secondary0
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_INITIAL_PRIMARY_HOST=mongodb-shard0-primary
      - MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=secondary
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard0
    volumes:
      - ./mongodb-shard/shard0-secondary0-data:/bitnami
  mongodb-shard0-arbiter:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard0-arbiter
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_INITIAL_PRIMARY_HOST=mongodb-shard0-primary
      - MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=arbiter
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard0
    volumes:
      - ./mongodb-shard/shard0-arbiter-data:/bitnami
  mongodb-shard1-primary:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard1-primary
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=primary
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard1
    volumes:
      - ./mongodb-shard/shard1-primary-data:/bitnami
  mongodb-shard1-secondary0:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard1-secondary0
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_INITIAL_PRIMARY_HOST=mongodb-shard1-primary
      - MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=secondary
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard1
    volumes:
      - ./mongodb-shard/shard1-secondary0-data:/bitnami
  mongodb-shard1-arbiter:
    image: docker.io/bitnami/mongodb-sharded:4.4
    environment:
      - MONGODB_ADVERTISED_HOSTNAME=mongodb-shard1-arbiter
      - MONGODB_SHARDING_MODE=shardsvr
      - MONGODB_MONGOS_HOST=mongodb-mongos
      - MONGODB_INITIAL_PRIMARY_HOST=mongodb-shard1-primary
      - MONGODB_INITIAL_PRIMARY_PORT_NUMBER=27017
      - MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password
      - MONGODB_REPLICA_SET_MODE=arbiter
      - MONGODB_REPLICA_SET_KEY=replicaSet123
      - MONGODB_REPLICA_SET_NAME=replicaSet-shard1
    volumes:
      - ./mongodb-shard/shard1-arbiter-data:/bitnami

架設

docker-compose.yaml寫完了，開始來啟動mongodb-sharding的服務吧。

新建volume資料夾

由於bitnami的官方文件有提到以下訊息，所以先新增資料夾以及更改權限

ℹ️

(ie. mkdir [directory] && chown 1001:1001 [directory] && chmod 777 [directory]) to all directories.

新建資料夾

mkdir mongodb-shard && mkdir mongodb-shard/mongos-data \
mongodb-shard/cfg-data \
mongodb-shard/shard0-primary-data \
mongodb-shard/shard0-secondary0-data \
mongodb-shard/shard0-arbiter-data \
mongodb-shard/shard1-primary-data \
mongodb-shard/shard1-secondary0-data \
mongodb-shard/shard1-arbiter-data

更改權限

chown -R 1001:1001 mongodb-shard && chmod -R 777 mongodb-shard

啟動服務

docker-compose up

結果

可以看到各元件都已經運行中

測試Sharding

啟用sharding以及新增資料
於Robo 3T連結後運行以下指令

use users
sh.enableSharding("users")
sh.shardCollection("users.users" , {
    _id : "hashed"
})
for(var i=1;i<10000; i++) {
    db.users.insert({
        userid:"user_"+i,
        username:"name_"+i,
        age: NumberInt(_rand()*100)
    })
}

取得資料分布在Shard的狀態

use users
db.users.getShardDistribution()

MongoDB sharding 架設 (Bitnami yaml 產生器)