MongoDB分片集群的搭建和使用

avatar 2019年11月22日22:19:43 评论 1,056 次浏览

MongoDB shard切片原理

1、先看一下mongodb的集群角色分布

在一个MongoDB集群中,会有三种角色:shard、config和routing

shard:每个shard节点都会包含数据集的一部分,是存储真正数据的节点;

config:主要存储元素菊或者配置信息,比如每一个切片的片键的范围,mongos(即routing)会根据这个服务的配置信息来决定到那个分片服务器上存取数据

routing:也就是mongos客户端,用来操作shard集群的接口。routing信息存存储在mongos上,主要是因为这些配置信息会动态改变的,因此直接提交给另一个进程更好。

通常,每一个shard都会是一个复制集,主要为了健壮性,因为切片集群一个挂一个切片,那么整个切片的数据都会丢失。每一个config更应该是一个复制集,因为一旦config挂了,那么整个集群就用不了了,所以在MongoDB的最新版本中,强制配置config节点为至少3个节点的复制集,这是很有必要的。

2.片键:

指的就是文档中的一组属性,mongos会根据这个属性来查看该条数据应该属于哪一个shard分片。片键的路由有两种实现方式,一种是范围,一种是哈希,其实本质上都是范围。

范围法:把片键的取值划分为很多区间,每一个区间对应一个分片(其实是chunk,后面再说);

哈希法:先根据片键值计算哈希值,然后把哈希值按范围划分,只是比范围法多了一步哈希。使用哈希法可以使得键值分布更加均匀。

3.chunk,块,理解这个概念很重要。块是比分片更小的单位,每一个shard分片都是由若干个chunk构成的,chunk有以下一些属性:

min和max,该chunk里最小和最大的片键,也就是这个chunk的数据范围,这其实是对应了我们之前说的片键区间。片键的每一个区间段都会对应一个chunk。每一个shard的配置信息就会包括其中所有的chunk块的范围,最终mongos就可以根据这些值来判定应该走哪一个切片的哪一个chunk。

maxSize,chunk大小的上限,这个和后面的内容有关,是一个阈值,会触发chunk的分割。

4.chunk分裂:

MongoDB不希望chunk块的size太大,所以一旦超过了maxSize,就会执行一次split操作,把chunk分割为几个更小的chunk。当发生insert或者update操作时,可能会除触发split操作。

5.balancer:

balancer是MongoDB里的一个后台进程,会检测每一个分片的chunk数目,一旦发现某一些分片的chunk数目太多,就会触发一次chunkMove,会把多出来的chunk移到chunk较少的分片上。每当chunk太大被分裂时,就很有可能导致某一个分片上的chunk数目变多,触发move。当我们增加或者删除shard机器时,也会导致上述move过程。

过程如下:balancer会给要迁移的shard发出一个move命令,该shard就会发送自己的数据给目标shard,目标shard就会同步数据,一旦同步完,源shard就会删除数据,最后config里面的配置信息也会被更新。

所以,MongoDB的shard集群为用户提供了一种水平扩展数据的功能,还提供了健壮性和可用性,健壮性是我们可以使用复制集技术来增加容错性,可用性是我们可以随意修改shard集群的节点,shard集群内部会做相应的balance,让每一个shard的空间都均摊所有的数据,其实这个还是很重要的,因为虽然我们可以部署很多机器,但是一旦最后的数据分布-不均匀,全部到了一台机器上,导致这个机器负载太高,装不下数据,应用肯定会挂,但是其余的机器却是空的,所以这里的balance功能非常重要。

实现思路大致是,首先保证每一个shard的chunk数目相当,但是光有这一点还不行,因为可能某一些chunk很大,所以还必须保证每一个chunk的大小均匀,这是通过split来实现,基于这两点,就可以保证每一个shard的大小相当。为什么shard的数据要以一个一个的chunk来组织?如果不以chunk来组织,那么如果要平衡数据,只能以shard为视角,把shard的一部分数据移到另一个shard,表面上好像也没什么,甚至更简单,但是实际上,比较难做到,因为当我们想把一个shard的一部分切下来时,该如何修改shard的片键范围呢?这是比较复杂的问题,切分的大小和片键的范围没有一一对应的关系,所以MongoDB就想出了让shard以chunk来做平衡的单位,每一个chunk会对应一个子区间,chunk本身有自己的大小管理,比如分裂,所以每一个shard里面的chunk基本可以认为大小一样,这样shard做平衡的依据就是chunk块的大小,如果数目差异较大,直接移动chunk即可,最后shard维护自己的片键范围也简单了很多。所以引入chunk的概念是一个很好的缓冲。

其次MongoDB的路由信息是基于范围的,这样在做balancing以后修改路由信息很简单,只需要修改shard的范围即可,如果采用除模等运算,还需要涉及一致性哈希等问题。基于范围和基于哈希函数或者除模运算的不同在于,范围的路由信息与插入的数据有关,不同的数据集可能会得到不同的范围集,是随着数据的插入而动态生成的。哈希函数这种有点静态的味道,只要给一个片值,不管有没有数据,不管其他数据如何,结果都是固定的。所以,基于范围可能在插入数据时需要维护路由信息,但是在修改时很容易,使用哈希法,虽然插入数据时不需要做事情,但是在动态修改拓扑结构以后,路由信息就会出问题。具体使用哪一个看场景看需求了,MongoDB要做balancing,所以肯定选择范围法更好。

MongoDB 多节点分片集群安装配置

一、基础环境

(1)操作系统:CentOS 7.3

(2)MongoDB:4.0.9 链接:https://pan.baidu.com/s/1dIfYNXyBRxDRn0uEZw6a6g 提取码:c4k3

二、节点角色规划

从图中可以看到有四个组件:mongos、config server、shard、replica set。

mongos,数据库集群请求的入口,所有的请求都通过mongos进行协调,不需要在应用程序添加一个路由选择器,mongos自己就是一个请求分发中心,它负责把对应的数据请求请求转发到对应的shard服务器上。在生产环境通常有多mongos作为请求的入口,防止其中一个挂掉所有的mongodb请求都没有办法操作。

config server,顾名思义为配置服务器,存储所有数据库元信息(路由、分片)的配置。mongos本身没有物理存储分片服务器和数据路由信息,只是缓存在内存里,配置服务器则实际存储这些数据。mongos第一次启动或者关掉重启就会从 config server 加载配置信息,以后如果配置服务器信息变化会通知到所有的 mongos 更新自己的状态,这样 mongos 就能继续准确路由。在生产环境通常有多个 config server 配置服务器,因为它存储了分片路由的元数据,防止数据丢失!

shard,分片(sharding)是指将数据库拆分,将其分散在不同的机器上的过程。将数据分散到不同的机器上,不需要功能强大的服务器就可以存储更多的数据和处理更大的负载。基本思想就是将集合切成小块,这些块分散到若干片里,每个片只负责总数据的一部分,最后通过一个均衡器来对各个分片进行均衡(数据迁移)。

replica set,中文翻译副本集,其实就是shard的备份,防止shard挂掉之后数据丢失。复制提供了数据的冗余备份,并在多个服务器上存储数据副本,提高了数据的可用性, 并可以保证数据的安全性。

仲裁者(Arbiter),是复制集中的一个MongoDB实例,它并不保存数据。仲裁节点使用最小的资源并且不要求硬件设备,不能将Arbiter部署在同一个数据集节点中,可以部署在其他应用服务器或者监视服务器中,也可部署在单独的虚拟机中。为了确保复制集中有奇数的投票成员(包括primary),需要添加仲裁节点做为投票,否则primary不能运行时不会自动切换primary。

简单了解之后,我们可以这样总结一下,应用请求mongos来操作mongodb的增删改查,配置服务器存储数据库元信息,并且和mongos做同步,数据最终存入在shard(分片)上,为了防止数据丢失同步在副本集中存储了一份,仲裁在数据存储到分片的时候决定存储到哪个节点。

那么我们将三台节点的角色进行如下规划:

10.211.55.19 10.211.55.20 10.211.55.21
mongos mongos mongos
config server config server config server
shard server1主节点 shard server1副节点 shard server1仲裁
shard server2仲裁 shard server2主节点 shard server2副节点
shard server3副节点 shard server3仲裁 shard server3主节点

三、端口分配

mongos:20000
config:21000
shard1:27001
shard2:27002
shard3:27003

四、安装所有节点

这里可以参考前面的安装方法

关闭防火墙

[root@Server01 bin]# systemctl stop firewalld.service
[root@Server01 bin]# systemctl disable firewalld.service

1、创建相关目录

分别在每台机器建立conf、mongos、config、shard1、shard2、shard3六个目录,因为mongos不存储数据,只需要建立日志文件目录即可

mkdir -p /usr/local/mongodb/bin/data/conf
mkdir -p /usr/local/mongodb/bin/data/mongos/log
mkdir -p /usr/local/mongodb/bin/data/config/{data,log}
mkdir -p /usr/local/mongodb/bin/data/shard1/{data,log}
mkdir -p /usr/local/mongodb/bin/data/shard2/{data,log}
mkdir -p /usr/local/mongodb/bin/data/shard3/{data,log}

已经创建完成,每台服务器都创建一下。这里不在叙述

[root@Server01 ~]# cd /usr/local/mongodb/bin/
[root@Server01 bin]# ls data/
conf  config  log  shard1  shard2  shard3

2、配置环境变量

vim /etc/profile
export MONGO=/usr/local/mongodb      #MongoDB的路径变量
export M2_HOME=/usr/local/maven
export JAVA_HOME=/usr/local/jdk
export PATH=$JAVA_HOME/bin:$PATH:$M2_HOME/bin:$MONGO/bin:$PATH     #添加路径
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JRE_HOME=$JAVA_HOME/jre
 source /etc/profile

五、配置config server服务器节点

1、配置,其他两台也是一样

[root@Server01 ~]# cd /usr/local/mongodb/bin/
[root@Server01 bin]# vim data/conf/config.conf 
pidfilepath = /usr/local/mongodb/bin/data/config/log/configsrv.pid
dbpath = /usr/local/mongodb/bin/data/config/data
logpath = /usr/local/mongodb/bin/data/config/log/congigsrv.log
logappend = true
bind_ip = 0.0.0.0
port = 21000
fork = true
configsvr = true
#副本集名称
replSet=configs
#设置最大连接数
maxConns=20000

2、启动服务

[root@Server01 bin]# ./mongod -f /usr/local/mongodb/bin/data/conf/config.conf 
about to fork child process, waiting until server is ready for connections.
forked process: 12101
child process started successfully, parent exiting

3、登录任意一台配置服务器,进行初始化副本集

[root@Server01 bin]# ./mongo --port 21000
MongoDB shell version v4.0.6
connecting to: mongodb://127.0.0.1:21000/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("0b714848-8cb3-48dc-8fd2-3dcb8e649cdb") }
MongoDB server version: 4.0.6
Server has startup warnings: 
2019-05-31T12:26:23.865+0800 I CONTROL  [initandlisten] 
2019-05-31T12:26:23.865+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-05-31T12:26:23.865+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-05-31T12:26:23.865+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-05-31T12:26:23.865+0800 I CONTROL  [initandlisten] 
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] 
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] 
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T12:26:23.866+0800 I CONTROL  [initandlisten] 
> config = {        #配置config变量,"_id" : "configs"应与配置文件中配置的replSet=configs一致,"members"中的"host"为三个节点的ip和port
...     _id : "configs",
...      members : [
...          {_id : 0, host : "10.211.55.19:21000" },
...          {_id : 1, host : "10.211.55.20:21000" },
...          {_id : 2, host : "10.211.55.21:21000" }
...      ]
...  }
{
        "_id" : "configs",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "10.211.55.19:21000"
                },
                {
                        "_id" : 1,
                        "host" : "10.211.55.20:21000"
                },
                {
                        "_id" : 2,
                        "host" : "10.211.55.21:21000"
                }
        ]
}
> rs.initiate(config)          # 初始化副本集
{
        "ok" : 1,
        "operationTime" : Timestamp(1559276899, 1),
        "$gleStats" : {
                "lastOpTime" : Timestamp(1559276899, 1),
                "electionId" : ObjectId("000000000000000000000000")
        },
        "lastCommittedOpTime" : Timestamp(0, 0),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559276899, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
configs:SECONDARY> 
configs:SECONDARY>

六、配置分片副本集(所有节点)

1、设置第一个分片副本集,其他节点也一样

[root@Server01 bin]# vim data/conf/shard1.conf

pidfilepath = /usr/local/mongodb/bin/data/shard1/log/shard1.pid
dbpath = /usr/local/mongodb/bin/data/shard1/data
logpath = /usr/local/mongodb/bin/data/shard1/log/shard1.log
logappend = true
bind_ip = 0.0.0.0
port = 27001
fork = true
#副本集名称
replSet=shard1
shardsvr = true
#设置最大连接数
maxConns=20000

启动服务

[root@Server01 bin]# ./mongod -f /usr/local/mongodb/bin/data/conf/shard1.conf 
about to fork child process, waiting until server is ready for connections.
forked process: 12142
child process started successfully, parent exiting

登录任意一台配置服务器mongo(非仲裁节点服务器),进行初始化副本集

[root@Server01 bin]# ./mongo --port 27001
MongoDB shell version v4.0.6
connecting to: mongodb://127.0.0.1:27001/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("06d5c72c-775f-42d6-860b-1a2167b8bb99") }
MongoDB server version: 4.0.6
Server has startup warnings: 
2019-05-31T11:30:25.159+0800 I CONTROL  [initandlisten] 
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] 
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] 
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] 
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T11:30:25.160+0800 I CONTROL  [initandlisten] 
> config = {
...     _id : "shard1",
...      members : [
...          {_id : 0, host : "10.211.55.19:27001" },
...          {_id : 1, host : "10.211.55.20:27001" },
...          {_id : 2, host : "10.211.55.21:27001" , arbiterOnly: true }      #"arbiterOnly":true 代表其为仲裁节点。
...      ]
...  }
{
        "_id" : "shard1",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "10.211.55.19:27001"
                },
                {
                        "_id" : 1,
                        "host" : "10.211.55.20:27001"
                },
                {
                        "_id" : 2,
                        "host" : "10.211.55.21:27001",
                        "arbiterOnly" : true
                }
        ]
}
> rs.initiate(config)
{
        "ok" : 1,
        "operationTime" : Timestamp(1559273841, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559273841, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
shard1:SECONDARY> 
shard1:SECONDARY> 
shard1:SECONDARY> 
shard1:SECONDARY> 
shard1:PRIMARY> 
shard1:PRIMARY>    #已经切换到主实例

2、设置第二个分片副本集(所有节点)

[root@Server01 bin]# vim data/conf/shard2.conf  
pidfilepath = /usr/local/mongodb/bin/data/shard2/log/shard2.pid
dbpath = /usr/local/mongodb/bin/data/shard2/data
logpath = /usr/local/mongodb/bin/data/shard2/log/shard2.log
logappend = true
bind_ip = 0.0.0.0
port = 27002
fork = true
#副本集名称
replSet=shard2
shardsvr = true
#设置最大连接数
maxConns=20000

启动服务

[root@Server01 bin]# ./mongod -f /usr/local/mongodb/bin/data/conf/shard2.conf  
about to fork child process, waiting until server is ready for connections.
forked process: 12353
child process started successfully, parent exiting

登陆任意一台服务器的mongo(非仲裁节点服务器),初始化副本集

[root@Server02 bin]# ./mongo --port 27002
MongoDB shell version v4.0.6
connecting to: mongodb://127.0.0.1:27002/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("c4542015-fd47-4348-abf1-a9efb3196bbb") }
MongoDB server version: 4.0.6
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] 
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] 
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] 
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] 
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T12:02:23.019+0800 I CONTROL  [initandlisten] 
> config = {
...     _id : "shard2",
...      members : [
...          {_id : 0, host : "10.211.55.19:27002"  , arbiterOnly: true },
...          {_id : 1, host : "10.211.55.20:27002" },
...          {_id : 2, host : "10.211.55.21:27002" }
...      ]
...  }
{
        "_id" : "shard2",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "10.211.55.19:27002",
                        "arbiterOnly" : true
                },
                {
                        "_id" : 1,
                        "host" : "10.211.55.20:27002"
                },
                {
                        "_id" : 2,
                        "host" : "10.211.55.21:27002"
                }
        ]
}
> rs.initiate(config)
{
        "ok" : 1,
        "operationTime" : Timestamp(1559275417, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559275417, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
shard2:SECONDARY> 
shard2:SECONDARY> 
shard2:SECONDARY>

3、设置第三个分片副本集

[root@Server01 bin]# vim data/conf/shard3.conf
pidfilepath = /usr/local/mongodb/bin/data/shard3/log/shard3.pid
dbpath = /usr/local/mongodb/bin/data/shard3/data
logpath = /usr/local/mongodb/bin/data/shard3/log/shard3.log
logappend = true
bind_ip = 0.0.0.0
port = 27003
fork = true
#副本集名称
replSet=shard3
shardsvr = true
#设置最大连接数
maxConns=20000

启动服务

[root@Server01 bin]# ./mongod -f /usr/local/mongodb/bin/data/conf/shard3.conf                                 
about to fork child process, waiting until server is ready for connections.
forked process: 12419
child process started successfully, parent exiting

登陆任意一台服务器的mongo(非仲裁节点服务器),初始化副本集

[root@Server01 bin]# ./mongo --port 27003
MongoDB shell version v4.0.6
connecting to: mongodb://127.0.0.1:27003/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("c01abf41-16ec-4109-8d8b-fc848b5c509d") }
MongoDB server version: 4.0.6
Server has startup warnings: 
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] 
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] 
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] 
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] 
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-05-31T11:48:46.690+0800 I CONTROL  [initandlisten] 
> config = {
...     _id : "shard3",
...      members : [
...          {_id : 0, host : "10.211.55.19:27003" },
...          {_id : 1, host : "10.211.55.20:27003" , arbiterOnly: true},
...          {_id : 2, host : "10.211.55.21:27003" }
...      ]
...  }
{
        "_id" : "shard3",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "10.211.55.19:27003"
                },
                {
                        "_id" : 1,
                        "host" : "10.211.55.20:27003",
                        "arbiterOnly" : true
                },
                {
                        "_id" : 2,
                        "host" : "10.211.55.21:27003"
                }
        ]
}
> rs.initiate(config)
{
        "ok" : 1,
        "operationTime" : Timestamp(1559274755, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559274755, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
shard3:SECONDARY> 
shard3:SECONDARY> 
shard3:SECONDARY>

4、配置mongos节点(所有节点)

[root@Server01 bin]# vim data/conf/mongos.conf
pidfilepath = /usr/local/mongodb/bin/data/mongos/log/mongos.pid
logpath = /usr/local/mongodb/bin/data/mongos/log/mongos.log
logappend = true
bind_ip = 0.0.0.0
port = 20000
fork = true
#监听的配置服务器,只能有1个或者3个 configs为配置服务器的副本集名字
configdb = configs/10.211.55.19:21000,10.211.55.20:21000,10.211.55.21:21000
#设置最大连接数
maxConns=20000

启动服务

[root@Server01 bin]# ./mongos -f /usr/local/mongodb/bin/data/conf/mongos.conf 
about to fork child process, waiting until server is ready for connections.
forked process: 12754
child process started successfully, parent exiting

八、串联节点

已经创建了mongodb配置服务器、路由服务器,各个分片服务器,应用服务器连接到mongos服务器,并不能使用分片机制,所以需要把节点串联起来,才能让分片生效。随机登录一台mongos

sh.addShard("shard1/10.211.55.19:27001,10.211.55.20:27001,10.211.55.21:27001")
sh.addShard("shard2/10.211.55.19:27002,10.211.55.20:27002,10.211.55.21:27002")
sh.addShard("shard3/10.211.55.19:27003,10.211.55.20:27003,10.211.55.21:27003")

在mongos上配置路由

[root@Server01 bin]# ./mongo --port 20000
MongoDB shell version v4.0.6
connecting to: mongodb://127.0.0.1:20000/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("be2351a6-7ba8-4f5f-ac84-dfe6ce0c33d6") }
MongoDB server version: 4.0.6
Server has startup warnings: 
2019-05-31T12:31:11.556+0800 I CONTROL  [main] 
2019-05-31T12:31:11.556+0800 I CONTROL  [main] ** WARNING: Access control is not enabled for the database.
2019-05-31T12:31:11.556+0800 I CONTROL  [main] **          Read and write access to data and configuration is unrestricted.
2019-05-31T12:31:11.556+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2019-05-31T12:31:11.556+0800 I CONTROL  [main] 
mongos> sh.addShard("shard1/10.211.55.19:27001,10.211.55.20:27001,10.211.55.21:27001")    #串联路由服务器与分片副本集
{
        "shardAdded" : "shard1",
        "ok" : 1,
        "operationTime" : Timestamp(1559277297, 6),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559277297, 6),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> sh.addShard("shard2/10.211.55.19:27002,10.211.55.20:27002,10.211.55.21:27002")     #串联路由服务器与分片副本集
{
        "shardAdded" : "shard2",
        "ok" : 1,
        "operationTime" : Timestamp(1559278348, 6),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559278348, 6),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> sh.addShard("shard3/10.211.55.19:27003,10.211.55.20:27003,10.211.55.21:27003")      #串联路由服务器与分片副本集
{
        "shardAdded" : "shard3",
        "ok" : 1,
        "operationTime" : Timestamp(1559278348, 11),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559278348, 11),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> sh.status()    #查看集群状态
--- Sharding Status --- 
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5cf0ad6f6006ca0b06ae1326")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/10.211.55.19:27001,10.211.55.20:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/10.211.55.20:27002,10.211.55.21:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/10.211.55.19:27003,10.211.55.21:27003",  "state" : 1 }
  active mongoses:
        "4.0.6" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1  1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 

mongos>

至此基础集群已经搭建完成,集群分为hash策略和range策略,上面已经提到过了,下面根据不同的策略做一下测试

九、开启分片

目前配置服务、路由服务、分片服务、副本集服务都已经串联起来了,但我们的目的是希望插入数据,数据能够自动分片。登录任意一台mongos,准备让指定的数据库、指定的集合分片生效。

mongos> use admin
switched to db admin
mongos> 
mongos> db.runCommand( { enablesharding :"test_db"}); #指定test_db分片生效
{
        "ok" : 1,
        "operationTime" : Timestamp(1559278939, 6),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559278939, 6),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> 
mongos> db.runCommand( { shardcollection : "test_db.table1",key : {id: "hashed"} } )     #指定数据库里需要分片的集合和片键
{
        "collectionsharded" : "test_db.table1",
        "collectionUUID" : UUID("9c4d0233-0032-4eb5-ba36-a24310d6e45b"),
        "ok" : 1,
        "operationTime" : Timestamp(1559278958, 38),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559278958, 38),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos>

test_db的table1表需要分片,根据id自动分片到shard1,shard2,shard3上面去。要这样设置是因为不是所有mongodb的数据库和表都需要分片!

mongos> use test_db
switched to db test_db
mongos> for (var i = 1; i <= 10000; i++)db.table1.save({id:i,"test1":"testval1"});
WriteResult({ "nInserted" : 1 })
mongos>

看看插入的数据分片情况

mongos> db.table1.stats();
{
        "sharded" : true,
        "capped" : false,
        "wiredTiger" : {
                "metadata" : {
                        "formatVersion" : 1
                },
                "creationString" : "access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),assert=(commit_timestamp=none,read_timestamp=none),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=,name=),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=false),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_custom=(prefix=,start_generation=0,suffix=),merge_max=15,merge_min=0),memory_page_image_max=0,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,type=file,value_format=u",
                "type" : "file",
                "uri" : "statistics:table:collection-24--2366925424540304775",
                "LSM" : {
                        "bloom filter false positives" : 0,
                        "bloom filter hits" : 0,
                        "bloom filter misses" : 0,
                        "bloom filter pages evicted from cache" : 0,
                        "bloom filter pages read into cache" : 0,
                        "bloom filters in the LSM tree" : 0,
                        "chunks in the LSM tree" : 0,
                        "highest merge generation in the LSM tree" : 0,
                        "queries that could have benefited from a Bloom filter that did not exist" : 0,
                        "sleep for LSM checkpoint throttle" : 0,
                        "sleep for LSM merge throttle" : 0,
                        "total size of bloom filters" : 0
                },
                "block-manager" : {
                        "allocations requiring file extension" : 9,
                        "blocks allocated" : 9,
                        "blocks freed" : 0,
                        "checkpoint size" : 57344,
                        "file allocation unit size" : 4096,
                        "file bytes available for reuse" : 0,
                        "file magic number" : 120897,
                        "file major version number" : 1,
                        "file size in bytes" : 69632,
                        "minor version number" : 0
                },
                "btree" : {
                        "btree checkpoint generation" : 61,
                        "column-store fixed-size leaf pages" : 0,
                        "column-store internal pages" : 0,
                        "column-store variable-size RLE encoded values" : 0,
                        "column-store variable-size deleted values" : 0,
                        "column-store variable-size leaf pages" : 0,
                        "fixed-record size" : 0,
                        "maximum internal page key size" : 368,
                        "maximum internal page size" : 4096,
                        "maximum leaf page key size" : 2867,
                        "maximum leaf page size" : 32768,
                        "maximum leaf page value size" : 67108864,
                        "maximum tree depth" : 3,
                        "number of key/value pairs" : 0,
                        "overflow pages" : 0,
                        "pages rewritten by compaction" : 0,
                        "row-store internal pages" : 0,
                        "row-store leaf pages" : 0
                },
                "cache" : {
                        "bytes currently in the cache" : 443294,
                        "bytes dirty in the cache cumulative" : 839,
                        "bytes read into cache" : 0,
                        "bytes written from cache" : 190812,
                        "checkpoint blocked page eviction" : 0,
                        "data source pages selected for eviction unable to be evicted" : 0,
                        "eviction walk passes of a file" : 0,
                        "eviction walk target pages histogram - 0-9" : 0,
                        "eviction walk target pages histogram - 10-31" : 0,
                        "eviction walk target pages histogram - 128 and higher" : 0,
                        "eviction walk target pages histogram - 32-63" : 0,
                        "eviction walk target pages histogram - 64-128" : 0,
                        "eviction walks abandoned" : 0,
                        "eviction walks gave up because they restarted their walk twice" : 0,
                        "eviction walks gave up because they saw too many pages and found no candidates" : 0,
                        "eviction walks gave up because they saw too many pages and found too few candidates" : 0,
                        "eviction walks reached end of tree" : 0,
                        "eviction walks started from root of tree" : 0,
                        "eviction walks started from saved location in tree" : 0,
                        "hazard pointer blocked page eviction" : 0,
                        "in-memory page passed criteria to be split" : 0,
                        "in-memory page splits" : 0,
                        "internal pages evicted" : 0,
                        "internal pages split during eviction" : 0,
                        "leaf pages split during eviction" : 0,
                        "modified pages evicted" : 0,
                        "overflow pages read into cache" : 0,
                        "page split during eviction deepened the tree" : 0,
                        "page written requiring cache overflow records" : 0,
                        "pages read into cache" : 0,
                        "pages read into cache after truncate" : 1,
                        "pages read into cache after truncate in prepare state" : 0,
                        "pages read into cache requiring cache overflow entries" : 0,
                        "pages requested from the cache" : 3284,
                        "pages seen by eviction walk" : 0,
                        "pages written from cache" : 8,
                        "pages written requiring in-memory restoration" : 0,
                        "tracked dirty bytes in the cache" : 0,
                        "unmodified pages evicted" : 0
                },
                "cache_walk" : {
                        "Average difference between current eviction generation when the page was last considered" : 0,
                        "Average on-disk page image size seen" : 0,
                        "Average time in cache for pages that have been visited by the eviction server" : 0,
                        "Average time in cache for pages that have not been visited by the eviction server" : 0,
                        "Clean pages currently in cache" : 0,
                        "Current eviction generation" : 0,
                        "Dirty pages currently in cache" : 0,
                        "Entries in the root page" : 0,
                        "Internal pages currently in cache" : 0,
                        "Leaf pages currently in cache" : 0,
                        "Maximum difference between current eviction generation when the page was last considered" : 0,
                        "Maximum page size seen" : 0,
                        "Minimum on-disk page image size seen" : 0,
                        "Number of pages never visited by eviction server" : 0,
                        "On-disk page image sizes smaller than a single allocation unit" : 0,
                        "Pages created in memory and never written" : 0,
                        "Pages currently queued for eviction" : 0,
                        "Pages that could not be queued for eviction" : 0,
                        "Refs skipped during cache traversal" : 0,
                        "Size of the root page" : 0,
                        "Total number of pages currently in cache" : 0
                },
                "compression" : {
                        "compressed pages read" : 0,
                        "compressed pages written" : 7,
                        "page written failed to compress" : 0,
                        "page written was too small to compress" : 1
                },
                "cursor" : {
                        "bulk-loaded cursor-insert calls" : 0,
                        "close calls that result in cache" : 0,
                        "create calls" : 2,
                        "cursor operation restarted" : 0,
                        "cursor-insert key and value bytes inserted" : 183841,
                        "cursor-remove key bytes removed" : 0,
                        "cursor-update value bytes updated" : 0,
                        "cursors reused from cache" : 3283,
                        "insert calls" : 3284,
                        "modify calls" : 0,
                        "next calls" : 2,
                        "open cursor count" : 0,
                        "prev calls" : 1,
                        "remove calls" : 0,
                        "reserve calls" : 0,
                        "reset calls" : 6572,
                        "search calls" : 0,
                        "search near calls" : 0,
                        "truncate calls" : 0,
                        "update calls" : 0
                },
                "reconciliation" : {
                        "dictionary matches" : 0,
                        "fast-path pages deleted" : 0,
                        "internal page key bytes discarded using suffix compression" : 13,
                        "internal page multi-block writes" : 0,
                        "internal-page overflow keys" : 0,
                        "leaf page key bytes discarded using prefix compression" : 0,
                        "leaf page multi-block writes" : 1,
                        "leaf-page overflow keys" : 0,
                        "maximum blocks required for a page" : 1,
                        "overflow values written" : 0,
                        "page checksum matches" : 0,
                        "page reconciliation calls" : 2,
                        "page reconciliation calls for eviction" : 0,
                        "pages deleted" : 0
                },
                "session" : {
                        "object compaction" : 0
                },
                "transaction" : {
                        "update conflicts" : 0
                }
        },
        "ns" : "test_db.table1",
        "count" : 10000,      ##总共10000条
        "size" : 540000,
        "storageSize" : 208896,
        "totalIndexSize" : 339968,
        "indexSizes" : {
                "_id_" : 122880,
                "id_hashed" : 217088
        },
        "avgObjSize" : 54,
        "maxSize" : NumberLong(0),
        "nindexes" : 2,
        "nchunks" : 6,
        "shards" : {
                "shard3" : {
                        "ns" : "test_db.table1",
                        "size" : 177336,
                        "count" : 3284,       #节点3上3284条
                        "avgObjSize" : 54,
                        "storageSize" : 69632,
                        "capped" : false,
                        "wiredTiger" : {
                                "metadata" : {
                                        "formatVersion" : 1
                                },
                                "creationString" : "access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),assert=(commit_timestamp=none,read_timestamp=none),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=,name=),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=false),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_custom=(prefix=,start_generation=0,suffix=),merge_max=15,merge_min=0),memory_page_image_max=0,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,type=file,value_format=u",
                                "type" : "file",
                                "uri" : "statistics:table:collection-24--2366925424540304775",
                                "LSM" : {
                                        "bloom filter false positives" : 0,
                                        "bloom filter hits" : 0,
                                        "bloom filter misses" : 0,
                                        "bloom filter pages evicted from cache" : 0,
                                        "bloom filter pages read into cache" : 0,
                                        "bloom filters in the LSM tree" : 0,
                                        "chunks in the LSM tree" : 0,
                                        "highest merge generation in the LSM tree" : 0,
                                        "queries that could have benefited from a Bloom filter that did not exist" : 0,
                                        "sleep for LSM checkpoint throttle" : 0,
                                        "sleep for LSM merge throttle" : 0,
                                        "total size of bloom filters" : 0
                                },
                                "block-manager" : {
                                        "allocations requiring file extension" : 9,
                                        "blocks allocated" : 9,
                                        "blocks freed" : 0,
                                        "checkpoint size" : 57344,
                                        "file allocation unit size" : 4096,
                                        "file bytes available for reuse" : 0,
                                        "file magic number" : 120897,
                                        "file major version number" : 1,
                                        "file size in bytes" : 69632,
                                        "minor version number" : 0
                                },
                                "btree" : {
                                        "btree checkpoint generation" : 61,
                                        "column-store fixed-size leaf pages" : 0,
                                        "column-store internal pages" : 0,
                                        "column-store variable-size RLE encoded values" : 0,
                                        "column-store variable-size deleted values" : 0,
                                        "column-store variable-size leaf pages" : 0,
                                        "fixed-record size" : 0,
                                        "maximum internal page key size" : 368,
                                        "maximum internal page size" : 4096,
                                        "maximum leaf page key size" : 2867,
                                        "maximum leaf page size" : 32768,
                                        "maximum leaf page value size" : 67108864,
                                        "maximum tree depth" : 3,
                                        "number of key/value pairs" : 0,
                                        "overflow pages" : 0,
                                        "pages rewritten by compaction" : 0,
                                        "row-store internal pages" : 0,
                                        "row-store leaf pages" : 0
                                },
                                "cache" : {
                                        "bytes currently in the cache" : 443294,
                                        "bytes dirty in the cache cumulative" : 839,
                                        "bytes read into cache" : 0,
                                        "bytes written from cache" : 190812,
                                        "checkpoint blocked page eviction" : 0,
                                        "data source pages selected for eviction unable to be evicted" : 0,
                                        "eviction walk passes of a file" : 0,
                                        "eviction walk target pages histogram - 0-9" : 0,
                                        "eviction walk target pages histogram - 10-31" : 0,
                                        "eviction walk target pages histogram - 128 and higher" : 0,
                                        "eviction walk target pages histogram - 32-63" : 0,
                                        "eviction walk target pages histogram - 64-128" : 0,
                                        "eviction walks abandoned" : 0,
                                        "eviction walks gave up because they restarted their walk twice" : 0,
                                        "eviction walks gave up because they saw too many pages and found no candidates" : 0,
                                        "eviction walks gave up because they saw too many pages and found too few candidates" : 0,
                                        "eviction walks reached end of tree" : 0,
                                        "eviction walks started from root of tree" : 0,
                                        "eviction walks started from saved location in tree" : 0,
                                        "hazard pointer blocked page eviction" : 0,
                                        "in-memory page passed criteria to be split" : 0,
                                        "in-memory page splits" : 0,
                                        "internal pages evicted" : 0,
                                        "internal pages split during eviction" : 0,
                                        "leaf pages split during eviction" : 0,
                                        "modified pages evicted" : 0,
                                        "overflow pages read into cache" : 0,
                                        "page split during eviction deepened the tree" : 0,
                                        "page written requiring cache overflow records" : 0,
                                        "pages read into cache" : 0,
                                        "pages read into cache after truncate" : 1,
                                        "pages read into cache after truncate in prepare state" : 0,
                                        "pages read into cache requiring cache overflow entries" : 0,
                                        "pages requested from the cache" : 3284,
                                        "pages seen by eviction walk" : 0,
                                        "pages written from cache" : 8,
                                        "pages written requiring in-memory restoration" : 0,
                                        "tracked dirty bytes in the cache" : 0,
                                        "unmodified pages evicted" : 0
                                },
                                "cache_walk" : {
                                        "Average difference between current eviction generation when the page was last considered" : 0,
                                        "Average on-disk page image size seen" : 0,
                                        "Average time in cache for pages that have been visited by the eviction server" : 0,
                                        "Average time in cache for pages that have not been visited by the eviction server" : 0,
                                        "Clean pages currently in cache" : 0,
                                        "Current eviction generation" : 0,
                                        "Dirty pages currently in cache" : 0,
                                        "Entries in the root page" : 0,
                                        "Internal pages currently in cache" : 0,
                                        "Leaf pages currently in cache" : 0,
                                        "Maximum difference between current eviction generation when the page was last considered" : 0,
                                        "Maximum page size seen" : 0,
                                        "Minimum on-disk page image size seen" : 0,
                                        "Number of pages never visited by eviction server" : 0,
                                        "On-disk page image sizes smaller than a single allocation unit" : 0,
                                        "Pages created in memory and never written" : 0,
                                        "Pages currently queued for eviction" : 0,
                                        "Pages that could not be queued for eviction" : 0,
                                        "Refs skipped during cache traversal" : 0,
                                        "Size of the root page" : 0,
                                        "Total number of pages currently in cache" : 0
                                },
                                "compression" : {
                                        "compressed pages read" : 0,
                                        "compressed pages written" : 7,
                                        "page written failed to compress" : 0,
                                        "page written was too small to compress" : 1
                                },
                                "cursor" : {
                                        "bulk-loaded cursor-insert calls" : 0,
                                        "close calls that result in cache" : 0,
                                        "create calls" : 2,
                                        "cursor operation restarted" : 0,
                                        "cursor-insert key and value bytes inserted" : 183841,
                                        "cursor-remove key bytes removed" : 0,
                                        "cursor-update value bytes updated" : 0,
                                        "cursors reused from cache" : 3283,
                                        "insert calls" : 3284,
                                        "modify calls" : 0,
                                        "next calls" : 2,
                                        "open cursor count" : 0,
                                        "prev calls" : 1,
                                        "remove calls" : 0,
                                        "reserve calls" : 0,
                                        "reset calls" : 6572,
                                        "search calls" : 0,
                                        "search near calls" : 0,
                                        "truncate calls" : 0,
                                        "update calls" : 0
                                },
                                "reconciliation" : {
                                        "dictionary matches" : 0,
                                        "fast-path pages deleted" : 0,
                                        "internal page key bytes discarded using suffix compression" : 13,
                                        "internal page multi-block writes" : 0,
                                        "internal-page overflow keys" : 0,
                                        "leaf page key bytes discarded using prefix compression" : 0,
                                        "leaf page multi-block writes" : 1,
                                        "leaf-page overflow keys" : 0,
                                        "maximum blocks required for a page" : 1,
                                        "overflow values written" : 0,
                                        "page checksum matches" : 0,
                                        "page reconciliation calls" : 2,
                                        "page reconciliation calls for eviction" : 0,
                                        "pages deleted" : 0
                                },
                                "session" : {
                                        "object compaction" : 0
                                },
                                "transaction" : {
                                        "update conflicts" : 0
                                }
                        },
                        "nindexes" : 2,
                        "totalIndexSize" : 110592,
                        "indexSizes" : {
                                "_id_" : 40960,
                                "id_hashed" : 69632
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1559279099, 3),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1559279079, 155),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1559279099, 3),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1559279109, 3),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1559279109, 3),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard1" : {
                        "ns" : "test_db.table1",
                        "size" : 180900,
                        "count" : 3350,       #节点1上3350条
                        "avgObjSize" : 54,
                        "storageSize" : 69632,
                        "capped" : false,
                        "wiredTiger" : {
                                "metadata" : {
                                        "formatVersion" : 1
                                },
                                "creationString" : "access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),assert=(commit_timestamp=none,read_timestamp=none),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=,name=),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=false),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_custom=(prefix=,start_generation=0,suffix=),merge_max=15,merge_min=0),memory_page_image_max=0,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,type=file,value_format=u",
                                "type" : "file",
                                "uri" : "statistics:table:collection-27-6567758907732878262",
                                "LSM" : {
                                        "bloom filter false positives" : 0,
                                        "bloom filter hits" : 0,
                                        "bloom filter misses" : 0,
                                        "bloom filter pages evicted from cache" : 0,
                                        "bloom filter pages read into cache" : 0,
                                        "bloom filters in the LSM tree" : 0,
                                        "chunks in the LSM tree" : 0,
                                        "highest merge generation in the LSM tree" : 0,
                                        "queries that could have benefited from a Bloom filter that did not exist" : 0,
                                        "sleep for LSM checkpoint throttle" : 0,
                                        "sleep for LSM merge throttle" : 0,
                                        "total size of bloom filters" : 0
                                },
                                "block-manager" : {
                                        "allocations requiring file extension" : 9,
                                        "blocks allocated" : 9,
                                        "blocks freed" : 0,
                                        "checkpoint size" : 57344,
                                        "file allocation unit size" : 4096,
                                        "file bytes available for reuse" : 0,
                                        "file magic number" : 120897,
                                        "file major version number" : 1,
                                        "file size in bytes" : 69632,
                                        "minor version number" : 0
                                },
                                "btree" : {
                                        "btree checkpoint generation" : 79,
                                        "column-store fixed-size leaf pages" : 0,
                                        "column-store internal pages" : 0,
                                        "column-store variable-size RLE encoded values" : 0,
                                        "column-store variable-size deleted values" : 0,
                                        "column-store variable-size leaf pages" : 0,
                                        "fixed-record size" : 0,
                                        "maximum internal page key size" : 368,
                                        "maximum internal page size" : 4096,
                                        "maximum leaf page key size" : 2867,
                                        "maximum leaf page size" : 32768,
                                        "maximum leaf page value size" : 67108864,
                                        "maximum tree depth" : 3,
                                        "number of key/value pairs" : 0,
                                        "overflow pages" : 0,
                                        "pages rewritten by compaction" : 0,
                                        "row-store internal pages" : 0,
                                        "row-store leaf pages" : 0
                                },
                                "cache" : {
                                        "bytes currently in the cache" : 451740,
                                        "bytes dirty in the cache cumulative" : 839,
                                        "bytes read into cache" : 0,
                                        "bytes written from cache" : 194640,
                                        "checkpoint blocked page eviction" : 0,
                                        "data source pages selected for eviction unable to be evicted" : 0,
                                        "eviction walk passes of a file" : 0,
                                        "eviction walk target pages histogram - 0-9" : 0,
                                        "eviction walk target pages histogram - 10-31" : 0,
                                        "eviction walk target pages histogram - 128 and higher" : 0,
                                        "eviction walk target pages histogram - 32-63" : 0,
                                        "eviction walk target pages histogram - 64-128" : 0,
                                        "eviction walks abandoned" : 0,
                                        "eviction walks gave up because they restarted their walk twice" : 0,
                                        "eviction walks gave up because they saw too many pages and found no candidates" : 0,
                                        "eviction walks gave up because they saw too many pages and found too few candidates" : 0,
                                        "eviction walks reached end of tree" : 0,
                                        "eviction walks started from root of tree" : 0,
                                        "eviction walks started from saved location in tree" : 0,
                                        "hazard pointer blocked page eviction" : 0,
                                        "in-memory page passed criteria to be split" : 0,
                                        "in-memory page splits" : 0,
                                        "internal pages evicted" : 0,
                                        "internal pages split during eviction" : 0,
                                        "leaf pages split during eviction" : 0,
                                        "modified pages evicted" : 0,
                                        "overflow pages read into cache" : 0,
                                        "page split during eviction deepened the tree" : 0,
                                        "page written requiring cache overflow records" : 0,
                                        "pages read into cache" : 0,
                                        "pages read into cache after truncate" : 1,
                                        "pages read into cache after truncate in prepare state" : 0,
                                        "pages read into cache requiring cache overflow entries" : 0,
                                        "pages requested from the cache" : 3350,
                                        "pages seen by eviction walk" : 0,
                                        "pages written from cache" : 8,
                                        "pages written requiring in-memory restoration" : 0,
                                        "tracked dirty bytes in the cache" : 0,
                                        "unmodified pages evicted" : 0
                                },
                                "cache_walk" : {
                                        "Average difference between current eviction generation when the page was last considered" : 0,
                                        "Average on-disk page image size seen" : 0,
                                        "Average time in cache for pages that have been visited by the eviction server" : 0,
                                        "Average time in cache for pages that have not been visited by the eviction server" : 0,
                                        "Clean pages currently in cache" : 0,
                                        "Current eviction generation" : 0,
                                        "Dirty pages currently in cache" : 0,
                                        "Entries in the root page" : 0,
                                        "Internal pages currently in cache" : 0,
                                        "Leaf pages currently in cache" : 0,
                                        "Maximum difference between current eviction generation when the page was last considered" : 0,
                                        "Maximum page size seen" : 0,
                                        "Minimum on-disk page image size seen" : 0,
                                        "Number of pages never visited by eviction server" : 0,
                                        "On-disk page image sizes smaller than a single allocation unit" : 0,
                                        "Pages created in memory and never written" : 0,
                                        "Pages currently queued for eviction" : 0,
                                        "Pages that could not be queued for eviction" : 0,
                                        "Refs skipped during cache traversal" : 0,
                                        "Size of the root page" : 0,
                                        "Total number of pages currently in cache" : 0
                                },
                                "compression" : {
                                        "compressed pages read" : 0,
                                        "compressed pages written" : 7,
                                        "page written failed to compress" : 0,
                                        "page written was too small to compress" : 1
                                },
                                "cursor" : {
                                        "bulk-loaded cursor-insert calls" : 0,
                                        "close calls that result in cache" : 0,
                                        "create calls" : 3,
                                        "cursor operation restarted" : 0,
                                        "cursor-insert key and value bytes inserted" : 187537,
                                        "cursor-remove key bytes removed" : 0,
                                        "cursor-update value bytes updated" : 0,
                                        "cursors reused from cache" : 3348,
                                        "insert calls" : 3350,
                                        "modify calls" : 0,
                                        "next calls" : 0,
                                        "open cursor count" : 0,
                                        "prev calls" : 1,
                                        "remove calls" : 0,
                                        "reserve calls" : 0,
                                        "reset calls" : 6702,
                                        "search calls" : 0,
                                        "search near calls" : 0,
                                        "truncate calls" : 0,
                                        "update calls" : 0
                                },
                                "reconciliation" : {
                                        "dictionary matches" : 0,
                                        "fast-path pages deleted" : 0,
                                        "internal page key bytes discarded using suffix compression" : 13,
                                        "internal page multi-block writes" : 0,
                                        "internal-page overflow keys" : 0,
                                        "leaf page key bytes discarded using prefix compression" : 0,
                                        "leaf page multi-block writes" : 1,
                                        "leaf-page overflow keys" : 0,
                                        "maximum blocks required for a page" : 1,
                                        "overflow values written" : 0,
                                        "page checksum matches" : 0,
                                        "page reconciliation calls" : 2,
                                        "page reconciliation calls for eviction" : 0,
                                        "pages deleted" : 0
                                },
                                "session" : {
                                        "object compaction" : 0
                                },
                                "transaction" : {
                                        "update conflicts" : 0
                                }
                        },
                        "nindexes" : 2,
                        "totalIndexSize" : 114688,
                        "indexSizes" : {
                                "_id_" : 40960,
                                "id_hashed" : 73728
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1559279099, 2),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1559279079, 156),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1559279099, 2),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1559279109, 3),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1559279109, 3),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                },
                "shard2" : {
                        "ns" : "test_db.table1",
                        "size" : 181764,
                        "count" : 3366,         #节点2上3366条
                        "avgObjSize" : 54,
                        "storageSize" : 69632,
                        "capped" : false,
                        "wiredTiger" : {
                                "metadata" : {
                                        "formatVersion" : 1
                                },
                                "creationString" : "access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),assert=(commit_timestamp=none,read_timestamp=none),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=,name=),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=false),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_custom=(prefix=,start_generation=0,suffix=),merge_max=15,merge_min=0),memory_page_image_max=0,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,type=file,value_format=u",
                                "type" : "file",
                                "uri" : "statistics:table:collection-24-6726620435359763885",
                                "LSM" : {
                                        "bloom filter false positives" : 0,
                                        "bloom filter hits" : 0,
                                        "bloom filter misses" : 0,
                                        "bloom filter pages evicted from cache" : 0,
                                        "bloom filter pages read into cache" : 0,
                                        "bloom filters in the LSM tree" : 0,
                                        "chunks in the LSM tree" : 0,
                                        "highest merge generation in the LSM tree" : 0,
                                        "queries that could have benefited from a Bloom filter that did not exist" : 0,
                                        "sleep for LSM checkpoint throttle" : 0,
                                        "sleep for LSM merge throttle" : 0,
                                        "total size of bloom filters" : 0
                                },
                                "block-manager" : {
                                        "allocations requiring file extension" : 9,
                                        "blocks allocated" : 9,
                                        "blocks freed" : 0,
                                        "checkpoint size" : 57344,
                                        "file allocation unit size" : 4096,
                                        "file bytes available for reuse" : 0,
                                        "file magic number" : 120897,
                                        "file major version number" : 1,
                                        "file size in bytes" : 69632,
                                        "minor version number" : 0
                                },
                                "btree" : {
                                        "btree checkpoint generation" : 65,
                                        "column-store fixed-size leaf pages" : 0,
                                        "column-store internal pages" : 0,
                                        "column-store variable-size RLE encoded values" : 0,
                                        "column-store variable-size deleted values" : 0,
                                        "column-store variable-size leaf pages" : 0,
                                        "fixed-record size" : 0,
                                        "maximum internal page key size" : 368,
                                        "maximum internal page size" : 4096,
                                        "maximum leaf page key size" : 2867,
                                        "maximum leaf page size" : 32768,
                                        "maximum leaf page value size" : 67108864,
                                        "maximum tree depth" : 3,
                                        "number of key/value pairs" : 0,
                                        "overflow pages" : 0,
                                        "pages rewritten by compaction" : 0,
                                        "row-store internal pages" : 0,
                                        "row-store leaf pages" : 0
                                },
                                "cache" : {
                                        "bytes currently in the cache" : 454530,
                                        "bytes dirty in the cache cumulative" : 839,
                                        "bytes read into cache" : 0,
                                        "bytes written from cache" : 195568,
                                        "checkpoint blocked page eviction" : 0,
                                        "data source pages selected for eviction unable to be evicted" : 0,
                                        "eviction walk passes of a file" : 0,
                                        "eviction walk target pages histogram - 0-9" : 0,
                                        "eviction walk target pages histogram - 10-31" : 0,
                                        "eviction walk target pages histogram - 128 and higher" : 0,
                                        "eviction walk target pages histogram - 32-63" : 0,
                                        "eviction walk target pages histogram - 64-128" : 0,
                                        "eviction walks abandoned" : 0,
                                        "eviction walks gave up because they restarted their walk twice" : 0,
                                        "eviction walks gave up because they saw too many pages and found no candidates" : 0,
                                        "eviction walks gave up because they saw too many pages and found too few candidates" : 0,
                                        "eviction walks reached end of tree" : 0,
                                        "eviction walks started from root of tree" : 0,
                                        "eviction walks started from saved location in tree" : 0,
                                        "hazard pointer blocked page eviction" : 0,
                                        "in-memory page passed criteria to be split" : 0,
                                        "in-memory page splits" : 0,
                                        "internal pages evicted" : 0,
                                        "internal pages split during eviction" : 0,
                                        "leaf pages split during eviction" : 0,
                                        "modified pages evicted" : 0,
                                        "overflow pages read into cache" : 0,
                                        "page split during eviction deepened the tree" : 0,
                                        "page written requiring cache overflow records" : 0,
                                        "pages read into cache" : 0,
                                        "pages read into cache after truncate" : 1,
                                        "pages read into cache after truncate in prepare state" : 0,
                                        "pages read into cache requiring cache overflow entries" : 0,
                                        "pages requested from the cache" : 3366,
                                        "pages seen by eviction walk" : 0,
                                        "pages written from cache" : 8,
                                        "pages written requiring in-memory restoration" : 0,
                                        "tracked dirty bytes in the cache" : 0,
                                        "unmodified pages evicted" : 0
                                },
                                "cache_walk" : {
                                        "Average difference between current eviction generation when the page was last considered" : 0,
                                        "Average on-disk page image size seen" : 0,
                                        "Average time in cache for pages that have been visited by the eviction server" : 0,
                                        "Average time in cache for pages that have not been visited by the eviction server" : 0,
                                        "Clean pages currently in cache" : 0,
                                        "Current eviction generation" : 0,
                                        "Dirty pages currently in cache" : 0,
                                        "Entries in the root page" : 0,
                                        "Internal pages currently in cache" : 0,
                                        "Leaf pages currently in cache" : 0,
                                        "Maximum difference between current eviction generation when the page was last considered" : 0,
                                        "Maximum page size seen" : 0,
                                        "Minimum on-disk page image size seen" : 0,
                                        "Number of pages never visited by eviction server" : 0,
                                        "On-disk page image sizes smaller than a single allocation unit" : 0,
                                        "Pages created in memory and never written" : 0,
                                        "Pages currently queued for eviction" : 0,
                                        "Pages that could not be queued for eviction" : 0,
                                        "Refs skipped during cache traversal" : 0,
                                        "Size of the root page" : 0,
                                        "Total number of pages currently in cache" : 0
                                },
                                "compression" : {
                                        "compressed pages read" : 0,
                                        "compressed pages written" : 7,
                                        "page written failed to compress" : 0,
                                        "page written was too small to compress" : 1
                                },
                                "cursor" : {
                                        "bulk-loaded cursor-insert calls" : 0,
                                        "close calls that result in cache" : 0,
                                        "create calls" : 3,
                                        "cursor operation restarted" : 0,
                                        "cursor-insert key and value bytes inserted" : 188433,
                                        "cursor-remove key bytes removed" : 0,
                                        "cursor-update value bytes updated" : 0,
                                        "cursors reused from cache" : 3364,
                                        "insert calls" : 3366,
                                        "modify calls" : 0,
                                        "next calls" : 0,
                                        "open cursor count" : 0,
                                        "prev calls" : 1,
                                        "remove calls" : 0,
                                        "reserve calls" : 0,
                                        "reset calls" : 6734,
                                        "search calls" : 0,
                                        "search near calls" : 0,
                                        "truncate calls" : 0,
                                        "update calls" : 0
                                },
                                "reconciliation" : {
                                        "dictionary matches" : 0,
                                        "fast-path pages deleted" : 0,
                                        "internal page key bytes discarded using suffix compression" : 13,
                                        "internal page multi-block writes" : 0,
                                        "internal-page overflow keys" : 0,
                                        "leaf page key bytes discarded using prefix compression" : 0,
                                        "leaf page multi-block writes" : 1,
                                        "leaf-page overflow keys" : 0,
                                        "maximum blocks required for a page" : 1,
                                        "overflow values written" : 0,
                                        "page checksum matches" : 0,
                                        "page reconciliation calls" : 2,
                                        "page reconciliation calls for eviction" : 0,
                                        "pages deleted" : 0
                                },
                                "session" : {
                                        "object compaction" : 0
                                },
                                "transaction" : {
                                        "update conflicts" : 0
                                }
                        },
                        "nindexes" : 2,
                        "totalIndexSize" : 114688,
                        "indexSizes" : {
                                "_id_" : 40960,
                                "id_hashed" : 73728
                        },
                        "ok" : 1,
                        "operationTime" : Timestamp(1559279119, 1),
                        "$gleStats" : {
                                "lastOpTime" : {
                                        "ts" : Timestamp(1559279079, 153),
                                        "t" : NumberLong(1)
                                },
                                "electionId" : ObjectId("7fffffff0000000000000001")
                        },
                        "lastCommittedOpTime" : Timestamp(1559279119, 1),
                        "$configServerState" : {
                                "opTime" : {
                                        "ts" : Timestamp(1559279109, 3),
                                        "t" : NumberLong(1)
                                }
                        },
                        "$clusterTime" : {
                                "clusterTime" : Timestamp(1559279119, 1),
                                "signature" : {
                                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                        "keyId" : NumberLong(0)
                                }
                        }
                }
        },
        "ok" : 1,
        "operationTime" : Timestamp(1559279119, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1559279119, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        }
}
mongos> 
###############################################或者##########################################
mongos> db.printShardingStatus()
--- Sharding Status --- 
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5cf0ad6f6006ca0b06ae1326")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/10.211.55.19:27001,10.211.55.20:27001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/10.211.55.20:27002,10.211.55.21:27002",  "state" : 1 }
        {  "_id" : "shard3",  "host" : "shard3/10.211.55.19:27003,10.211.55.21:27003",  "state" : 1 }
  active mongoses:
        "4.0.6" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1  1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 
        {  "_id" : "test_db",  "primary" : "shard3",  "partitioned" : true,  "version" : {  "uuid" : UUID("ef994ebd-8eb7-4ce7-b2e4-63ec7401e68c"),  "lastMod" : 1 } }
                test_db.table1
                        shard key: { "id" : "hashed" }      #hash策略
                        unique: false
                        balancing: true
                        chunks:
                                shard1  2
                                shard2  2
                                shard3  2
                        { "id" : { "$minKey" : 1 } } -->> { "id" : NumberLong("-6148914691236517204") } on : shard1 Timestamp(1, 0) 
                        { "id" : NumberLong("-6148914691236517204") } -->> { "id" : NumberLong("-3074457345618258602") } on : shard1 Timestamp(1, 1) 
                        { "id" : NumberLong("-3074457345618258602") } -->> { "id" : NumberLong(0) } on : shard2 Timestamp(1, 2) 
                        { "id" : NumberLong(0) } -->> { "id" : NumberLong("3074457345618258602") } on : shard2 Timestamp(1, 3) 
                        { "id" : NumberLong("3074457345618258602") } -->> { "id" : NumberLong("6148914691236517204") } on : shard3 Timestamp(1, 4) 
                        { "id" : NumberLong("6148914691236517204") } -->> { "id" : { "$maxKey" : 1 } } on : shard3 Timestamp(1, 5) 

mongos>

十、启动关闭

mongodb的启动顺序是,先启动所有节点配置服务器,在启动所有节点的分片,最后启动所有节点的mongos

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/config.conf

/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard1.conf
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard2.conf
/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard3.conf

/usr/local/mongodb/bin/mongos -f /usr/local/mongodb/bin/data/conf/mongos.conf

关闭时,直接杀掉

killall mongod
killall mongos

系统默认没有killall命令,所以需要安装

yum install psmisc

十一、设置开机自启

cd /etc/systemd/system

分别为config、shard1、shard2、shard3、mongos编写service文件,如下

vim mongo-config.service

[Unit]
Description=mongo-config

[Service]
user=root
group=root
Type=forking
ExecStart=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/config.conf
TimeoutStopSec=0
Restart=always
RestartSec=15s
TimeoutStartSec=30s
#ExecStop=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/config.conf --shutdown

[Install]
WantedBy=multi-user.target
Alias=mongoc
#########################################分割线##############################################
vim mongo-shard1.service

[Unit]
Description=mongo-shard1
Requires=mongos.service
After=mongos.service

[Service]
user=root
group=root
Type=forking
ExecStart=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard1.conf
TimeoutStopSec=0
Restart=always
RestartSec=15s
TimeoutStartSec=30s
#ExecStop=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard1.conf --shutdown

[Install]
WantedBy=multi-user.target
Alias=mongosh1
#########################################分割线##############################################
vim mongo-shard2.service

[Unit]
Description=mongo-shard2
Requires=mongo-shard1.service
After=mongo-shard1.service

[Service]
user=root
group=root
Type=forking
ExecStart=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard2.conf
TimeoutStopSec=0
Restart=always
RestartSec=15s
TimeoutStartSec=30s
#ExecStop=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard2.conf --shutdown

[Install]
WantedBy=multi-user.target
Alias=mongosh2
#########################################分割线##############################################

vim mongo-shard3.service

[Unit]
Description=mongo-shard3
Requires=mongo-shard2.service
After=mongo-shard2.service

[Service]
user=root
group=root
Type=forking
ExecStart=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard3.conf
TimeoutStopSec=0
Restart=always
RestartSec=15s
TimeoutStartSec=30s
#ExecStop=/usr/local/mongodb/bin/mongod -f /usr/local/mongodb/bin/data/conf/shard3.conf --shutdown

[Install]
WantedBy=multi-user.target
Alias=mongosh3

#########################################分割线##############################################
vim mongos.service

[Unit]
Description=mongos
Requires=mongo-config.service
After=mongo-config.service

[Service]
user=root
group=root
Type=forking
ExecStart=/usr/local/mongodb/bin/mongos -f /usr/local/mongodb/bin/data/conf/mongos.conf
TimeoutStopSec=0
Restart=always
RestartSec=15s
TimeoutStartSec=30s
#ExecStop=/usr/local/mongodb/bin/mongos -f /usr/local/mongodb/bin/data/conf/mongos.conf

[Install]
WantedBy=multi-user.target

添加文件权限,设置开机自启

chmod +x mongo*.service

systemctl daemon-reload

systemctl enable mongo-config.service
systemctl enable mongo-shard1.service
systemctl enable mongo-shard2.service
systemctl enable mongo-shard3.service
systemctl enable mongos.service

扩展命令

mongos> db.runCommand( { listshards : 1 } )     #列出分片
mongos> sh.status();       #查看总体状态
db.comment.getShardDistribution()       #需要切换到db下查看book集合的分片状态

激活数据库分片功能

语法:( { enablesharding : "数据库名称" } )
mongos> db.runCommand( { enablesharding : "test" } )

指定分片建对集合分片,范围片键--创建索引

mongos> use test 
mongos> db.vast.ensureIndex( { id: 1 } )
mongos> use admin
mongos> db.runCommand( { shardcollection : "test.vast",key : {id: 1} } )

集合分片验证

mongos> use test
mongos> for(i=0;i<20000;i++){ db.vast1.insert({"id":i,"name":"clsn","age":70,"date":new Date()}); }
mongos> db.vast.stats()

分片集群的操作

范围片键

admin> sh.shardCollection("数据库名称.集合名称",key : {分片键: 1}  )
或
admin> db.runCommand( { shardcollection : "数据库名称.集合名称",key : {分片键: 1} } )
eg:

admin > sh.shardCollection("test.vast",key : {id: 1}  )
或
admin> db.runCommand( { shardcollection : "test.vast",key : {id: 1} } )

哈希片键

admin > sh.shardCollection( "数据库名.集合名", { 片键: "hashed" } )

创建哈希索引

admin> db.vast.ensureIndex( { a: "hashed" } )
admin > sh.shardCollection( "test.vast", { a: "hashed" } )

分片集群的操作

判断是否Shard集群

admin> db.runCommand({ isdbgrid : 1})

列出所有分片信息

admin> db.runCommand({ listshards : 1})

列出开启分片的数据库

admin> use config
config> db.databases.find( { "partitioned": true } )
config> db.databases.find() //列出所有数据库分片情况

查看分片的片键

config> db.collections.find()
{
    "_id" : "test.vast",
    "lastmodEpoch" : ObjectId("58a599f19c898bbfb818b63c"),
    "lastmod" : ISODate("1970-02-19T17:02:47.296Z"),
    "dropped" : false,
    "key" : {
        "id" : 1
    },
    "unique" : false
}

查看分片的详细信息

admin> db.printShardingStatus()
或
admin> sh.status()

删除分片节点

sh.getBalancerState()
mongos> db.runCommand( { removeShard: "shard2" } )

balance操作

查看mongo集群是否开启了 balance 状态

mongos> sh.getBalancerState()
true

当然你也可以通过在路由节点mongos上执行sh.status() 查看balance状态。

如果balance开启,查看是否正在有数据的迁移

连接mongo集群的路由节点

mongos> sh.isBalancerRunning()
false

设置balance 窗口

(1)连接mongo集群的路由节点

(2)切换到配置节点

use config

(3)确定balance 开启中

 sh.getBalancerState()

如果未开启,执行命令

sh.setBalancerState( true )

(4)修改balance 窗口的时间

db.settings.update(
   { _id: "balancer" },
   { $set: { activeWindow : { start : "<start-time>", stop : "<stop-time>" } } },
   { upsert: true }
)
eg:

db.settings.update({ _id : "balancer" }, { $set : { activeWindow : { start : "00:00", stop : "5:00" } } }, true )

当你设置了activeWindow,就不能用sh.startBalancer() 启动balance

NOTE

The balancer window must be sufficient to complete the migration of all data inserted during the day.

As data insert rates can change based on activity and usage patterns, it is important to ensure that the balancing window you select will be sufficient to support the needs of your deployment.

(5)删除balance 窗口

use config
db.settings.update({ _id : "balancer" }, { $unset : { activeWindow : true } })

关闭balance

默认balance 的运行可以在任何时间,只迁移需要迁移的chunk,如果要关闭balance运行,停止一段时间可以用下列方法:

(1) 连接到路由mongos节点

(2) 停止balance

sh.stopBalancer()

(3) 查看balance状态

sh.getBalancerState()

(4)停止balance 后,没有迁移进程正在迁移,可以执行下列命令

use config
while( sh.isBalancerRunning() ) {
          print("waiting...");
          sleep(1000);
}

重新打开balance

如果你关闭了balance,准备重新打开balance

(1) 连接到路由mongos节点

(2) 打开balance

 sh.setBalancerState(true)

如果驱动没有命令 sh.startBalancer(),可以用下列命令

use config
db.settings.update( { _id: "balancer" }, { $set : { stopped: false } } , { upsert: true } )

关于集合的balance

关闭某个集合的balance

sh.disableBalancing("students.grades")

打开某个集合的balance

sh.enableBalancing("students.grades")

确定某个集合的balance是开启或者关闭

db.getSiblingDB("config").collections.findOne({_id : "students.grades"}).noBalance;

问题解决

mongodb在做自动分片平衡的时候,或引起数据库响应的缓慢,可以通过禁用自动平衡以及设置自动平衡进行的时间来解决这一问题。

(1)禁用分片的自动平衡

// connect to mongos
> use config
> db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true );

(2)自定义 自动平衡进行的时间段

// connect to mongos
> use config
> db.settings.update({ _id : "balancer" }, { $set : { activeWindow : { start : "21:00", stop : "9:00" } } }, true )

问题扩展:

提示:please create an index that starts with the shard key before sharding

场景:

mongo数据库test分片已经启用,在新增加一个collection之后,并且该表已经有数据写入,当对该表启用分片的时候报错:please create an index that starts with the shard key before sharding

解决思路:

大概意思是在启用分片前需要在分片key上建立索引。我并没有立刻在分片key上建立索引,因为之前在启用表分片,在分片key上会自动建立索引,为什么要我手动建立呢?所以先google之看有没有相关资料,没想到搜索的结果大失所望,总共才5条结果。2条英文结果,3条日文。第一条好像是mongodb源码,当时没注意。没办法只能看下根据提示关键字搜索下mongodb源码,在src/mongo/s/commands_admin.cpp文件中544行找到这个错误信息

好了找到根源了,现在解决问题:在分片key上个建立索引

use test  db.ttl.ensureIndex({date:1,node_id:1})

然后对表进行分片

sh.shardCollection('test.ttl',{date:1,node_id:1})

结果显示:

{ "collectionsharded" : "test.ttl", "ok" : 1 }

OK.至此问题解决.

avatar

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: