月度归档:2019年10月

mongo分片及副本集搭建例子

https://github.com/senssei/mongo-cluster-docker

基于上面仓库,改造了一个副本集。
https://github.com/kxtry/mongo-cluster-docker

1.副本集docker-compose脚本

version: '3'
services:
    mgoprimary:
        container_name: "mgoprimary"
        image: mongo:4.2
        ports:
            - "27017:27017"
        command: mongod --replSet vmlocal --port 27017  --oplogSize 16
        links:
            - mgoreplica1:mgoreplica1
            - mgoreplica2:mgoreplica2
        restart: always
        extra_hosts:
            - "mgo.db.com:{{extern_ip}}"
        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
            - ./data/mgoprimary/data:/data/db

    mgoreplica1:
        container_name: "mgoreplica1"
        image: mongo:4.2
        ports:
            - "27018:27017"
        command: mongod --replSet vmlocal --port 27017  --oplogSize 16
        restart: always
        extra_hosts:
            - "mgo.db.com:{{extern_ip}}" 
        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
            - ./data/mgoreplica1/data:/data/db

    mgoreplica2:
        container_name: "mgoreplica2"
        image: mongo:4.2
        ports:
            - "27019:27017"
        command: mongod --replSet vmlocal --port 27017  --oplogSize 16        
        restart: always
        extra_hosts:
            - "mgo.db.com:{{extern_ip}}"
        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
            - ./data/mgoreplica2/data:/data/db

    setup-vmlocal:
        container_name: "setup-vmlocal"
        image: mongo:4.2
        depends_on:
            - "mgoprimary"
            - "mgoreplica1"
            - "mgoreplica2"
        extra_hosts:
            - "mgo.db.com:{{extern_ip}}"
        links:
            - mgoprimary:mgoprimary
            - mgoreplica1:mgoreplica1
            - mgoreplica2:mgoreplica2
        volumes:
            - ./scripts:/scripts
        environment: 
            - MONGO1=mgo.db.com:27017
            - MONGO2=mgo.db.com:27018
            - MONGO3=mgo.db.com:27019
            - RS=vmlocal
        entrypoint: [ "/scripts/setup.sh" ]

3.与docker-compose.yml相同子目录下的./scripts/setup.sh脚本

#!/bin/bash 

mongodb1="${MONGO1}"
mongodb2="${MONGO2}"
mongodb3="${MONGO3}"

echo "Waiting for startup.."
until mongo --host ${mongodb1} --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' &>/dev/null; do
  printf '.'
  sleep 1
done

echo "Started.."

echo setup.sh time now: `date +"%T" `
mongo --host ${mongodb1} <

3.构建脚本build_cluster.sh

#!/bin/sh                                                                                                                                                                                                   
                                                                                                                                                                                                            
path_current=`pwd`                                                                                                                                                                                          
path_script=$(cd "$(dirname "$0")"; pwd)                                                                                                                                                                    
                                                                                                                                                                                                            
mode=$1                                                                                                                                                                                                     
extern_ip=$2                                                                                                                                                                                                
                                                                                                                                                                                                            
case "$mode" in                                                                                                                                                                                             
     'build')                                                                                                                                                                                               
        if [ -f $path_script/docker-compose.yml ]; then                                                                                                                                                     
           echo 'the docker-compose.yml had been exist. if you want to continue, remove it.'                                                                                                                
           exit 1                                                                                                                                                                                           
        fi                                                                                                                                                                                                  
        /bin/cp $path_script/docker-compose.yml.template $path_script/docker-compose.yml && echo "$path_script/docker-compose.yml" | xargs /bin/sed -i "s#{{extern_ip}}#$extern_ip#g"                       
        echo 'success to build the file.'                                                                                                                                                                   
     ;;                                                                                                                                                                                                     
     *)                                                                                                                                                                                                     
       basename=`basename "$0"`                                                                                                                                                                             
       echo "Usage: $basename  {build}{extern_ip}"                                                                                                                                                          
       echo "---$basename build 192.168.10.104"                                                                                                                                                             
     ;;                                                                                                                                                                                                     
esac             

Dockerfile的CMD和ENTRYPOINT的关系

以下示范,表示该形式下的CMD与ENTRYPOINT的关系。
CMD相当于应用程序的参数,ENTRYPOINT相当于应用的main入口或主程序入口。

FROM centos 
CMD ["echo 'p222 in cmd'"]     #传递给ENTRYPOINT的参数项。
ENTRYPOINT ["echo"]     #应用入口,相当于程序的main函数

1.构建

docker build  -t test .
2.执行以下指令执行默认的CMD命令。
docker run test
输出结果:
echo 'p222 in cmd'

3.修改程序输入参数

docker run test abct123
输出结果:
abct123

mongo的测试脚本

conn = new Mongo("mongodb://wps_credit:2f25da558f6e79c452a61bea4a8d762f@10.100.2.216:27017,10.100.2.217:27017,10.100.2.218/wps_credit?maxPoolSize=300&replicaSet=kae-mongo-test42");
db = conn.getDB("wps_credit");

var t1 = new Date();
printjson(db.getCollectionNames());

var t2 = new Date().getTime();
printjson(t1, t2, t2 - t1);

mongodb脚本执行

1.编写脚本:job.js,内容如下:

conn = new Mongo("mongodb://abc:b=abc@10.0.1.183:27017,10.0.1.184:27017/wps_credit?maxPoolSize=300&replicaSet=c45134ec-6a5f-461e-8745-7081b46b0d87");
db = conn.getDB("abc");

var result = db.transaction_5.find().limit(10);
while(result.hasNext()) {
    printjson(result.next());
}
var tm = new Date();
printjson(db.transaction_6.count());
db.transaction_6.remove({});
printjson(db.transaction_7.count());
db.transaction_7.remove({});
printjson(db.transaction_8.count());
db.transaction_8.remove({});

var t2 = new Date().getTime() - t1;
printjson(t2);

2.执行它

mongo -nodb job.js

mongodb主从数据库同步

简单点说,就是从库都是新的mongodb,通过主从切换完成空间的清理。
————————-
1)先删除数据,remove不会阻塞住整个db;
找业务不忙的时间操作。
db.collection1.remove({})
db.collection2.remove({})

2)然后,secondary上的数据重新同步,这样secondary删除的空间就释放了。
同步完之后进行一次主从切换,Secondary升级为Primary

3)新Secondary再同样进行重新同步数据,同样释放空间了
4)drop掉需要清理的空表
db.collection1.drop()
db.collection2.drop()