寮步建设网站,一键创建网站,服务推广软文,做视频网站需要什么条件集群命令
cluster info#xff1a;打印集群的信息。
cluster nodes#xff1a;列出集群当前已知的所有节点#xff08;node#xff09;的相关信息。
cluster meet ip port#xff1a;将ip和port所指定的节点添加到集群当中。
cluster addslots slot…集群命令
cluster info打印集群的信息。
cluster nodes列出集群当前已知的所有节点node的相关信息。
cluster meet ip port将ip和port所指定的节点添加到集群当中。
cluster addslots slot [slot ...]将一个或多个槽slot指派assign给当前节点。
cluster delslots slot [slot ...]移除一个或多个槽对当前节点的指派。
cluster slots列出槽位、节点信息。
cluster slaves node_id列出指定节点下面的从节点信息。 比如 cluster slaves 11f9169577352c33d85ad0d1ca5f5bf0deba3209 这个实际查的是nodes.conf
cluster replicate node_id将当前节点设置为指定节点的从节点。
cluster saveconfig 手动执行命令保存保存集群的配置文件集群默认在配置修改的时候会自动保存配置文件。
cluster keyslot key 列出key被放置在哪个槽上。 例如cluster keyslot 9223372036854742675
cluster flushslots 移除指派给当前节点的所有槽让当前节点变成一个没有指派任何槽的节点。
cluster countkeysinslot slot 返回槽目前包含的键值对数量。
cluster getkeysinslot slot槽号 key的数量返回count个槽中的键。 比如cluster getkeysinslot 202 3
cluster setslot slot node node_id 将槽指派给指定的节点如果槽已经指派给另一个节点那么先让另一个节点删除该槽然后再进行指派。
cluster setslot slot migrating node_id 将本节点的槽迁移到指定的节点中。
cluster setslot slot importing node_id 从 node_id 指定的节点中导入槽 slot 到本节点。
cluster setslot slot stable 取消对槽 slot 的导入import或者迁移migrate。
cluster failover 手动进行故障转移。
cluster forget node_id 从集群中移除指定的节点这样就无法完成握手过期时为60s60s后两节点又会继续完成握手。
cluster reset [HARD|SOFT]重置集群信息soft是清空其他节点的信息但不修改自己的idhard还会修改自己的id不传该参数则使用soft方式。
cluster count-failure-reports node_id列出某个节点的故障报告的长度。
cluster SET-CONFIG-EPOCH 设置节点epoch只有在节点加入集群前才能设置。集群环境搭建
搭建集群至少需要6台服务器或虚拟机如果想要搭建测试可以根据端口号模拟服务器虚拟运行6个redis实例。
1下载redis安装包编译安装指定目录/usr/local/redis/
2创建/usr/local/redis-cluster/目录然后把安装目录下的bin目录复制到redis-cluster/下重命名为redis01~06
3修改配置文件#bind 127.0.0.1 注释允许其他ip访问
protected-mode no # 改为no; yes 保护模式只允许本地链接no 保护模式关闭
port 7001/7002/7003/7004/7005/7006 # 六个配置文件分别配置不同端口
daemonize: yes # 后台运行
masterauth 1234 # 主机密码
requirepass 1234 # 本机密码
cluster-enabled yes # 开启集群
cluster-enabled cluster-config-file nodes-7001.conf # 集群配置文件集群创建成功会自动生成不配置默认nodes.conf
cluster-node-timeout 15000 # 节点失联时间超过该时间毫秒集群自动进行主从切换。
appendonlyyes # 开启AOF持久化
4创建集群中途会有个提示输入yes回车即可/usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234[rootVM-0-12-centos /]# cd /usr/local/redis
[rootVM-0-12-centos redis]# wget https://github.com/redis/redis/archive/7.0.4.tar.gz
[rootVM-0-12-centos redis]# tar -xzvf 7.0.4.tar.gz
[rootVM-0-12-centos redis]# ll
total 2932
-rw-r--r-- 1 root root 2994242 Aug 8 17:03 7.0.4.tar.gz
drwxrwxr-x 8 root root 4096 Jul 18 21:04 redis-7.0.4
[rootVM-0-12-centos redis]# cd redis-7.0.4
[rootVM-0-12-centos redis]# make make install PREFIX/usr/local/redis # 指定安装目录,会在redis目录下生成一个bin/目录,如果不指定安装目录会在 redis-7.0.4/src/ 生成启动文件我这里为了自己操作方便些直接在 redis/bin 目录下操作
[rootVM-0-12-centos redis]# cd ..
[rootVM-0-12-centos redis]# ls
7.0.4.tar.gz bin redis-7.0.4
[rootVM-0-12-centos redis]# cd bin/
[rootVM-0-12-centos bin]# ls
redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-sentinel redis-server
[rootVM-0-12-centos bin]# cp ../redis-7.0.4/redis.conf ./
[rootVM-0-12-centos bin]# ls
redis-benchmark redis-check-aof redis-check-rdb redis-cli redis.conf redis-sentinel redis-server
[rootVM-0-12-centos bin]# vim ./redis.conf # 修改daemonize 的值为 yes开启后台启动
[rootVM-0-12-centos bin]# ./redis-server ./redis.conf
[rootVM-0-12-centos bin]# ./redis-cli
127.0.0.1:6379
rootVM-0-12-centos bin]# cd ../..
[rootVM-0-12-centos local]# mkdir redis-cluster
[rootVM-0-12-centos local]# cd redis-cluster/
[rootVM-0-12-centos redis-cluster]# cp -r /usr/local/redis/bin/ ./redis01/
[rootVM-0-12-centos local]# rm -rf ./redis01/dump.rdb # 删除redis01目录下的快照文件dump.rdb
[rootVM-0-12-centos redis-cluster]# vim ./redis01/redis.conf
#bind 127.0.0.1 注释允许其他ip访问
protected-mode no
port 7001
masterauth 1234
requirepass 1234 #最好设置一下密码
cluster-enabled yes
cluster-enabled yes 打开集群模式
cluster-config-file nodes-7001.conf 设定节点配置文件名
cluster-node-timeout 15000
appendonlyyes
[rootVM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis02/
[rootVM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis03/
[rootVM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis04/
[rootVM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis05/
[rootVM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis06/
[rootVM-0-12-centos redis-cluster]# vim ./redis01/redis.conf # 逐个修改端口号
[rootVM-0-12-centos redis-cluster]# vim start-all.sh # 创建启动文件也可手动启动所有服务
cd redis01
./redis-server ./redis.conf
cd ..
cd redis02
./redis-server ./redis.conf
cd ..
cd redis03
./redis-server ./redis.conf
cd ..
cd redis04
./redis-server ./redis.conf
cd ..
cd redis05
./redis-server ./redis.conf
cd ..
cd redis06
./redis-server ./redis.conf
cd ..
[rootVM-0-12-centos redis-cluster]# vim stop-all.sh # 创建关闭文件也可手动停止所有服务
cd redis01
./redis-cli -h 127.0.0.1 -p 7001 -a 1234 shutdown
cd ..
cd redis02
./redis-cli -h 127.0.0.1 -p 7002 -a 1234 shutdown
cd ..
cd redis03
./redis-cli -h 127.0.0.1 -p 7003 -a 1234 shutdown
cd ..
cd redis04
./redis-cli -h 127.0.0.1 -p 7004 -a 1234 shutdown
cd ..
cd redis05
./redis-cli -h 127.0.0.1 -p 7005 -a 1234 shutdown
cd ..
cd redis06
./redis-cli -h 127.0.0.1 -p 7006 -a 1234 shutdown
cd ..
[rootVM-0-12-centos redis-cluster]# chmod x start-all.sh
[rootVM-0-12-centos redis-cluster]# chmod x stop-all.sh
[rootVM-0-12-centos redis-cluster]# ./start-all.sh
[rootVM-0-12-centos redis-cluster]# ps aux | grep redis
root 17885 0.1 0.1 163152 3724 ? Ssl 17:52 0:01 ./redis-server *:6379
root 23399 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7001 [cluster]
root 23401 0.0 0.1 165712 3660 ? Ssl 18:20 0:00 ./redis-server *:7002 [cluster]
root 23403 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7003 [cluster]
root 23405 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7004 [cluster]
root 23411 0.0 0.1 171856 3664 ? Ssl 18:20 0:00 ./redis-server *:7005 [cluster]
root 23413 0.0 0.1 165712 3672 ? Ssl 18:20 0:00 ./redis-server *:7006 [cluster]
root 23495 0.0 0.0 112816 976 pts/0 R 18:21 0:00 grep --colorauto redis
[rootVM-0-12-centos redis-cluster]# ls
redis01 redis02 redis03 redis04 redis05 redis06 start-all.sh stop-all.sh
[rootVM-0-12-centos redis-cluster]# /usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
Warning: Using a password with -a or -u option on the command line interface may not be safe.Performing hash slots allocation on 6 nodes...
Master[0] - Slots 0 - 5460
Master[1] - Slots 5461 - 10922
Master[2] - Slots 10923 - 16383
Adding replica 182.254.134.174:7005 to 182.254.134.174:7001
Adding replica 182.254.134.174:7006 to 182.254.134.174:7002
Adding replica 182.254.134.174:7004 to 182.254.134.174:7003Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d 182.254.134.174:7001slots:[0-5460] (5461 slots) master
M: 1194702cf2b7ef4f5d5e7854f526aa60d47f2577 182.254.134.174:7002slots:[5461-10922] (5462 slots) master
M: b120e1a401f11a06f8d81f19df2607c71c5cf9df 182.254.134.174:7003slots:[10923-16383] (5461 slots) master
S: 28362fb68d33eb50322f514daf4e7a447e3eccb3 182.254.134.174:7004replicates 1194702cf2b7ef4f5d5e7854f526aa60d47f2577
S: 52b90d0718c06990de002dbe00be512c0f4eccd1 182.254.134.174:7005replicates b120e1a401f11a06f8d81f19df2607c71c5cf9df
S: 2aedbda7dadc48e545ec0870b7d3585bb4637c47 182.254.134.174:7006replicates e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d
Can I set the above configuration? (type yes to accept): yesNodes configuration updatedAssign a different config epoch to each nodeSending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..Performing Cluster Check (using node 182.254.134.174:7001)
M: e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d 182.254.134.174:7001slots:[0-5460] (5461 slots) master1 additional replica(s)
M: 1194702cf2b7ef4f5d5e7854f526aa60d47f2577 182.254.134.174:7002slots:[5461-10922] (5462 slots) master1 additional replica(s)
S: 2aedbda7dadc48e545ec0870b7d3585bb4637c47 182.254.134.174:7006slots: (0 slots) slavereplicates e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d
M: b120e1a401f11a06f8d81f19df2607c71c5cf9df 182.254.134.174:7003slots:[10923-16383] (5461 slots) master1 additional replica(s)
S: 28362fb68d33eb50322f514daf4e7a447e3eccb3 182.254.134.174:7004slots: (0 slots) slavereplicates 1194702cf2b7ef4f5d5e7854f526aa60d47f2577
S: 52b90d0718c06990de002dbe00be512c0f4eccd1 182.254.134.174:7005slots: (0 slots) slavereplicates b120e1a401f11a06f8d81f19df2607c71c5cf9df
[OK] All nodes agree about slots configuration.Check for open slots...Check slots coverage...
[OK] All 16384 slots covered.
[rootVM-0-12-centos redis-cluster]#模拟主机宕机情况看看是否自动 slave自动升为master
[rootVM-0-12-centos redis-cluster]# ./redis01/redis-cli -p 7001 -a 1234
Warning: Using a password with -a or -u option on the command line interface may not be safe.
127.0.0.1:7001 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:886
cluster_stats_messages_pong_sent:887
cluster_stats_messages_sent:1773
cluster_stats_messages_ping_received:882
cluster_stats_messages_pong_received:886
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1773
total_cluster_links_buffer_limit_exceeded:0
127.0.0.1:7001 cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:700317003 master - 0 1660036015000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:700517005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660036017989 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 182.254.134.174:700217002 master - 0 1660036016986 2 connected 5461-10922
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:700117001 myself,master - 0 1660036015000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:700617006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660036016000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:700417004 slave cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 0 1660036015982 2 connected
重新开一个ssh远程连接,连接7002服务关闭redis等待15秒
[rootVM-0-12-centos redis-cluster]# ./redis02/redis-cli -p 7002 -a 1234
Warning: Using a password with -a or -u option on the command line interface may not be safe.
127.0.0.1:7002 shutdown
再次查看节点信息会发现7004已经已经自动升为master
127.0.0.1:7001 cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:700317003 master - 0 1660036206861 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:700517005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660036204840 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 182.254.134.174:700217002 master,fail - 1660036178557 1660036172498 2 disconnected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:700117001 myself,master - 0 1660036203000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:700617006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660036207000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:700417004 master - 0 1660036207872 7 connected 5461-10922
127.0.0.1:7001
操作中遇到的问题
1、[ERR] Node 182.254.134.174:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.[rootVM-0-12-centos redis-cluster]# /usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
Warning: Using a password with -a or -u option on the command line interface may not be safe.
[ERR] Node 182.254.134.174:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.原因上面的命令只能在新创健集群的时候执行一次目的是为了建立内部各个节点的对应关系比如主从关系这些关系仅且只能在一个集群中初始化时对应一次 解决办法删除aofrdbnodes.conf文件重新启动我写了一个清除文件脚本 clear.sh
cd ..
cd redis02
rm -f nodes-7002.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis03
rm -f nodes-7003.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis04
rm -f nodes-7004.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis05
rm -f nodes-7005.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis06
rm -f nodes-7006.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..[rootVM-0-12-centos redis-cluster]# ./clear.sh
[rootVM-0-12-centos redis-cluster]# ./start-all.sh
/usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
正常启动重复代码就不粘贴了...00 master,fail,noaddr - 1660036178557 1660036172498 2 disconnected
7004自动升为master后重新启动7002查看节点状态后发现发现7002已脱离集群且id都已发生了变化
127.0.0.1:7001 cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:700317003 master - 0 1660039460000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:700517005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660039461528 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 :00 master,fail,noaddr - 1660036178557 1660036172498 2 disconnected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:700117001 myself,master - 0 1660039460000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:700617006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660039460000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:700417004 master - 0 1660039460525 7 connected 5461-10922原因7002已脱离集群 解决办法将该从节点剔出集群重新加入 任意节点上执行 cluster forget命令
127.0.0.1:7001 cluster forget cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 # 踢出集群
OK
127.0.0.1:7001 cluster meet 182.254.134.174 7002 # 握手加入集群
OK
127.0.0.1:7001 cluster nodes
37d2363fbc60a64a3cb5be0cb75c4b418795b02e 182.254.134.174:700217002 master - 0 1660040192000 0 connected
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:700317003 master - 0 1660040195000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:700517005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660040193966 3 connected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:700117001 myself,master - 0 1660040195000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:700617006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660040196975 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:700417004 master - 0 1660040195972 7 connected 5461-10922然再在7002上和7004重新配置主从关系,
127.0.0.1:7002 cluster replicate 2e939c53fbc695e136bd0e63a07d6f86f9e25cb3
127.0.0.1:7002 cluster nodes
232fb95877a5f2b3093b1f7d94b632c10f55c976 182.254.134.174:700117001 master - 0 1660040830000 1 connected 0-5460
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:700317003 master - 0 1660040828781 3 connected 10923-16383
37d2363fbc60a64a3cb5be0cb75c4b418795b02e 172.17.0.12:700217002 myself,slave 2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 0 1660040827000 7 connected
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:700517005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660040829000 3 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:700417004 master - 0 1660040829785 7 connected 5461-10922
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:700617006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660040830789 1 connectedprometheus granafa 监控
下载
https://github.com/oliver006/redis_exporter/releases配置添加
prometheus.yml
在prometheus.yml添加两个jobredis_exporter和redis_exporter_targetsredis_exporter_targets才是真正监控redis集群的。 ## config for scraping the exporter itself- job_name: redis_exporterstatic_configs:- targets:- prod-xxx-xxxx-redis01:9121## config for the multiple Redis targets that the exporter will scrape- job_name: redis_exporter_targetsstatic_configs:- targets:- redis://172.16.0.122:7000- redis://172.16.0.122:7001- redis://172.16.0.120:7002- redis://172.16.0.120:7003- redis://172.16.0.129:7004- redis://172.16.0.129:7005metrics_path: /scraperelabel_configs:- source_labels: [__address__]target_label: __param_target- source_labels: [__param_target]target_label: instance- target_label: __address__replacement: prod-xxxx-xxxx-redis01:9121部署redis_exporter
REDIS_ADDR为你redis集群的其中一个节点即可默认redis_exporter 使用9121端口
redis_exporter -redis.addr $ip:6379 -redis.password $password -web.listen-address :9333 /dev/null 21 常用选项
-redis.addr 连接redis地址:端口
-redis.password 连接redis的密码
-web.listen-address 指定redis_exporter的服务端口格式为-web.listen-address :9121
然后重启prometheus。 grafana模板
id 763 文章地址: https://www.cnblogs.com/weifeng1463/p/16617374.html https://blog.csdn.net/weixin_43844718/article/details/126250467 文章转载自： http://www.morning.ybgpk.cn.gov.cn.ybgpk.cn http://www.morning.rglzy.cn.gov.cn.rglzy.cn http://www.morning.ykwgl.cn.gov.cn.ykwgl.cn http://www.morning.kjyhh.cn.gov.cn.kjyhh.cn http://www.morning.rfjmy.cn.gov.cn.rfjmy.cn http://www.morning.leyuhh.com.gov.cn.leyuhh.com http://www.morning.mjzgg.cn.gov.cn.mjzgg.cn http://www.morning.trqsm.cn.gov.cn.trqsm.cn http://www.morning.yrnll.cn.gov.cn.yrnll.cn http://www.morning.nrlsg.cn.gov.cn.nrlsg.cn http://www.morning.pqjpw.cn.gov.cn.pqjpw.cn http://www.morning.nhpmn.cn.gov.cn.nhpmn.cn http://www.morning.hhxwr.cn.gov.cn.hhxwr.cn http://www.morning.fyglr.cn.gov.cn.fyglr.cn http://www.morning.smrty.cn.gov.cn.smrty.cn http://www.morning.bftr.cn.gov.cn.bftr.cn http://www.morning.phzrq.cn.gov.cn.phzrq.cn http://www.morning.woyoua.com.gov.cn.woyoua.com http://www.morning.fbrshjf.com.gov.cn.fbrshjf.com http://www.morning.zkbxx.cn.gov.cn.zkbxx.cn http://www.morning.gpryk.cn.gov.cn.gpryk.cn http://www.morning.khntd.cn.gov.cn.khntd.cn http://www.morning.jkzq.cn.gov.cn.jkzq.cn http://www.morning.trjdr.cn.gov.cn.trjdr.cn http://www.morning.wxfjx.cn.gov.cn.wxfjx.cn http://www.morning.dbylp.cn.gov.cn.dbylp.cn http://www.morning.rpjr.cn.gov.cn.rpjr.cn http://www.morning.jybj.cn.gov.cn.jybj.cn http://www.morning.nkbfc.cn.gov.cn.nkbfc.cn http://www.morning.rzcbk.cn.gov.cn.rzcbk.cn http://www.morning.pxwzk.cn.gov.cn.pxwzk.cn http://www.morning.wjqyt.cn.gov.cn.wjqyt.cn http://www.morning.sjqml.cn.gov.cn.sjqml.cn http://www.morning.bwrbm.cn.gov.cn.bwrbm.cn http://www.morning.4r5w91.cn.gov.cn.4r5w91.cn http://www.morning.nkjxn.cn.gov.cn.nkjxn.cn http://www.morning.lnwdh.cn.gov.cn.lnwdh.cn http://www.morning.wsnbg.cn.gov.cn.wsnbg.cn http://www.morning.yckwt.cn.gov.cn.yckwt.cn http://www.morning.krtcjc.cn.gov.cn.krtcjc.cn http://www.morning.ncqzb.cn.gov.cn.ncqzb.cn http://www.morning.nlmm.cn.gov.cn.nlmm.cn http://www.morning.hrnrx.cn.gov.cn.hrnrx.cn http://www.morning.mxnhq.cn.gov.cn.mxnhq.cn http://www.morning.bjsites.com.gov.cn.bjsites.com http://www.morning.ptwqf.cn.gov.cn.ptwqf.cn http://www.morning.xstfp.cn.gov.cn.xstfp.cn http://www.morning.gfmpk.cn.gov.cn.gfmpk.cn http://www.morning.jzgxp.cn.gov.cn.jzgxp.cn http://www.morning.rksg.cn.gov.cn.rksg.cn http://www.morning.cznsq.cn.gov.cn.cznsq.cn http://www.morning.bnxnq.cn.gov.cn.bnxnq.cn http://www.morning.khyqt.cn.gov.cn.khyqt.cn http://www.morning.bnfsw.cn.gov.cn.bnfsw.cn http://www.morning.bwmq.cn.gov.cn.bwmq.cn http://www.morning.qmncj.cn.gov.cn.qmncj.cn http://www.morning.dwwbt.cn.gov.cn.dwwbt.cn http://www.morning.mdmc.cn.gov.cn.mdmc.cn http://www.morning.gyylt.cn.gov.cn.gyylt.cn http://www.morning.bmpjp.cn.gov.cn.bmpjp.cn http://www.morning.pqppj.cn.gov.cn.pqppj.cn http://www.morning.xfmzk.cn.gov.cn.xfmzk.cn http://www.morning.rbkgp.cn.gov.cn.rbkgp.cn http://www.morning.clqpj.cn.gov.cn.clqpj.cn http://www.morning.kqfdrqb.cn.gov.cn.kqfdrqb.cn http://www.morning.tsycr.cn.gov.cn.tsycr.cn http://www.morning.kqcqr.cn.gov.cn.kqcqr.cn http://www.morning.kpbn.cn.gov.cn.kpbn.cn http://www.morning.mwmtk.cn.gov.cn.mwmtk.cn http://www.morning.twwts.com.gov.cn.twwts.com http://www.morning.wwthz.cn.gov.cn.wwthz.cn http://www.morning.srjbs.cn.gov.cn.srjbs.cn http://www.morning.lprfk.cn.gov.cn.lprfk.cn http://www.morning.qynpw.cn.gov.cn.qynpw.cn http://www.morning.nxstj.cn.gov.cn.nxstj.cn http://www.morning.gxcit.com.gov.cn.gxcit.com http://www.morning.kgxrq.cn.gov.cn.kgxrq.cn http://www.morning.tqbw.cn.gov.cn.tqbw.cn http://www.morning.jpbky.cn.gov.cn.jpbky.cn http://www.morning.rrjzp.cn.gov.cn.rrjzp.cn
