I have tried to add a new member to one of mongodb replica set. Data sync has been completed, but in oplog sync stage master have started using more cpu and many of requests on the backend has been timeouted.
Sometimes in oplog sync stage I have got a socket timeout on the new replica member and sync cannot be completed:
Socket recv() timeout 10.0.xx.xx:27018
[rsBackgroundSync] SocketException: remote: 10.0.xx.xx:27018 error: 9001 socket exception [RECV_TIMEOUT] server [10.0.xx.xx:27018]
[rsBackgroundSync] DBClientCursor::init call() failed
in loop
but network is Ok.
Another indications (on master):
- flush avg is higher than normal
- lock % is higer
- read queue > 4K
- non-mapped virtual memory increased from from 15Gb to 18Gb
- Connections count increased from 11K to 15K
configured oplog size rs4: 51200MB (36h)
mongodb 2.6.5
the cluseter have 6 replica sets with 4 members each (i2.2xlarge)
You can speed up the sync of a new secondary considerably by seeding it with data files from one of the other secondaries in the replica set.
Yes, but sometimes I need to resync data from scratch and it's not Ok when mongodb have unusual activity
댓글 없음:
댓글 쓰기