I am load testing my mongodb. I am trying to get 100k insert/sec.
i5 2400 quad-core 32 GB ram win7 spinning disk
mongos + 3 config + 4 shards on localhost
c++ load generator is build in vs2013 x64 c++ driver w/ boost 1.57 64 bit
3k insert / sec at 1 thread load generator
4k insert / sec at 2 thread load generator
10k insert / sec at 4 8 16 32 load generator
mongos cpu utilization is up to 25-33%, i.e. 1-2 cores
mongod x 4 cpu utilization is up to 10%
I am inserting the same 500 byte document w/ an auto-generated _id field repeatedly. The shard key is {"_id": "hashed"}.
The 4 shards are all getting ~25% of the data. Total data size is less than 1GB
Below please find the cygwin bash scripts for starting the cluster.
With robomongo and javascript I have got 18k insert/sec. mongos cpu% is up to 40%.
==> createShardedDb.js <==
sh.addShard("localhost:27030")
sh.addShard("localhost:27031")
sh.addShard("localhost:27032")
sh.addShard("localhost:27033")
sh.enableSharding("test");
sh.shardCollection("test.
sh.shardCollection("test.
==> startMongoCluster.sh <==
#!/bin/sh
set -o xtrace
set -o nounset
set -o errexit
./killAllMongo.sh
sleepTime=2
sleep $sleepTime
rm -Rf /cygdrive/c/data
rm -Rf /data
mkdir -p /cygdrive/c/data
ln -s /cygdrive/c/data /data
pushd /data
mkdir -p log
mkdir -p configdb0 configdb1 configdb2
mkdir -p shard0 shard1 shard2 shard3
# always 3 config servers
mongod --configsvr --dbpath /data/configdb0 --port 27040 --logpath /data/log/configdb0.log &
sleep $sleepTime
mongod --configsvr --dbpath /data/configdb1 --port 27041 --logpath /data/log/configdb1.log &
sleep $sleepTime
mongod --configsvr --dbpath /data/configdb2 --port 27042 --logpath /data/log/configdb2.log &
sleep $sleepTime
# shards
mongod --dbpath /data/shard0 --port 27030 --logpath /data/log/shard0.log &
sleep $sleepTime
mongod --dbpath /data/shard1 --port 27031 --logpath /data/log/shard1.log &
sleep $sleepTime
mongod --dbpath /data/shard2 --port 27032 --logpath /data/log/shard2.log &
sleep $sleepTime
mongod --dbpath /data/shard3 --port 27033 --logpath /data/log/shard3.log &
sleep $sleepTime
# query router
mongos -vv --port 27017 --configdb localhost:27040,localhost:
sleep $sleepTime
Disk i/o on the mongod shard server seems to almost always be the bottlneck. Remember that with 4 mongod instances on a single machine, they're all sharing the same disk bottleneck. Take a look at ioping for disk i/o benchmarking. Run ioping before the load to get a baseline of what IOPS your disk setup is capable of, then run during the load to see if the shards are indeed saturating your i/o capabilities.
Can the windows OS disk cache be the bottleneck? OS disk caches defer flushing to disk. In windows task manager > performance tab, ~30% of cpu usage is red (kernel times).
My data size is less than 1GB. My 32GB memory is much larger than that. The HDD LED rarely turns on. I tried starting with an empty database, or a database with a few gigs of data.
In the Java API there is a parameter to use UNKNOWLEDGED write concern. How can I defer disk writes on the mongodb server? Thx.
C:\downloads\ioping-0.9-win32>
ioping: non-cached read I/O not supported by this platform
ioping: you can use write I/O to get reliable results
4 KiB from . ( ): request=1 time=0 us
4 KiB from . ( ): request=2 time=0 us
4 KiB from . ( ): request=3 time=0 us
4 KiB from . ( ): request=4 time=0 us
4 KiB from . ( ): request=5 time=0 us
4 KiB from . ( ): request=6 time=0 us
4 KiB from . ( ): request=7 time=0 us
4 KiB from . ( ): request=8 time=0 us
4 KiB from . ( ): request=9 time=0 us
4 KiB from . ( ): request=10 time=0 us
4 KiB from . ( ): request=11 time=0 us
4 KiB from . ( ): request=12 time=0 us
4 KiB from . ( ): request=13 time=0 us
I have moved the logs, 3 configs and the 4 shards to a 5GB ram disk. I have fed 270 MB into it. I am still getting 10k insert / sec.
mongos cpu usage is up to 34%
mongod cpu usage is up to 12% each
In windows resource monitor > overview, there are 4 sections (cpu/disk/network/memory). None of them is over 80%. Where is the bottleneck? cpu 55%, disk 0, network 150 Mbps, memory 0 page fault. I am sending data to mongodb using a single thread load generator. All network traffic is in localhost. Thx.
If you are using a single mongos it's very possible that this is your bottleneck at the moment. You have four shards? I would recommend trying four mongos (and 4x the number of client threads as you currently are).
P.S. I may have missed the version of MongoDB you are running - there've been Windows related issues some of which have been fixed in later versions.
mongodb-win32-x86_64-2008plus-
If your data is coming from one thread, why are you expecting to reach 100K ops? Aren't you going to be bound by the client at this point?
He said 150k tps here
I have tried inserting to just mongod. I got about 20-30k tps, which is still far from 100k tps.
I have tried inserting to mongos -> mongod, i.e. 1 shard. Both mongos and mongod are cpu-bound.
The data has to come from a single client thread. No concurrency. Sometimes I get less than 10k tps.
If your data is coming from one thread, why are you expecting to reach 100K ops? Aren't you going to be bound by the client at this point?
He said 150k tps here
댓글 없음:
댓글 쓰기