MongoDB 2.6 doesn't reuse empty space.

I used MongoDB 2.4.9

I have huge collection. And I use TTL index to set "time to live" my documents ( http://docs.mongodb.org/manual/tutorial/expire-data/ )

All was fine. Files on HDD was about 80 GB and IMPORTANT!!!! summary size of files didn't grow.

Now I have upgraded to MondoDB 2.6.5

From those day mongodb constantly generates new collection.num files and eats disk space.

Here I can see http://docs.mongodb.org/manual/faq/storage/

"Empty records.

MongoDB maintains lists of empty records in data files when deleting documents and collections. MongoDB can reuse this space, but will never return this space to the operating system."

MongoDB  2.4 reused empty space. And MongoDB 2.6 doesn't reuse :(

Why?!

Do you have any capped collections which you drop and recreate?

We don't use capped collections.

Other than a possible bug, I'm not aware of any reason why your collections would continue growing with 2.6.5 when they were stable in 2.4.9.

Is the growth rate constant?  How long have the collections been growing?   Have they stabilized at some point?

Hi there, 

I have to confirm this crazy behavior of  2.6. mongodb series.

Our collection had 1.2 M rows and 1.3 TB. (scanned A4 documents)

In July we had deleted 500k rows, so status was 800 MB and 700k rows.

Every day we insert 5 GB of data and during night we delete 4-5 GB of obsolete data (1 year retention)

Under 2.6.[1.,3.,5.] versions the space was continually (linearly) growing although logically it should be steady for some time because of the 500k row deletion.

Last week we have downgraded to 2.4.12 and .....yes the space is not growing anymore and it looks like all deleted space is re-used by mongodb. 

1 week is of course short period to say some conclusions, but it looks promising.

Were you able to figure this out? If so, is there an easy repro for this? I'm working on a 2.6 upgrade now and this will definitely affect us. 

FWIW, I tried reproducing the problem on 2.6.5 using the following steps:

> use repro

> db.testcol.ensureIndex({"createdAt": 1}, {background: true, expireAfterSeconds: 60})

> for (i = 0; i < 10000000; i++) { db.testcol.insert({ createdAt: new Date(), data: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}) }

The TTL index is reaping the expired documents just fine and the doc storage size isn't growing.

I've been theorizing about this and the only thing I can come up with is that maybe for people who are seeing this happen the document size changed over the last year (or whatever the TTL period is) to be larger.   If the documents deleted are smaller and the new ones are all ones that cannot be accommodated in the freed space then I can see more space being allocated, but that should have been happening in 2.4 as well (except the behavior would be the same if TTL index is used in both versions).

It sounds like the OP was manually deleting the expired docs, rather than using TTL...

Try to disable newCollectionsUsePowerOf2Sizeshttp://docs.mongodb.org/manual/reference/parameters/#param.newCollectionsUsePowerOf2Sizes

It is help me!

Have you filed a Jira ticket (in jira.mongodb.org)?

This sounds like a bug to me.