Replica-set member not working properly.

My replica-set consists of a primary mongod, aribiter and a secondary mongod.

Secondary mongod is unable to start properly. I'm unable to open connect to this secondary mongod through mongo shell. 

I'm seeing this error in the logs :

Tue Oct 28 12:46:42 [rsStart] error: getFile() called in a read lock, yet file to return is not yet open

Tue Oct 28 12:46:42 [rsStart]        getFile(40) _files.size:2 /mongo/shard1/local.40

Tue Oct 28 12:46:42 [rsStart]        context ns: local.system.replset openallfiles:1

Tue Oct 28 12:46:42 [rsStart]  local.system.replset Assertion failure false src/mongo/db/database.cpp 257

The logs can be found at http://pastebin.com/SWy0TKTQ

I want to know what is the problem with mongod and what is the solution by which this secondary mongod can work again properly.

It is quite likely that you ran out of file descriptors.

This is supported by the following info from the logs:

1. Tue Oct 28 12:46:42 [initandlisten] fd limit hard:4096 soft:1024 max conn: 819

This indicates you have ulimit 1024 for open files.    And you are running mongod with options that cause 4x number of files to be created as "normal"  - smallfiles: true.

Normally mongod would have allocated 2GB files for the oplog but in your case they are 512MB files - plus I don't know how big your oplog was sized but default is 5% of available space (I recommend you calculate how much you actually need and specify it with oplogSize parameter).

You should review the production notes in the MongoDB docs, set appropriate ulimit values and try again.   

http://docs.mongodb.org/manual/administration/production-notes/#recommended-configuration

Three more secondary mongods belonging to different replica-sets are running on the machine. They are working fine but only this mongod is facing problem.

All the secondary mongods are identical. The only difference between them is that they belong to different sets. ulimit is set in the starting only for all the mongods.

Enough file descriptors are available.If other mongods are able to run then why this is unable to run properly. Even when other mongods are not in running state, this mongod is unable to start.

How many files does this instance have in its Data DB directory?