A bit of help needed with setting up MMS on an AWS EC2 Instance

I have an AWS EC2 instance running with my MMS-agent installed on it and the agent is detected and seems to function correctly. 

In the Servers tab of MMS, the monitoring agent is signaled with a green dot at version 2.8.0 so that seems OK.

The problem is, I can't seem to get any data from added hosts.... 

I have set my ports on the instance and allowed 443 outbound for the agent to connect to the MMS server.

My issue is with defining the hosts as it seems. 

I have trouble defining the right host as well so I am not even sure the hosts available (why does it find more than one?) are correct. 

In the "servers"-tab of MMS, all processes are signaling "!".

Can someone who knows how this needs to be setup with an AWS EC2 instance please assist a bit?

How is the host name supposed to be found?

I have tried many things, like (where {description} is exchanged with the actual IP address, or DNS data):

{ipaddress}:27017 

ip-{public DNS} 

ip-{private IP}

But when I click on those server names in the deployment tap, mms tells me that "no data has been collected for this host by the Monitoring Agent".

So now I have several hosts in the servers tab, and none of them seem to actually work?

I am stuck here. I have looked at the troubleshooting section of documentation. 

I can connect over terminal to the mongo instance when I am "in the box". 

The mongod instance is running, I can connect to it with either mongo --port 27017, or mongo 127.0.0.1:27017 but as soon as I try it with any of the hostnames in front of :27017 it fails.

Any help is appreciated !

Are you already running MongoDB and trying to "attach" an already-running mongoD process to the new MMS? This is not yet supported. Right now, you can take your Automation Agent(s) and deploy a new MongoDB installation. However, if you have a "MMS Classic" group (indicated by the "Classic" text in the upper-left), you can monitor existing deployments.

Which circumstance are you in?

No that is not the case. 

I have a "classic" account, and according to Andrew Erlichson's statement in another thread here in this group, with the classic account I should be able to connect to an existing mongod process/server.

But I can not. So something else must be going on. I wonder if it has anything to do with the fact that the monitoring agent AND the mongod instance are on the same AWS EC2 instance??

That should not be a problem, however, be aware that if you have other things being monitored in this group, all monitoring agents must be able to see all MongoDB hosts by their hostnames in MMS. We have only one active Monitoring Agent at any given time per group. Another possible cause could be that your Monitoring Agent config file does not have your Agent API key in it – check the installation instructions for your OS on the Administration... Agents page. If you needed to add the API key, please restart your Monitoring Agent.

Thanks for your assistance !!

Although I was already confident that I had put in the API key as per installation instructions, I just double-checked anyway just to make sure.

Being placed in the /etc/mongodb-mms directory, the monitoring-agent.config file indeed already contained the API key: mmsApiKey= xxxxxxxxxxxxxxx

To make further sure, I went to the Administration tab in the MMS management console, Group settings, and checked whether the API key configured matched with the API key stated there. It did match.

So I think that can't be the culprit, the API key is OK.

Next, from the installation FAQ, I took it that often the hostnames can't be resolved. So I pulled up my hosts and hostnames files from the AWS EC2 instance. But first, I entered "hostname" in the terminal window on the instance to see what it would return. It returned: ip-xxx-xx-x-xxx

The command "hostname -f" returnes exactly the same: ip-xxx-xx-x-xxx

So, that is obviously the hostname of the instance.

I installed my host with host type: standalone, internal Hostname as: ip-xxx-xx-x-xxx (which I just found) and Port 27017. Next, hit "verify host".

The MMS management console starts contacting the host. It answers with: Unable to detect host, and it refers to he monitoring agent logs. Looking at those logs reveils the following error:

Failure dialing host after 961us. Skipping all `MongoDB Blocking` tasks. Failure on MongoDB dial to `ip-xxx-xx-x-xxx:27017`. Err: `no reachable servers` at monitoring-agent/components/conf.go:152 at monitoring-agent/components/bus.go:362 at monitoring-agent/components/bus.go:391 at monitoring-agent/components/bus.go:337 at pkg/runtime/proc.c:1445

So that does not look good. No reachable server there. But at the same time,  when I look at my agent data in the MMS management console in the Administration tab, under Agents, it shows the Agent is active at the very same hostname: ip-xxx-xx-x-xxx

Since both the mongod process AND the agent are on the same server, it astonishes me a bit that it seems to CAN find the Agent, but it CAN NOT find the mongod instance. How come?

In between I looked up the contents of my hosts file. it's in tthe /etc directory, and the command "cat hosts" shows the following contents:

127.0.0.1 localhost

xxx.xx.x.xxx ip-xxx-xx-x-xx

Same hostname as I entered. 

When I look back at the management console, in the Servers tab, it shows that the monitoring agent is active, but the server at port 27017 is not.

However, when I go back to the terminal window (where I am ssh- ed into the instance) and call "mongo --port 27017" it responds immediately with the mongodb shell, as it should, and replies that I am connected to 127.0.0.1:27017/test

So this totally beats me. I have no clue what is happening here.

I don't know what to do next.

Could you please try running the mongo shell but connecting to ip-xx-xxx-xx-x:27017 directly, a la:

mongo ip-xx-xxx-xx-x:27017

This is what the Monitoring Agent itself does, so if it's having a problem with that, then I'd suggest looking into correcting the hostnames. Also, when you are connecting, please give me the output of db.hostInfo() to see how the mongoD refers to itself.

Finally, if you're still stuck, please try restarting the Monitoring Agent, sometimes bad logins are cached. We are working to eliminate this issue, but it can still be a problem. Please be aware that the agent may take up to 5 minutes after restart to become Active again.

{ name     : "Peter C. Gravelle",
  title    : "MMS Technical Account Manager",
  location : "New York, NY"}










Loggin in to the mongo shell with the ip as suggested resulted in this:

MongoDB shell version: 2.6.5
connecting to: ip-xxx-xx-x-xxx:27017/test
2014-11-26T20:57:48.082+0000 warning: Failed to connect to xxx.xx.x.xxx:27017, reason: errno:111 Connection refused
2014-11-26T20:57:48.082+0000 Error: couldn't connect to server ip-xxx-xx-x-xxx:27017 (xxx.xx.x.xxx), connection attempt failed at src/mongo/shell/mongo.js:148
exception: connect failed

Next, I logged in using the command: mongo --port 27017 and issued the command db.hostInfo() in the shell as requested. Result:

{

"system" : {

"currentTime" : ISODate("2014-11-26T21:02:56.922Z"),

"hostname" : "ip-xxx-xx-x-xxx",

"cpuAddrSize" : 64,

"memSizeMB" : 992,

"numCores" : 1,

"cpuArch" : "x86_64",

"numaEnabled" : false

},

"os" : {

"type" : "Linux",

"name" : "Ubuntu",

"version" : "14.04"

},

"extra" : {

"versionString" : "Linux version 3.13.0-39-generic (buildd@toyol) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #66-Ubuntu SMP Tue Oct 28 13:30:27 UTC 2014",

"libcVersion" : "2.19",

"versionSignature" : "Ubuntu 3.13.0-39.66-generic 3.13.11.8",

"kernelVersion" : "3.13.0-39-generic",

"cpuFrequencyMHz" : "2500.036",

"cpuFeatures" : "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm xsaveopt fsgsbase smep erms",

"pageSize" : NumberLong(4096),

"numPages" : 254061,

"maxOpenFiles" : 64000

},

"ok" : 1

}

Thanks for that update. I suspect the issue may be a bindIp setting. Check your config file and command line arguments. In your previous note you said you assigned the hostname to the IP assigned by AWS. You may have better results adding the "ip-xxxxxx" domain name to the same line as localhost is on.

Sorry Peter, I don't quite get that.... 

• What config file exactly do you need me to check, and on/for what exactly?
• How do you want me to adjust the hosts file in /etc ?
  do you mean it should become like this: 127.0.0.1 localhost ip-xxx-xx-x-xx ?

By the way, I just stopped and started again the agent, to no avail.

Apologies, I was referring to the mongod config file. If you are using bindIp, try disabling it, restarting your mongod, and see if that allows you to connect via the hostname.

Thank you for staying on this, and trying to help me out. I appreciate that!

Due to the time differences, I had to back off from this problem for a few hours. But, I am back at it :-) trying to get this to work.

I do not understand why this is proving to be that difficult, and it does not look like it we come any closer to a solution here.

This is a standard, basic AWS EC2 instance. I did nothing special to its setup. So why does this not work right of the bat?

• What should I change to the "mongod config file" ? You say I need to check the mongod config file. But what to check? What to change? What am I looking for here?
• I have no clue whether AWS uses "bindIp", or where to find it or how to use it or disable it (if that is a viable option to begin with, I mean, when it is used, it probably is for a reason?)
• How do you want me to adjust the hosts file in /etc ? 

I would appreciate some specific instructions on what to do next, since at this point, I am stuck again and it still does not work.

ISSUE RESOLVED

OK. So I struggled a day further with resolving this issue. The comment of Peter Gravelle about bindIp and the mongod config file made me think. Although at first I had no idea what he meant or wanted me to do, I thought it a good idea whilst waiting for his "turkey to sink in" to explore a bit.

So I pulled up the mongod config file to be found at etc/mongod.conf

And there it actually was: the line bind_ip=127.0.0.1 was actually added to the config file, which sort of closes the mongod instance from all other trafic then through the local server.....

So that is in the end why the mms agent could not get to the mongod instance. It was not with the security groups, and it was not a hostname resolving issue.

It was simple the mongod instance itself, setup to refuse connection from anywhere else then from the local server.

This setup actually was created by the setup software for the application living on top of the mongod instance. This app does communicate with the mongod instance itself, and does not need anyone from outside to work that mongod instance :-) So probably, on setup of the application, it adjusted the config file by adding this line. 

So what I did was simply commenting it out, and the MMS monitoring system started working like a charm at once: the MMS Agent got connection with the mongod instance, and relayed its data to the management console as it should.

So, in my case the resolution proved to be the editing of the mongod.conf file by commenting out the line bind_ip=127.0.0.1

Thanks for effort taken and help offered to those who did !

I'm so glad to hear you got it sorted! Thanks for waiting for my turkey – it was delicious :-D