MongoDB's move to get MMS customers into cluster management a mistake or even really necessary?

Looking at the group, it seems MMS is having some issues. My post is about what I think is a major business issue actually. I mentioned this in another thread, once I realized the implications of the MMS changes and it has again been brought up recently and now I'd like to start its own thread.

I think MongoDB made a mistake with the recent change to "insist" a cluster be managed by MMS to also get the monitoring and backup services. Mongo basically locked themselves into the smallest of customer niches they have, which is people wanting managed Mongo clusters. There are people who just want the monitoring and/ or just the backup functionality. Now, as a customer, you can't have either, unless you allow cluster management too. That is simply a big mistake IMHO.

For instance, we are planning to offer customers a PaaS with Mongo, as the main datastore. We will be running/ managing their Mongo instance for them. What we won't be doing is offering them off-location storage from the datacenter of their database (at least not at first and if it had worked with MMS, probably not ever). Had Mongo kept the different MMS services as separate and available choices, we could have nicely said, "Go to MMS to get off-location storage of your data!", and we could have worked as partners. Now we can't make this recommendation. As I see it, you've actually manouvered yourselves out of a lot of possible business. It would have been a cool partnership too, because we are learning, although cloud computing has been around for some time now, people not really involved with the technology still are very unsure about its security and their main concern is about their data 1. not being "theirs" and 2. it getting zapped by circumstances beyond their control. With an MMS partnership for off-location storage, we could have given our customers real peace of mind, for those willing to pay for it.

Anyhow, is there really a need to get customers into the cluster management to give them any other part of the MMS services? I can understand it lowers issues with the other two services, but the monitoring and backup services seemed to be working well before the changes, weren't they?

I fully agree with you Scott. It was a not well thought out move. As you already stated, not every user WANTS to get their mongo process deployed by MMS. That's true.

However, I want to add just another reason besides "not wanting". 

I use the Meteor platform in most projects, and Meteor installs a Mongod instance of its own with a communication layer above it named "MiniMongo" that communicates between Meteor and the Mongod instance.

Which means, that the Mongod instance is ALREADY DEPLOYED and does not need to be deployed again.

Commercially, I don't see the justification for limiting your customer group either. MMS is a service that should basically be available to ALL Mongodb users, and not just to a niche group in it.

That's my opinion.

Besides, it is extremely easy to deploy these instances, so one could also ask for the need to do it with MMS, but that is besides the point. I can imagine some like graphical approach, especially those coming from the Windows side of technology. The Linux adepts won't care for that much I guess.

I am advocating to get MMS management/monitoring back for all mongod instances (deployed by MMS or deployed independently!) and leaving deployment as an option in it to use, or not.

To add to the discussion.

We have a fair amount of mongodb installed in our systems, and we would by principle never allow a 3rd party system anywhere near our systems.  We install our own instances and clusters. Our network security folks would shut us down if we allowed a third party system to do reads and writes inside our firewall. Especialy on systems that contain company confidential data and mission critical information.

Maybe on cloud systems it may be allowed, but on private colocs , no chance. 

Thanks M_R_O and Tim.

Tim. I think Mongo offers an on-premise version of MMS with their Enterprise version, so, that could be a solution for your purposes, possibly?

They do. But also in that case, the question remains where the monitoring agent is sending its info to. If that goes from INSIDE the network (where the agent lives and gathers its data) to OUTSIDE the network (if the Monitoring Service runs on the mongodb.network) then it would still be in violation of the internal security protocols since who guarantees how what data exactly is gathered and sent out?

If the monitoring service runs locally, then indeed no data leaves the network. I am not sure what is the case, or what exact policies exist on Tim's company network.

I am also not sure, but I would think on-premise means on-premise, for exactly the reasons Tim mentioned and why Mongo has it in their Enterprise offering. Corporate data rules are strict in some businesses and to enter this very lucrative market, Mongo would need a "closed network" system.

The on-prem version of MMS that includes automation, slated for release with MongoDB 2.8, won't send any data outside your network. It will also allow a la carte installation of monitoring or backup without running automation. Specifically, you can run

• monitoring alone
• monitoring and backup
• automation, which monitors and optionally backs up

Scott, I was involved in the decisions involving restricting the way MMS (cloud) allows its parts to be used. The goal was to simplify MMS for the vast majority of users and give it a gestalt that was centered around running MongoDB at scale. By centering the experience around deploying and running MongoDB, the user no longer needs to understand how to configure and run backup or monitoring, because automation sets those parts up.

The tradeoff is between ease of use and flexibility. And while it might seem that we limited the addressable market for MMS by neglecting all the folks who want just monitoring or backup, we potentially increase the overall appeal by making the automation pieces easier to grok.

MMS already bites off a lot by allowing you to deploy MongoDB on the infrastructure of your choice (AWS, your own datacenter or laptop), allowing additional degrees of freedom seemed to overcomplicate it. 

Time will tell whether we made the right choice around MMS, but so far the initial reception has been good. There is certainly some confusion around there being both "classic" MMS groups, that allow monitoring and backup but no automation, and new MMS groups, which bundle the automation in. That should go down over time.

Ultimately there is always the question of what you want a product to be. You don't get much room to position a product in a user's mind. I have a preference for simplicity.

That said, our on-prem version retains the complexity because our belief was that in that market, enterprise customers value flexibility over all else. 

Thanks for the reply Andrew.

You said, "The goal was to simplify MMS for the vast majority of users"

I must retort. What is it you wanted to simplify and for whom? It wasn't complicated to work with MMS to begin with. Was it? Really? I mean, adding a daemon to a server and getting the daemon and MMS to connect was, for a vast majority of the users, simple. 

So what have you simplified? 

I think what Mongo has done is a great job to make deployment and maintenance of a Mongo server cluster easier, but not the use of MMS. You've actually made it impossible to use, for those who only want monitoring or back-up. I am absolutely certain, the people who want automated cluster management are very happy. Those who don't, and only want monitoring or back-up, are left in the dark.

At any rate. I personally think the decision needs rethinking. There are a whole lot of people out there experimenting with Mongo, including myself, who would like to visualize their experiment's usage of Mongo for little ( to no :-)) cost. You could see that with the (I believe it was) 40+K monitoring users you've attracted. That functionality can and I think should be the open door to MMS to get new customers "in". You might even be able to charge a small but very fair fee to cover the usage costs for monitoring. That means making some money and still a wide open door to let people in to using even more services in MMS and earning more money later. That is usually the beauty of SaaS. Low initial costs for new customers.

And I can't imagine it would be hard to do, as the "old" users can still use only monitoring or back-up. The solution is already there. Isn't it?

Does MMS Cloud flexibility really mean too much complexity? I don't think so. :-) So why deny your cloud customers flexibility too and yourselves a bunch of possible business?

Reading all of this, I sort of got a feeling of fear would I be in that situation to not know anything on how to deploy a mongod instance, or a sharded cluster or a replica set.

Would I for all of that only be able to depend on a piece of software without having the actual knowledge itself..... brr.

That's not how most developers roll. I think we WANT to, no, NEED TO, understand what is happening and how it works COMPLETELY.

How can you otherwise depend on such an important part of a production process where your livelihood depends on, if you don't even have the knowledge about how to do it manually?

You would put way too much weight on a simple monitoring/deployment tool, if you ask me. A tool should be not more than that: a tool.

As soon as the tool takes away the knowledge of the underlying processes, in my humble opinion, it becomes dangerous instead of helpful.

Deploying a cluster by hand gives a way better "feeling" on how it is working together etc etc, and the Mongodb University is a great place to gather that information/knowledge/skills if you don't have it already.

I would prefer that above a tool doing it "automatically" in the background any time.

This is why I prefer MMS as a monitoring tool only, and would rather not use any automatic deployment. It isn't that difficult to begin with. 

Getting the mms tool to actually work proves to be a tad more painful to me :-) :-)

Ah, and that is the current downside for the acceptance of SaaS. People wanting the control and not trusting the service completely to do what is right all the time. As services grow and show reliability and trustworthiness, this fear will die down for sure. But, it is an issue for growth now. Most definitely.

Scott, I don't think it has much to do with "trust". If you own a process, and depend on it, you need to be "in control" of that process and you need to know how it works precisely. At least, that's my opinion.

Um, sure. It has everything to do with trust. In order to relinquish the ownership of control of any process to someone else, first you must be willing to relinquish that control, and to be willing, there must be a degree of trust that the new owner will do just as good or better job with that process or processes. Usually, you need trust AND a good portion of convenience too. Theoretically everyone could have hosted their own websites from home. But, that would be seriously inconvenient, so hosting companies popped up. Now you have cloud computing and automation is popping up and as SaaS. Why? Because it is much more convenient and simplifies a lot of work.   

Then, there are some people, who simply won't or can't relinquish the ownership of the processes they control. For them, SaaS simply isn't an option. Tim is an example.

> Would I for all of that only be able to depend on a piece of software
> without having the actual knowledge itself..... brr.
>
> That's not how most developers roll. I think we WANT to, no, NEED TO,
> understand what is happening and how it works COMPLETELY.
>
> How can you otherwise depend on such an important part of a production
> process where your livelihood depends on, if you don't even have the
> knowledge about how to do it manually?I wish I could agree with you, but I've got experience with thousands,
if not tens of thousands of developers (just look over on
StackOverflow) who have no idea what's under the hood of whatever they
are running and many of them are asking questions of the form "just
tell me what button to press to make it go" rather than "tell me how
it works so I can figure it out".

So while in the perfect world, engineers would actually want to
understand how tools they use work, in the real world, this is about
as far from that as I never would have wanted to imagine.

I just noticed the on-premise version of MMS is free to download here: http://www.mongodb.com/subscription/downloads/mms

Does that mean we could create our own monitoring service for our own private use?

"who have no idea what's under the hood of whatever they
are running and many of them are asking questions of the form "just
tell me what button to press to make it go" rather than "tell me how
it works so I can figure it out"."That is unfortaunatly the topic of SO in general, it is supposed to a straight to the point Q&A site, which means over explanatory answers are sometimes even hounded out for not being straight enough.

It is a question with an a to b answer, re-enforced by its meta.

I dislike Stack Overflow. Forums and even Google Groups are better than SO.

Yes, you have slackers anywhere. That doesn't mean that they are the standard to uphold, or the max to reach for.

A lot of those developers get hunted forward by managers who only want results, so who can blame them for making that their sole task?

The consequences of that behavior are picked up by the support engineers who can mop up the sad results of that policy.

An integrated approach, with a bit of helicopter view, leads to a more comprehensive look at things.

The road ahead is not just the 10 centimeters in front of your feet.

> I just noticed the on-premise version of MMS is free to download here:
> http://www.mongodb.com/subscription/downloads/mmsThen I'm sure you saw this sentence:

Please note that for those who are not yet MongoDB customers, download and use constitutes acceptance of the Development, Test and Evaluation agreement.

> Does that mean we could create our own monitoring service for our own
> private use?

Certainly not, as that would violate the agreement's explicit statement in item 1. that the software is for 

"development, testing, and evaluation purposes only. In no event shall Customer be entitled to use the Technology in a production or commercial environment under this Agreement."

IANAL, but this seems to be in pretty plain English to me.

The natural evolution of technological progress is that you abstract away the details of an underlying layer in order to allow a person to create something without completely understanding its underpinnings. There are many python programmers out there who could not build a processor chip, a JIT compiler or garbage collection. And yet those programmers are very productive and add huge value to the world. 

I believe that it would be preferable if users could build large sharded MongoDB configurations without completely understanding how to do construct such a configuration "manually." Even better, I believe it would desirable if you could just point me to a TCP endpoint where I could store and retrieve documents and not understand what a replica set or shard is. That's a long way away.

The secret, I think, is that whatever tools or technology you are talking about needs to be good enough that I don't need to understand how it works. Making it nearly impossible for me to service my own car is fine with me if I can reliably drive from point A to point B w/o becoming a mechanic. But if my car is going to require me to fix it on the road, it better be serviceable with simple intuitive tools.

Today's MMS is a step in the direction of allowing user to worry less about the configuration of large systems, but if you see the way Automation was included, it's more of a wizard for wizards, not something that turns a mortal into a wizard. We will get there.

There is no doubt that we removed utility from MMS for some class of users who only want monitoring or backup. You mentioned that we have 40,000 monitoring users. Two thoughts there. First, we did not take away anything from those users, and second, there are millions of MongoDB deployments. 

Focus is essential to getting anything done in this world, whether it be personally or as a corporation. We all know this, but we all allow ourselves to be distracted. The larger problem we are working on solving is enabling more people to run MongoDB at scale. Having the system monitor and backup only what it deploys makes the install simpler for the user, abstracts away the details of monitoring and backup and allows us to better use our own finite resources. 

I understand you might not agree with the decision. Regardless, we truly appreciate you taking the time to share your feedback.

Ok, sorry it wasn't clear on my part. Private use to me is testing.:) 

Yup. We'll have to agree to disagree Andrew. Still, thank you for taking time to answer.

Just noting here another thread with some more customers concerned about the decision to have MMS be a pure deployment management system first.

https://groups.google.com/forum/#!topic/mongodb-user/wQFFAAo_xGA