Use mongoose or native Nodejs MongoDB driver - that's the question

To mongoose or not mongoose - that's the question  coming all the time and time again among Node/MEAN stack coders.
Usually I don't take sides on this kind of arguments because discussion can become real hot and quite inflammatory leading to ad-hominem attacks, but sometimes I am asked and have to provide my professional opinion. So when I was asked again recently this Shakespearean question I answered it with a short Shakespearean answer:  "thou shalt avoid mongoose".
Surprisingly the discussion didn't go any further, may be I was convincing or probably my opinion was rejected silently.
Any way I feel I must give a more detailed answer (though not complete), I owe this to the mongoDB  ecosystem and to mongoose maintainers who have done a fantastic job writing/maintaining mongoose code base.
The complete answer is it all depends and there are a lot of factors like:

• learning curve:
  mongoose can save you from the steep learning curve of the native driver, also will protect you from making critical design mistakes like opening/closing a mongoDB connection on each and every request served instead of reusing a connection from a connection pool - a common bad practice. 
• use case:
  things are different if you are developing a small POV app, then taking shortcuts offered by mongoose can be beneficial but if you are working on a large scale production system mongoose can be an obstacle you have to fight all the time in order to improve efficiency.
• point in time:
  there were times when mongoose data validation was really helpful and it was a real plus,  not not so match anymore as currently mongoDB supports validation natively 
• support/features:
  As mongoose is not directly supported by mongoDB you can expect it to be a step behind relative new features, bug fixing etc.
• ODM:If you feel you absolutely need the ODM layer provided by mongoose then go for it.

TL;DR: mongoose is good in making sensible design decisions for you that are good enough most of the time and certainly much better than what a newbie coder could possibly make, but if you want to get the extra mile in efficiency/features/support etc... you better use native driver.

PS: Let me quote:

• guys who wrapped a thin layer around official mongo js driver:
  "Mongoose and many other ODMs are ridden with bugs (no offense) when you push them beyond the basics"
• @matteocollina who managed to summarize this argument in < 140 chars.

MongoDB v 3.2 + silently breaks backward compatibility affecting behavior of capped collections

Capped collections are collection with some very special characteristics,  although these are used internally by MongoDB for replication (oplog) are also offered as a feature for general use, their limitations and  advantages are well documented.
Because of those advantages developers are using capped collections widely for many purposes, and this usage it is not a hack or undocumented feature exploitation because capped collections are well documented and a well known feature of mongoDB.
I have seen capped collections used as :

1. FIFO buffers
2. Trigger-like mechanisms
   (as a matter of fact a well known framework is based on this feature) 
3. PubSub architectures
4. Message queues
   Yes you can implement a message queue with mongoDB, although it will not be as fast as ZeroMQ or some other tools, it has some attractive benefits as persistence, built in redundancy, limiting stack complexity etc. (My implementation of a PubSub/message queue)  

Except that you can't delete a document in a capped collection the other major limitation used to be that a document couldn't grow up in size as a result of an update. This is well understood by the developers and could be addressed by various techniques as:
- pre-filing field(s) with dummy values
- un-setting a field during an update operation to make space for a new field(s) etc.

Now all of a sudden MongoDB from version  3.2.0-rc0 + decided to introduce one more limitation "the document can't shrink in size" as a result of an update.
This breaks backward compatibility of a well published and widely used feature and although I understand the technical reasons behind this decision as described here, still I can't accept that such a decision can be taken so lighthearted without consulting with the developers/ecosystem. If you follow the discussion in the above ticket in Jira you get the impression that the only thing that they really cared about is how this can possibly affect their internal use of the feature and since they found no side-effect they went ahead with this.
To keep adding salt to injury the breaking change is not yet published in the manual as of today, neither I can find any reference in change logs.
So it is left to developers to find it out the hard way when their code breaks after an update.
I filled a ticket, then I realized a related ticket already exists but there is no action taken yet.
Of course those tickets deal with the documentation only since the code breaking change is there and we have to live with it.
Too bad many developers didn't realize this change was planed, that's why a compatibility breaking change should get as much publicity as possible so that developers don't get caught of guard, since is impossible for a developer to follow each and every ticket in Jira.

This is a sad story that  I hope will not be repeated in future since some of the things that made mongoDB so successful IMO are:
a) developers are not caught up by surprises
b) high quality of manuals

MongoDB 3.2: Now Powered by PostgreSQL ?

Having said that MongoDB is listening to developers/ecosystem some months ago, today I have read an article MongoDB 3.2: Now Powered by PostgreSQL by @jdegoes that contradicts my own experience.

I will not categorize this article as one more mongophobic article that makes headlines from time to time here and there  like: 

• MongoDB has 'architectural' problems and has done nothing to improve 
• 600TB MongoDB Database 'accidentally' exposed on the Internet

Although I understand writer has some personal interest as he has invested in mongoDB Analytics, I have to admit that he is making a case that technically looks very sound to me "flattening out the data and using a different database to execute the SQL" is not the way to go for MongoDB BI solutions.

I also agree with much of his arguments when he tries to describe of what is going wrong with mongoDB's ecosystem.

Still I am not as pessimistic as the writer and I hope that:

• a) This connector to BI tools is only a temporary quick fix solution and there are better tools coming in the pipeline.
• b) MongoDB will listen to his arguments and comes with a revised policy regarding its partners and the ecosystem at large.
  The story of how $lookup ended up been part of community edition makes me believe that mongoDB can do that.

MongoDB is listening to developers/ecosystem

Today MongoDB 3.1.7 is released and shell includes the new CRUD API.
I am happy for this feature to be implemented and get into production so fast and feel really glad that I am the one who has triggered the introduction of this new API.
It all started 5 months ago when PyMongo 3.0 was introduced and A. Jesse Jiryu Davis the coauthor of pymongo wrote about new CRUD API in his blog where I posted a comment complaining that this is a step backword unless it is implemented in the shell API as well.
Jesse engulfed the idea opened a ticket at mongoDB's Jira then things started rolling. Today I was reading again his blog and was excited to realise suggestion was in production. I am also thankful to Jesse for his kind words and attribution and feel obliged to repeat those here:

The official announcement focuses on bug fixes, but I'm much more excited about a new feature: the mongo shell includes the new CRUD API! In addition to the old insert, update, and remove, the shell now supports insertMany, replaceOne, and a variety of other new methods. Why do I care about this, and why should you? MongoDB's next-generation drivers, released this spring, include the new API for CRUD operations, but the shell did not initially follow suit. My reader Nick Milon commented that this is a step in the wrong direction: drivers are now less consistent with the shell. He pointed out, "developers switch more often between a driver and shell than drivers in different programming languages." So I proposed the feature, Christian Kvalheim coded it, and Kay Kim is updating the user's manual. It's satisfying when a stranger's suggestion is so obviously right that we hurry to implement it.
......
I'm so glad we took the time to implement the new CRUD API in the shell. It was a big effort building, testing, and documenting it—the diff for the initial patch alone is frightening—but it's well worth it to give the next generation of developers a consistent experience when they first learn MongoDB. Thanks again to Nick Milon for giving us the nudge.

In an other occasion I requested a feature this time from pymongo's team a few days ago, that was a trivial one and easy to implement that was to name the threads that pymongo is creating for debugging purposes, of course I could have done it myself and request a pull from github but it involved naming conventions for which I was not sure, so I prefered to post a feature request in jira, Bernie Hackett responded "it seems a good idea" and to my suprise I saw this implemented in next release few days later.
What this story tells us developers is that we can request for features/fixes and can expect to see those implemented in reasonable time even if it is a major effort provided that:

• our requests are reasonable and technically sound.
• we document those properly.
• we use proper channels to communicate.
• the company/organization has a culture/history of listening to developers/ecosystem and those 2 examples proove that mongoDB is one of those.