Drivers and 'the wire'
From the perspective of the driver developer a MongoDB driver:
- Marshalls data from the language's native types into the format the MongoDB server requires (== a BSON payload proceeded by a few classically simple network fields in each packet's header area).
- Sends and receives that info, keeping track of which reply from a server matches which request.
- When using a connection pool it also keeps a track of which thread the requests were sent from
- Goes to error handling when there are network interruptions like abrupt socket closure and other TCP casuality situations
- Implements a lot of detail in the 'Meta' API driver specification for server discovery and monitoring ("SDAM") and server selection.
From the application developer's perspective the MongoDB driver:
- Presents the database as an object you can push data in and pull data out of.
- The API provided is idiomatic for your language. E.g. where Java programmers run a find() method on a collection object, C driver users run a mongoc_collection_find() function that takes a mongoc_collection_t* pointer argument, etc.
To look at it from another side this is what the driver API doesn't do:
- Involve the application programmer in maintaining the TCP socket connections.
- Involve the application programmer in determining which remote servers are the current primaries (i.e. the one that the writes happen on first)
- Expose network packet data in the wire protocol format
Apart from the fact that you open a connection, and there can be exceptions thrown when a server crashes or the network is disconnected, there is limited expression in the API that the database is on a remote server.
There are no network-conscious concepts the user must engage with such as 'queue this request', 'pop reply off incoming message stack', etc.
Many drivers; one Wire Protocol
Regardless of which driver you are using, at the Wire Protocol layer they are all the same fundamentally. If they are contemporary versions there's a good chance the BSON payload in each Wire protocol packet is identical excluding ephemeral fields like a timestamps.
The format of data in MongoDB Wire Protocol requests and responses is relatively simple, but it is a binary one and is far from being human-readable. The below comes from TCP payloads captured using tcpdump, manually unwrapped using command line tools od and bsondump according to the info in the MongoDB wire protocol documentation.
| Example find in various APIs |
|
MongoDB wire packet |
|
mongod code |
mongo shell db.foo.find({"x": 99};
PyMongo db.foo.find({"x": 99})
Java db.getCollection("foo").find(eq("x", 99))
PHP $db->foo->find(['x' => 99]);
Ruby client[:foo].find(x: 99) |
→ |
OP_MSG
length=180;requestID=0x1b73a9;responseTo=0;opCode=2013(=OP_MSG type)
flags=0x00.0x00
section 1/1 = {
"find":"foo",
"filter":{"x":99.0},
"$clusterTime":{ ... }},
"signature":{ ... },
"$db":"test"
} |
→ |
mongo::FindCmd::run |
| (A cursor object with first batch results) |
← |
OP_MSG (as a reply)
length=180;requestID=0xb5a;responseTo=0x1b73a9;opCode=2013(=OP_MSG type)
flags=0x00.0x00
section 1/1 = {
"cursor":{
"id":{"$numberLong":"0"},
"ns":"test.foo",
"firstBatch":[
{"_id":ObjectId("5b3433ad88d64ee7afb5dc80"), "x":99.0,"order_cust_id":"AF4R2109"}
]
},
"ok":1.0,
"operationTime":{ ... },
"$clusterTime":{ ... },
"signature":{ ... },
"keyId":{"$numberLong":"0"}}}
} |
← |
↲ |
OP_QUERY and early generations
An optional detour for those who knew the original Wire protocol messages (OP_QUERY, OP_INSERT, etc.) and are interested in what traffic looked like with these.
Expand me...
The above is latest-and-greatest OP_MSG format. At time of writing only the 3.6+ mongo shell and dev-branch drivers would be using it. In truth most driver versions are still being shoe-horned into the legacy OP_QUERY message type.
Per its name OP_QUERY was meant to only be for queries, but was repurposed for mostly any type of request message. In its network packet fields it included a "fullCollectionName" field because queries always need a a db and collection name scope). But there are commands that don't have a collection scope (eg. replicaSetGetStatus, createUser) but don't have a dedicated wire protocol message type either. How to send them? The workaround for those cases was that "$cmd" was used as a dummy collection name at the end of the "fullCollectionName" field. This workaround became so standard that it is even set this way for commands such as find which do need a collection scope. You can see in the example below that the collection name "foo" has moved inside the BSON and is absent outside.
| Legacy wire packet examples |
OP_QUERY
length=215;requestId=0x6633483;responseTo=0;opCode=2004(=OP_QUERY type)
fullCollectionName="test.$cmd" //N.b. the dummy "$cmd" collection name
numberToSkip=0;numberToReturn=0xffff
document = {
"find":"foo",
"filter":{"x":99},
"lsid":{ ... },
"$clusterTime":{ ... },
"signature":{ ... },
"keyId":{"$numberLong":"0"}}}
} |
OP_REPLY
length=301;requestId=0xbb8;responseTo=0x6633483;opCode=1(=OP_REPLY type)
responseFlags=0x08(=AwaitCapable)
cursorID=0 //important for getMore cmds that follow, if any;
startingFrom=0;numberReturned=1
document = {
"cursor"
{
"firstBatch":[
{"_id":ObjectId("5b3433ad88d64ee7afb5dc80"), "x":99.0,"order_cust_id":"AF4R2109"}
],
"id":{"$numberLong":"0"},
"ns":"test.foo"},
"ok":1.0,
"operationTime":{ ... },
"signature":{ .. }
}
} |
My way of looking at is:
- In the beginning there were just collection editing or reading commands (query, insert, update, delete) and four wire packet types for those, plus a reply message type. The db+collection namespace was put in a network field, outside the BSON payload document.
- Soon there many more command types that the database server accepted. A generic command wire packet format was needed. The existing drivers (that needed to be supported for some time) started using OP_QUERY overloaded for this purpose.
- A generic command wire packet type OP_COMMAND was invented! And used by mongo shell v3.4(?) and between nodes in clusters and replica sets. But it didn't go mainstream.
- Instead the OP_MSG type has become the new standard, to be used by 4.2? era drivers. Neither the collection name or database name is in the network header fields - they'll be in "ns" (namespace) inside the BSON payload instead.
Database command type
You might have noticed that there's no primary / headlined / specially labeled value in the BSON command object that indicates what sort of command the client is sending.
You might be wondering 'Does the server run through a list of key-value pairs in fixed order until it gets a match?' (E.g. if (commandMessage.hasKey("find") then --> FindCmd:run(), else if commandMessage.hasKey("update") -> UpdateCmd::run(), etc. ....?).
Nope, a simpler mechanism is used. From util/net/op_msg.h:
StringData getCommandName() const {
return body.firstElementFieldName();
}
Take the key name from the first key-value pair. End of function.
A lesson from this is that order in BSON can matter (at least to MongoDB). Important for driver developers, but not application programmers as the driver API will take care of this point for you.
What it looks like to the programmer
I don't want to re-invent the documentation wheel for this part. MongoDB's official documentation tutorials are good and cover many language samples in one page. Some links for a couple of types of operations:
- Document insert example (Python, Java, Node.js, PHP, C#, Perl, Ruby, Scala)
- Query example (Python, Java, Node.js, PHP, C#, Perl, Ruby, Scala)