Category: mongoengine

MongoEngine – Query Operators ”; Previous Next In addition to = operator to check equality, the following logical operators are defined in MongoEngine. ne not equal to lt less than lte less than or equal to gt greater than gte greater than or equal to not negate a standard check, may be used before other operators in value is in list nin value is not in list mod value % x == y, where x and y are two provided values all every item in list of values provided is in array size the size of the array is exists value for field exists These operators must be attached to field name with double underscore __. To use greater than (gt) operator, use the following format − #using greater than operator for product in products.objects(price__gt=10000): print (”ID:”,product.ProductID, ”Name:”,product.Name, ”Price:”,product.price) Output ID: 1 Name: Laptop Price: 25000 ID: 2 Name: TV Price: 50000 ID: 5 Name: Printer Price: 12500 The in operator is like Python’s in operator. For name of product matching with names in list, the following code is used − for product in products.objects(Name__in=[”TV”, ”Printer”]): print (”ID:”,product.ProductID, ”Name:”,product.Name, ”Price:”,product.price) Output ID: 2 Name: TV Price: 50000 ID: 5 Name: Printer Price: 12500 You can use following operators as shortcut for regex expressions for applying filter to queries − exact string field exactly matches value iexact string field exactly matches value (case insensitive) contains string field contains value icontains string field contains value (case insensitive) startswith string field starts with value istartswith string field starts with value (case insensitive) endswith string field ends with value iendswith string field ends with value (case insensitive) match performs an $elemMatch so you can match an entire document within an array For example, the following code prints product details for name containing ‘o’ in name − for product in products.objects(Name__contains=”o”): print (”ID:”,product.ProductID, ”Name:”,product.Name, ”Price:”,product.price) Output ID: 1 Name: Laptop Price: 25000 ID: 3 Name: Router Price: 2000 In another example of string query, the following code displays name ending with ‘er’− for product in products.objects(Name__endswith=”er”): print (”ID:”,product.ProductID, ”Name:”,product.Name, ”Price:”,product.price) Output ID: 3 Name: Router Price: 2000 ID: 4 Name: Scanner Price: 5000 ID: 5 Name: Printer Price: 12500 Print Page Previous Next Advertisements ”;

MongoEngine – Sorting ”; Previous Next QuerySet’s order_by() function is used to obtain the query result in a sorted manner. The usage is as follows − Qset.order_by(‘fieldname’) By default, the sort order is ascending. For descending order, attach – sign to name of field. For example, to get price wise list in ascending order − from mongoengine import * con=connect(”newdb”) class products (Document): ProductID=IntField(required=True) company=StringField() Name=StringField() price=IntField() for product in products.objects.order_by(”price”): print (“Name:{} company:{} price:{}”.format(product.Name, product.company, product.price)) Output Name:Router company:Iball price:2000 Name:Scanner company:Cannon price:5000 Name:Printer company:Cannon price:12500 Name:Laptop company:Acer price:25000 Name:TV company:Philips price:31000 Name:Laptop company:Dell price:45000 Name:TV company:Samsung price:50000 Following code will get the list in descending order of name − for product in products.objects.order_by(”-Name”): print (“Name:{} company:{} price:{}”.format(product.Name, product.company, product.price)) Output Name:TV company:Samsung price:50000 Name:TV company:Philips price:31000 Name:Scanner company:Cannon price:5000 Name:Router company:Iball price:2000 Name:Printer company:Cannon price:12500 Name:Laptop company:Acer price:25000 Name:Laptop company:Dell price:45000 You can also get sorting done on multiple fields. This code will get you companywise, pricelist in ascending order. for product in products.objects.order_by(”company”,”price”): print (“Name:{} company:{} price:{}”.format(product.Name, product.company, product.price)) Output Name:Laptop company:Acer price:25000 Name:Scanner company:Cannon price:5000 Name:Printer company:Cannon price:12500 Name:Laptop company:Dell price:45000 Name:Router company:Iball price:2000 Name:TV company:Philips price:31000 Name:TV company:Samsung price:50000 Print Page Previous Next Advertisements ”;

MongoEngine – Indexes ”; Previous Next An indexed collection results in faster processing of queries. By default, every collection is automatically indexed on _id field. In addition, you can create index on one or more fields. Using Compass, we can build index very easily. Click on CREATE INDEX button on Indexes tab as shown in figure below− A dialog box appears as shown. Choose name of index, field on which to index, order of index (ascending or descending) and other options. While using MongoEngine, indexes are created by specifying ‘indexes’ key in meta dictionary of definition of Document class. Value of indexes property is a list of fields. In the following example, we ask documents in student collection be indexed according to name field. from mongoengine import * con=connect(”mydata”) class student(Document): name=StringField(required=True) course=StringField() meta = {”indexes”:[”name”]} s1=student() s1.name=”Avinash” s1.course=”DataScience” s1.save() s2=student() s2.name=”Anita” s2.course=”WebDesign” s2.save() By default, indexing order is ascending. Order may be specified by prepending ‘+’ for ascending or ‘-‘ for descending order. To create compound index, use a tuple of field names, optionally having + or – symbol attached to indicate sort order. In the following example, student document class contains definition of compound index on name and course (note – symbol prefixed to course field which means index is built namewise ascending and coursewise descending order. from mongoengine import * con=connect(”mydata”) class student(Document): name=StringField(required=True) course=StringField() meta = {”indexes”:[(”name”,”-course”)]} s1=student() s1.name=”Avinash” s1.course=”DataScience” s1.save() s2=student() s2.name=”Anita” s2.course=”WebDesign” s2.save() MongoDB Compass will show indexes as below − Value of ‘indexes’ may be a dictionary of various options as below − fields The fields to index. cls If allow_inheritance is turned on, you can configure whether the index should have the _cls field added automatically. sparse Whether the index should be sparse. unique Whether the index should be unique. expireAfterSeconds automatically expire data from a collection by setting the time in seconds name Allows you to specify a name for the index collation Allows to create case insensitive indexes Following example creates index on name field that expires after 3600 seconds. from mongoengine import * con=connect(”mydata”) class student(Document): name=StringField(required=True) course=StringField() meta = {”indexes”:[{ ”fields”: [”name”], ”expireAfterSeconds”: 3600 } ] } To specify text index, prefix field name with ‘$’ sign and for hashed index, use ‘#’ as prefix. Indexes so specified are created automatically as documents are added in the collection. To disable automatic creation, set ‘auto_create_index’ to False in meta attribute. We have list_indexes() method with Document class that displays list of available indexes. print (student.list_indexes()) [[(”name”, 1)], [(”_id”, 1)]] To create index on a field not in the meta dictionary, use create_index() method. The following code will create index on course field − class student(Document): name=StringField(required=True) course=StringField() meta = {”indexes”:[{ ”fields”: [”name”], ”expireAfterSeconds”: 3600 } ]} student.create_index([”course”]) Print Page Previous Next Advertisements ”;

MongoEngine – Useful Resources ”; Previous Next The following resources contain additional information on MongoEngine. Please use them to get more in-depth knowledge on this. Python Programming Certification 2024 Most Popular  9 Courses     1 eBooks Tutorialspoint More Detail Artificial Intelligence and Machine Learning Certification 2024 Most Popular  7 Courses     1 eBooks Tutorialspoint More Detail Java Certification 2024 Best Seller  7 Courses     1 eBooks Tutorialspoint More Detail Print Page Previous Next Advertisements ”;

MongoEngine – Dynamic Schema ”; Previous Next One of the advantages of MongoDB database is that it supports dynamic schema. To create a class that supports dynamic schema, subclass it from DynamicDocument base class. Following is the Student class with dynamic schema − >>> class student(DynamicDocument): … name=StringField() The first step is to add first Document as before. >>> s1=student() >>> s1.name=”Tara” >>> connect(”mydb”) >>> s1.save() Now add another attribute to second document and save. >>> s2=student() >>> setattr(s2,”age”,20) >>> s2.name=”Lara” >>> s2.save() In the database, student collection will show two documents with dynamic schema. The meta dictionary of document class can use a Capped Collection by specifying max_documents and max_size. max_documents − The maximum number of documents that is allowed to be stored in the collection. max_size − The maximum size of the collection in bytes. max_size is rounded up to the next multiple of 256 by MongoDB internally and mongoengine before. If max_size is not specified and max_documents is, max_size defaults to 10485760 bytes (10MB). Other parameters of Document class are listed below − objects A QuerySet object that is created lazily on access. cascade_save() Recursively save any references and generic references on the document. clean() Hook for doing document level data cleaning before validation is run. create_index() Creates the given indexes if required. drop_collection() Drops the entire collection associated with this Document type from the database. from_json() Converts json data to a Document instance. modify() Perform an atomic update of the document in the database and reload the document object using updated version. pk Get the primary key. save() Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created. Returns the saved object instance. delete() Delete current document from database. insert() Performs bulk insert operation. Print Page Previous Next Advertisements ”;

MongoEngine – Aggregation ”; Previous Next The term ‘aggregation’ is used for the operation that processes data and returns computed result. Finding sum, count and average on one or more fields of documents in a collection can be called as aggregation functions. MongoEngine provides aggregate() function that encapsulates PyMongo’s aggregation framework. Aggregation operation uses a collection as input and returns one or more documents as a result. MongoDB uses concept of data processing pipelines. A pipeline can have multiple stages. Basic stage provides that provide filter and operate like queries. Others provide tools for grouping and/or sorting by one or more fields, string concatenation tasks, array aggregation tools, etc. Following stages are defined in MongoDB pipeline creation − Name Description $project Reshapes each document in the stream, by adding new fields or removing existing fields. $match Filters the document stream to allow only matching documents to pass unmodified into the next stage. $match uses standard MongoDB queries. $redact Reshapes each document by restricting the content for each document based on information stored in the documents themselves. $limit Limits documents to be passed unmodified to the pipeline $skip Skips the first n documents and passes the remaining documents unmodified to the pipeline. $group Groups input documents by a given identifier expression and applies the accumulator expressions to each group. The output documents only contain the identifier field and accumulated fields. $sort Reorders the document stream by a specified sort key. $out Writes the resulting documents of the aggregation pipeline to a collection. Aggregation expressions use field path to access fields in the input documents. To specify a field path, use a string that prefixes with a dollar sign $$$ the field name. Expression can use one or more Boolean operators ($and, $or, $not) and comparison operators ($eq, $gt, $lt, $gte, $lte and $ne). Following arithmetic expressions are also used for aggregation − $add Adds numbers to return the sum. Accepts any number of argument expressions $subtract Returns the result of subtracting the second value from the first $multiply Multiplies numbers to return the product. Accepts any number of argument expressions $divide Returns the result of dividing the first number by the second. Accepts two argument expressions $mod Returns the remainder of the first number divided by the second. Accepts two argument expressions Following string expression can also be used in aggregation − $concat Concatenates any number of strings $substr Returns a substring of a string, starting at a specified index position up to a specified length $toLower Converts a string to lowercase. Accepts a single argument expression $toUpper Converts a string to uppercase. Accepts a single argument expression $strcasecmp Performs string comparison and returns 0 if two strings are equivalent, 1 if first is greater than second, and -1 if first string is less than second To demonstrate how aggregate() function works in MongoEngine, let us first define a Document class called orders. from mongoengine import * con=connect(”mydata”) class orders(Document): custID = StringField() amount= IntField() status = StringField() We then add following documents in orders collection − _id custID amount status ObjectId(“5eba52d975fa1e26d4ec01d0”) A123 500 A ObjectId(“5eba536775fa1e26d4ec01d1”) A123 250 A ObjectId(“5eba53b575fa1e26d4ec01d2”) B212 200 D ObjectId(“5eba540e75fa1e26d4ec01d3”) B212 400 A The aggregate() function is to be used to find sum of amount field for each custID only when status equals ‘A’. Accordingly, the pipeline is constructed as follows. First stage in pipeline uses $match to filter documents with status=’A’. Second stage uses $group identifier to group documents on CustID and performs sum of amount. pipeline = [ {“$match” : {“status” : “A”}}, {“$group”: {“_id”: “$custID”, “total”: {“$sum”: “$amount”}}} ] This pipeline is now used as argument to aggregate() function. docs = orders.objects().aggregate(pipeline) We can iterate over the document cursor with a for loop. The complete code is given below − from mongoengine import * con=connect(”mydata”) class orders(Document): custID = StringField() amount= IntField() status = StringField() pipeline = [ {“$match” : {“status” : “A”}}, {“$group”: {“_id”: “$custID”, “total”: {“$sum”: “$amount”}}} ] docs = orders.objects().aggregate(pipeline) for doc in docs: print (x) For the given data, the following output is generated − {”_id”: ”B212”, ”total”: 400} {”_id”: ”A123”, ”total”: 750} Print Page Previous Next Advertisements ”;

Discuss MongoEngine ”; Previous Next MongoEngine is a Python library that acts as an Object Document Mapper with MongoDB, a NOSQL database. It is similar to SQLAlchemy, which is the Object Relation Mapper (ORM) for SQL based databases. Print Page Previous Next Advertisements ”;

MongoEngine – Extensions ”; Previous Next MongoEngine integrates beautifully with the following libraries − marshmallow_mongoengine marshmallow is an ORM/ODM/framework independent serialization/deserialization library for converting complex datatypes, such as objects, to and from native Python datatypes. Using this extension of MongoEngine, we can easily perform serialize/deserialize operations. First, create a Document class as usual as follows − import mongoengine as me class Book(me.Document): title = me.StringField() Then generate marshmallow schema with the code below − from marshmallow_mongoengine import ModelSchema class BookSchema(ModelSchema): class Meta: model = Book b_s = BookSchema() Save a document using the code: book = Book(title=”MongoEngine Book”).save() And perform serialization/deserialization using dump(0 and load() using the code below − data = b_s.dump(book).data b_s.load(data).data Flask-MongoEngine This is a Flask extension that provides integration with MongoEngine. Connection management of MongoDB database for your app is handled easily by this library. You can also use WTForms as model forms for your models. After installation of flask-mongoengine package, initialize flask app with the following settings − from flask import Flask from flask_mongoengine import MongoEngine app = Flask(__name__) app.config[”MONGODB_SETTINGS”] = { ”db”: ”mydata”, ”host”: ”localhost”, ”port”:27017 } db = MongoEngine(app) Then define a Document sub class using the below code − class book(me.Document): name=me.StringField(required=True) Declare an object of above class and call save() method when a particular route is visited. @app.route(”/”) def index(): b1=book(name=”Introduction to MongoEngine”) b1.save() return ”success” extras-mongoengine This extension contains additional Field Types and any other wizardry. Eve-MongoEngine Eve is an open source Python REST API framework designed for human beings. It allows to effortlessly build and deploy highly customizable, fully featured RESTful Web Services. Eve is powered by Flask and Cerberus and it offers native support for MongoDB data stores. Eve-MongoEngine provides MongoEngine integration with Eve. Install and import the extension using the code below − import mongoengine from eve import Eve from eve_mongoengine import EveMongoengine Configure the settings and initialize the Eve instance. my_settings = { ”MONGO_HOST”: ”localhost”, ”MONGO_PORT”: 27017, ”MONGO_DBNAME”: ”eve_db” app = Eve(settings=my_settings) # init extension ext = EveMongoengine(app) Define a Document class as shown below − class Person(mongoengine.Document): name = mongoengine.StringField() age = mongoengine.IntField() Add the model and run the application, finally using the below code − ext.add_model(Person) app.run() Django-MongoEngine This extension aims to integrate MongoEngine with Django API, a very popular Python web development framework. This project is still under development. Print Page Previous Next Advertisements ”;

MongoEngine – Add/Delete Document ”; Previous Next We have already used save() method of Document class to add a document in the collection. The save() method can be further customized with the help of following arguments − force_insert Default is False, if set to True doesn’t allow updates of existing documents. validate validates the document; set to False to skip. clean call the document clean method, validate argument should be True. write_concern will be used as options for the resultant getLastError command. For example, save(…, write_concern={w: 2, fsync: True}, …) will wait until at least two servers have recorded the write and will force an fsync on the primary server. cascade Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__. cascade_kwargs optional keyword arguments to be passed throw to cascading saves. Equivalent to cascade=True. _refs A list of processed references used in cascading saves save_condition only perform save if matching record in db satisfies condition(s). Raises OperationError if the conditions are not satisfied signal_kwargs kwargs dictionary to be passed to the signal calls. You can set cleaning rules for validation of documents before calling save(). By providing a custom clean() method, you can do any pre validation/data cleaning. class MyDocument(Document): … … def clean(self): if <condition>==True: msg = ”error message.” raise ValidationError(msg) Note that Cleaning is only called if validation is turned on and when calling save(). Document class also has insert() method to perform bulk insert. It has following parameters − doc_or_docs A document or list of documents to be inserted load_bulk If True, returns the list of document instances write_concern Extra keyword arguments are passed down to insert() which will be used as options for the resultant getLastError command. signal_kwargs (optional) kwargs dictionary to be passed to the signal calls If document contains any ReferenceField objects, then by default the save() method will not save any changes to those objects. If you want all references to be saved also, noting each save is a separate query, then passing cascade as True to the save method will cascade any saves. Deleting a document from its collection is very easy, by calling delete() method. Remember that it will only take effect if the document has been previously saved. The delete() method has following arguments − signal_kwargs (optional) kwargs dictionary to be passed to the signal calls. write_concern Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. To delete entire collection from database use drop_collecction() method. It drops the entire collection associated with this Document type from the database. The method raises OperationError if the document has no collection set (i.g. if it is abstract). The modify() method in document class performs atomic update of the document in the database and reloads its updated version. It returns True if the document has been updated or False if the document in the database does not match the query. Note that all unsaved changes that have been made to the document are rejected if the method returns True. Parameters query The update will be performed only if the document in the database matches the query update Django-style update keyword arguments Print Page Previous Next Advertisements ”;

MongoEngine – Installation ”; Previous Next To use MongoEngine, you need to have already installed MongoDB and MongoDB server should be running as described earlier. Easiest way to install MongoEngine is by using PIP installer. pip install mongoengine If your Python installation does not have Setuptools installed, you will have to download MongoEngine from https://github.com/MongoEngine/mongoengine and run the following command − python setup.py install MongoEngine has the following dependencies − pymongo>=3.4 six>=1.10.0 dateutil>=2.1.0 pillow>=2.0.0 To verify the correct installation, run import command and check version as follows − >>> import mongoengine >>> mongoengine.__version__ ”0.19.1” Print Page Previous Next Advertisements ”;