Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.Django lets the developers build deep, dynamic, interesting sites in an extremely short time. The framework is designed to let the developer focus on the fun, interesting parts of the job while easing the pain of the repetitive bits. In doing so, it provides high-level abstractions of common Web development patterns, shortcuts for frequent programming tasks, and clear conventions on how to solve problems. At the same time, Django tries to stay out of the way, letting the developer work outside the scope of the framework as needed.
Django is usually called an MVC framework, and justifiably so. It is very heavily influenced by classical MVC and it’s even possible to argue that Django improves the architectural pattern. In Django, the three core layers are the Model, the View, and the Template. A key advantage of such an approach is that components are loosely coupled.
Having come to know of the interesting features of Django, we recently worked on a pilot product development project, called Xaffo, which had demanding needs for a web framework. Xaffo, a cloud-based social media monitoring tool,allows users to analyze the popularity of their brands among leading social networks.
Why we decided that Xaffo needs a web framework like Django
Xaffo basically deals with large chunks of social media analysis data shuttled between different web services, with huge task lists in the background that requires higher performance and scalability.The Xaffo prototype was initially built on Google App Engine and therefore in Python,and keeping the python code was also an essential part in our decision-making.
Meeting performance expectations when handling large sets of data was the challenge where features of Django became relevant and interesting. Handling large number of tasks in the background, Celery (explained below) came in handy to support Django to dynamically add or remove workers to handle the tasks.
On account of handling large sets of data, Django makes a better pair with MongoDB, which brings out easy database connectivity with high performance.
Xaffo is hosted behind nginx web server with uWSGI for high performance and static and dynamic content serving.The combination of this hosting environment and Django proved optimal. Xaffo is hosted on Amazon EC2 achieving the scalability where Django becomes the perfect web framework to match all these components.
Django made it easy to achieve tedious tasks that Xaffo demanded, with its appealing features such as,
- Object Relational Mapping
- Template System
- URL Resolver
- Forms
- Admin Site
In addition to the aforementioned features of Django, Xaffo relies largely on Django’s modularity.
We used several 3rd party Django packages, such as:
- Celery
- MongoEngine
- Flower
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.The execution units, called tasks, are executed concurrently on a single or multiple worker servers using multiprocessing, Eventlet, or gevent. Tasks can be executed asynchronously (in the background) or synchronously (wait until ready).Celery is used in production systems to process millions of tasks a day.
Celery is used with Xaffo to manage the periodic tasks that are executed to fetch and calculate data gathered from various social network APIs. RabbitMQ is used as the message broker.
MongoEngine is an object document mapper for Django, which is very similar to Django’s own ORM. This enables developers who are already familiar with the Django ORM to interact with MongoDB without having to go through a whole new API documentation.
Celery Flower is a tool used with Xaffo to monitor the periodic tasks executed by Celery. This tool provides a web interface with information such as task progress, graphs and statistics.
To wind up, Xaffo was a successful project with Django, proving to us that Django web framework is highly suitable for projects that require higher performance,scalability, and high load of backend processing with large chunks of data.Its also a RAD framework in this context, and facilitates meeting tough deadlines.
Django takes away the tedious tasks of the development environment and makes it easier to build better web apps more quickly with less code. We at Calcey recommend it wholeheartedly!
More info:
Django – http://www.djangoproject.com
MongoDB – http://mongoengine.org/
Celery –http://celeryproject.org/
Flower – http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monito
RabbitMQ- http://www.rabbitmq.com/features.html