Trying JSON in Django and PostgreSQL (and compare with MongoDB)

Trying JSON in Django and PostgreSQL

New JSONField will be added in Django 1.9, it can be used with PostgreSQL >= 9.4. Let's try to work with it and find out, in what situations it can be useful.

Currently django 1.9 alpha is available, final version is scheduled on December 2015. Alpha can be installed with pip:

pip install --pre django

Now imagine that we have an e-commerce site, where we offer products of different types. For example ...

Posted on 2015-09-30

OAuth and django rest framework

OAuth and django rest framework

This is a well known topic, but i can't find the existing solution that will fully satisfy me. So i write it by myself :).

Assume we have a "single page" web site, that is talking with backend via REST API. Client side can be written with ember, angularjs or some like this. Backend - django rest framework (DRF). We've got a task - add social login (OAuth protocol).

How it will look like in case ...

Posted on 2015-07-18

Tornado and pgettext

Tornado and pgettext

Recently (26.05.2015) new tornado 4.2 was released. It contains different updates, the most valuable i suppose are modules tornado.locks and tornado.queues. They migrated from package Toro, look detailed explanation in Jesse Jiryu Davis post.

Here i want to tell about another helpful function, that was added with my help - pgettext.

It can be useful, when you are creating the translation for ambiguous strings. Let's say we have word "bat ...

Posted on 2015-06-05

OAuth step by step

OAuth step by step

OAuth protocol has two versions: 1.0 and 2.0.

Most of services today use version 2.0, i suppose because it is easier to implement. Also, 2.0 can be realized in standalone applications (those, that don't have a server).

To understand the protocols very useful to have a look at their realisation. Here i'll show several scripts that talk to OAuth providers of different versions. Scripts will implement client application ...

Posted on 2015-05-16

Python tips & tricks

Base python tips & tricks

Recently i've read the book Learning Python, 5th Edition by Mark Lutz. Here is a list of most interesting insights for me.

  • set generation:

    {x for x in [1,2]}
    set(x for x in [1,2])
    assert set(x for x in [1,2]) == {x for x in [1,2]}
  • dict generation:

    {x:x**2 for x in [1,2]}
    dict((x, x**2) for x in [1,2])
    assert {x:x**2 ...

Posted on 2015-04-30

Listen wifi with wireshark

Listen wifi with wireshark

I always knew, that it is possible to catch wifi network packets. But haven't done it in practise (i was analysing network packets, but not in HTTP protocol). So i decided to do it, as this is interesting and useful. Such experience help to understand TCP-IP and HTTP protocols and also to pay more attention for web security.

We'll spy the network traffic with Wireshark program. There are a lot of tools for ...

Posted on 2015-04-17

Debug SQL in django test

Debug SQL in django test

In django tests we can measure number of sql queries:

def test_home(self):
    with self.assertNumQueries(1):
        response = self.client.get('/')
    self.assertEqual(response.status_code, 200)

If code in context of assertNumQueries will make other number of DB attempts than expected (here 1), test will throw error. But when test fails it is sometimes hard to understand, what unexpected query was made. To debug such case very useful to log SQL expressions to console. Below ...

Posted on 2015-03-04

Tornado i18n and l10n

Tornado i18n and i10n

Let's talk about i18n, i10n and tornado implementation. This post have a lot of text, but i wanted to describe many things that i faced during realization of i18n in tornado project. The step by step instruction is placed in the second part of this article.

Common definitions


i18n - shorthand from internationalization. It mean the process of multiple languages support in application. It is not the translation itself, but technical part of the ...

Posted on 2015-01-31

Timestamp and ObjectId in mongoDB

Timestamp and ObjectId from mongoDB

Every record in mongoDB has field _id, that must be unique inside collection. By default type of this field is ObjectId, and it is assigned automatically if field is not set.

Lets look at ObjectId more carefully.
It is 12 bytes that includes:

  • a 4-byte value representing the seconds since the Unix epoch,
  • a 3-byte machine identifier,
  • a 2-byte process id, and
  • a 3-byte counter, starting with a random value.

As we see, first 4 ...

Posted on 2015-01-18

Set url for Tornado handlers

Set url for Tornado handlers

To set url for tornado handlers we can pass list of tuples (url regex, handler) into application initialisation:

application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/some/path/page/(?P<pk>[0-9]+)$", PageHandler),

But it is more convenient to use wrapper tornado.web.url, that allows to assign meaningful names for paths (similar to django url).

Nevertheless, in a couple of production projects that i had to work with, this wrapper wasn't used. Also in ...

Posted on 2014-12-15

Django celery setup

Django celery setup

To enable celery in new django project i often look in previous ones to refresh in my memory some steps: what settings should be specified, how to launch, how to stop and so on.

Here i want to combine all together in one place.

What we must get as a result

  1. Add delayed task executing in django project, controlled by celery, to not load current django process. Examples of such tasks: email sending, working with ...

Posted on 2014-11-02

Nested SQL queries in Django

Nested SQL queries in Django

Did you know, that Django ORM can do nested SQL queries? Shame on me, but i've found it not so long ago.

So, lets say we have such models of Nursery and Pet:

class Nursery(models.Model):
    title = models.CharField(max_length=50)

class Pet(models.Model):
    name = models.CharField(max_length=50)
    nursery = models.ForeignKey(Nursery, related_name='pets')

We need to get all pets, that are placed in given nurseries. For example, in nurseries with ...

Posted on 2014-09-09

Free t-shirt from New Relic

Free shirt from New Relic

New Relic - service for site monitoring. It shows statistic of your application, where it spend most of time, how often it access the database and many other information. For description of this cool service i need to write a separate post. Here i want to tell about another thing - how i've got a free t-shirt from New Relic.

Actually, it is needed just to register and setup New Relic on a working web site ...

Posted on 2014-05-29

Remote url for localhost server

Remote url for localhost server

There is a nice tool called ngrok. It allows to bind the URL for your localhost server!

For example, you've launched a django development server on you computer:

python runserver

and this project can be accessed by remote URL.

What for?

At least i can imagine such tasks:

  • demonstrate project to customer
  • check your site integration with payment system. For example paypal, where for receiving IPN messages you need a working site ...

Posted on 2014-05-09

Send email in django project with mandrill service

Send email in django project with mandrill service

To send email messages from server we can just use SMTP protocol. But there is another way - special email services. I'll describe one of them here,


  • Detailed statistic of sent emails. How many were sent, to whom, when. How many were opened, what links were clicked.
  • Message templates. They can be modified through mandrill service, no need to create anything in django admin. It is possible to use variables in ...

Posted on 2014-03-18

What you should know about mongodb indexes

What it is needed to know about mongodb indexes

Recently i've completed course "M101P: MongoDB for Developers" (periodically repeats, next starts at April). During this course i've found to myself interesting features of mongodb.

1. Index selection.

Suppose we have collection with such document format:

{ "_id" : ..., "a" : 81810, "b" : 97482, "c" : 44288 }
{ "_id" : ..., "a" : 11734, "b" : 27893, "c" : 19485 }
// and so on.

Total 99999 documents. Collection has indexes:{a: 1, b: 1, c: 1}){c: -1 ...

Posted on 2014-03-07

Async Bitcoin RPC client

To work with Bitcoin RPC from python there is a library Python-BitcoinRPC.

But recently i need to call API from tornado application. Mentioned lib works in synchronous, i.e. blocking mode. For tornado it will be much better to use asynchronous version. Tried to search for existing solution, but can't find it. So i create my async fork, that use tornado's AsyncHTTPClient:

Example (print current number of blocks ...

Posted on 2014-01-30

Tornado web application example

Tornado web application example

Tornado - async web framework for python. I'll cover shortly pros and cons about tornado and introduce typical web project, that is built on top of it.

By describing pros and cons i mean my own point of view in compare with django.

Tornado pros

1. Asynchronous.

In tornado core there is a infinite loop called "ioloop", that listen for events. All that happens in one single thread. For example, somebody wants to fetch /home ...

Posted on 2013-12-29

Django project: from beginning to working server

In this post i'll describe my experience of deploying django project to VPS-hosting.

Steps of deploying and server setup:

  1. Creation of django project
  2. Deploy setup
  3. Server setup
  4. Deploy

Creation of django project

Assume, that project is called myproject.

First, create it on local machine. We'll need pip and virtualenv (if not installed yet).

Create virtual environment:

virtualenv venv-myproject

Acitvate it:

# on *nix system
source venv-myproject/bin/activate

REM on windows
venv\Scripts\activate ...

Posted on 2013-11-13

Django logging settings

Let's look at default django logging settings and try to make them more useful.

Here what we have in after command startproject project_name (django 1.5):

# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See for
# more details on how ...

Posted on 2013-09-26